How to get device lpath for a book

[meme]

I'm looking at using the db to get my book information instead of the written metadata.calibre file (thanks to Chaley for the starting point) so that the device doesn't need to be disconnected/reconnected if columns are added/books deleted, etc.

I can get the data from the db for books on the connected Kindle device, but I can't get the lpath value - it always shows None.

db = self.gui.library_view.model().db
query = 'ondevice:True'
id_list = db.search_getting_ids(query, None)
for id in id_list:
m = db.get_metadata(id, index_is_id=True)
print m.lpath

I'd like to get the lpath value as it will be written into the metadata.calibre file on the Kindle - preferably without reading the file (which I do at the moment), or at least the path after 'documents'. If I just use db.title or path then I think I'll just get Calibre's version of the title/path - not the version that might be modified by a save template, etc.

[chaley]

There is no convenient way to get the lpath if the device is not connected. It is computed by the device driver.

If the device is connected, then there are two ways to get it.

1) from metadata.calibre. You already know how to do this. You get the relative lpath this way, which is what you say you want.

2) from the 'device view database'. This will give you the 'book' entry that metadata.calibre is built from. It contains both path and lpath.

To see how to do the second, look at the remove_matching_books_from_device function (gui2.actions.delete.py). It calls paths_for_db_ids (gui2.library.models.py) to get a list of tuples of books on the device matching a list of db_ids. The tuple is (row, book), where row is the device view gui row (not useful to you I think) and book is the metadata instance for the book.

The complexity you see comes from the fact that each memory location on the device has its own db, so all of them must be checked. The loop:

        for model,name in ((self.gui.memory_view.model(), _('Main memory')),
                           (self.gui.card_a_view.model(), _('Storage Card A')),
                           (self.gui.card_b_view.model(), _('Storage Card B'))):
            to_delete[name] = (model, model.paths_for_db_ids(ids))

does that by looping through all the possible memory locations. You probably don't need the name, and the kindle might not use all the memory locations.

Once you have the three (or fewer) lists, then you should create a map of db_id to book(s). The complete code will look something like:

  # using your id_list ....
  matching = []
  for model in [self.gui.memory_view.model(),
                    self.gui.card_a_view.model(),
                    self.gui.card_b_view.model()]:
    # append a list of books. Throw away the row value
    matching.append([tup[1] for tup in model.paths_for_db_ids(id_list)])

  db_id_map = defaultdict(set)
  for blist in matching: # blist == list of (row,book) tuples for the mem location
    for book in blist:
      db_id_map[book.application_id].add(book)

At this point you have a map of db_id to sets of matching books. You can extract whatever information you need, such as lpath.

[meme]

Great, I'll look into option 2. It may allow me to ignore the metadata.calibre file altogether, but I'll have to test.

[chaley]

Originally Posted by meme

Great, I'll look into option 2. It may allow me to ignore the metadata.calibre file altogether, but I'll have to test.

There is nothing in metadata.calibre that isn't in the device db booklist, with the exception that metadata.calibre can contain entries for books deleted during the current connection.

[meme]

Its looking good from testing. Why do you use defaultdict instead of just dict?

I have to test using the library_view versus the memory view for columns, etc - if a new column is created I'm not sure if the device view will have the most current list of columns for books.

[chaley]

Originally Posted by meme

Its looking good from testing. Why do you use defaultdict instead of just dict?

It is a convenience to avoid the need to test for existence of the key before adding to the set. If there is no item indexed by the key, then defaultdict creates one for you.

If you use a normal dict, then you must do something like:

  db_id_map = {}
  for blist in matching: # blist == list of (row,book) tuples for the mem location
    for book in blist:
      if book.application_id not in db_id_map: # defaultdict eliminates these 2 lines
        db_id_map[book.application_id] = set([])
      db_id_map[book.application_id].add(book)

I have to test using the library_view versus the memory view for columns, etc - if a new column is created I'm not sure if the device view will have the most current list of columns for books.

The device_view databases are copies of the information in metadata.calibre. They will not contain updated metadata. You should use the library_view for all metadata except lpath and path and, if you use it, device_collections.

[chaley]

BTW: if you switch to using the library_view for metadata, then you can probably drop the requirement that people use automatic metadata management. The only items you need from the device are path/lpath, and they don't change.

[meme]

Yes - that's exactly why I started looking at this and was the first thing I removed in the new code

Now I can also do per library customizations - to show only those custom fields for this library and save them per library name.

[chaley]

Originally Posted by meme

Now I can also do per library customizations - to show only those custom fields for this library and save them per library name.

Have you found the field_metadata class/instance? (self.gui.db.field_metadata, code in library.field_metadata.py)? It contains data descriptions of all the fields in the current library, custom and standard. You can get column datatypes, sort terms, column headings, the meta2 column the data is in, and all sorts of other things from it.

[meme]

No, hadn't seen that, but will look it over - it looks extensive. I've been able to get what I need from the functions in database2 at least so far. We'll see when I do more testing. Don't want to go too far down the rabbit hole too soon