05-14-2017, 06:28 AM | #1 |
Grand Sorcerer
Posts: 24,907
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
Problem with Metadata smart_update
@Kovid:
I have had a problem with the collections displayed in the device list for some time. Occasionally, they wouldn't show. Unplugging the device and plugging it back in would fix it. Because of that, I haven't been bothered about it. But, I've had a couple of reports recently about problems with collections, including a report that when Metadata management was set to automatic, the collections weren't showing. So, I've had a better look at it. I found a problem in the KoboTouch driver that caused of the problems when the collection management was turned off or there were no collection columns. After fixing that, there were still issues. The problem is most obvious when there is no metadata.calibre file on the device. So, the second time the device is connected, everything is OK. The problem is that something is clearing the "device_collections" attribute between the "Get list of books on device" and "Send metadata to device" jobs. I tracked it down to where the metadata from the device is merged with that from the library. This is in gui2/device.py in method "update_book". That calls ebooks/metadata/books/base.py, Metadata:smartupdate. This is called with "replace_metadata=True". The problem is that smartupdate with that options sets some attributes to None if they are not in the source object. The list of attributes to be treated this way includes "device_collections". As the source object is coming from the database, "device_collections" won't be in the source object and hence will be set to None. The fix I have is to add "device_collections" to the list SC_FIELDS_COPY_NOT_NULL in ebooks/metadata/book/__init__.py. With that, all the cases I have tested, the collections are displayed correctly. Searching through the code, I don't think that this will cause a problem. I do not have any non-Kobo devices to test with, so, I can't prove it that way. And, of course, I might have missed something in the code. |
05-14-2017, 07:39 AM | #2 |
creator of calibre
Posts: 43,850
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
I'll take a look.
|
05-14-2017, 08:13 AM | #3 |
creator of calibre
Posts: 43,850
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Doesn't putting it in there mean that it will no longer be copied at all by smart_update(). That is likely to have rather far reaching consequences. A more localized fix would probably be to change the update() book method to preserve the contents of that field instead. Something like
Code:
dc = getattr(book, 'device_collections', None) book.smart_update() if not getattr(book, 'device_collections', None): book.device_collections = dc |
05-16-2017, 08:47 AM | #4 |
Grand Sorcerer
Posts: 24,907
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
That should work, but it feels clumsy.
The code in smart_update that does the actual copy is: Code:
if replace_metadata: # SPECIAL_FIELDS = frozenset(['lpath', 'size', 'comments', 'thumbnail']) for attr in SC_COPYABLE_FIELDS: setattr(self, attr, getattr(other, attr, 1.0 if attr == 'series_index' else None)) self.tags = other.tags self.cover_data = getattr(other, 'cover_data', NULL_VALUES['cover_data']) self.set_all_user_metadata(other.get_all_user_metadata(make_copy=True)) for x in SC_FIELDS_COPY_NOT_NULL: copy_not_none(self, other, x) if callable(getattr(other, 'get_identifiers', None)): self.set_identifiers(other.get_identifiers()) # language is handled below The current value of SC_FIELDS_COPY_NOT_NULL is: Code:
SC_FIELDS_COPY_NOT_NULL = frozenset(['lpath', 'size', 'comments', 'thumbnail']) |
05-16-2017, 11:06 AM | #5 |
creator of calibre
Posts: 43,850
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Ah ok, makes sense. I was mis-remembering what smart_update did.
Go ahead and send a PR, I'll merge it. |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Metadata problem | pookiebr | Library Management | 13 | 03-19-2013 10:14 AM |
Metadata problem | tm3 | Library Management | 2 | 11-14-2012 09:23 AM |
problem with metadata | FranSmith | Calibre | 17 | 01-23-2011 11:54 AM |
edit metadata problem | donschjr | Calibre | 3 | 09-30-2010 11:33 AM |
Problem with metadata in Connect | tiohn | Sony Reader | 3 | 11-13-2007 02:03 PM |