Clarify Size & Date fields in library and device views?

[unboggling]

I thought I'd ask, to check my assumptions:

Assumption 1: Size shows in both library view and device view, with the same lookup name {size}, so I assume the size data on device view is populated from the same field as in library view, rather than from the filesize on the device at connect-device. So if the size of file was changed on the device, calibre wouldn't know.

Assumption 2: Date {date} on library view (for when book is added) is different lookup name than Date {timestamp} on device view, so Date {timestamp} indicates when format was added to device, populated from data on the device at connect-device.

Are those assumptions true? Did I get any pieces wrong?

[davidfor]

For the Kobo ereaders, the size displayed on the device list, should be the size on the device. You have the dates you correct. This might be different for other devices.

[eschwartz]

I am pretty sure the metadata for device is handled independently. If the size depended on the size in calibre, any books NOT in your library wouldn't show any size in calibre at all.

The metadata is set in metadata.calibre, either when you send the book, or when the device is scanned for books not in the metadata.calibre, and once there, calibre doesn't assume a need to rescan for a changing size.

So calibre wouldn't know if the filesize changed, but only because it is populated from the book, at the time of indexing.

The date is likewise set at the time of indexing, and depends on when calibre sent it, or else when indexing found boks, the actual timestamp of the file.

[aleyx]

Originally Posted by eschwartz

The metadata is set in metadata.calibre, either when you send the book, or when the device is scanned for books not in the metadata.calibre, and once there, calibre doesn't assume a need to rescan for a changing size.

So calibre wouldn't know if the filesize changed, but only because it is populated from the book, at the time of indexing

Yet Calibre knows whether the book exists in the library (in Device view) or on the Device (in Library view). This implies a common ID field between the two DB. At least for those books Calibre sent, obviously.

So another question is: are there ways to add more comparisons between the two DB? For example, given a book sent by Calibre on the device, is there a way to know whether it is up-to-date, by adding a calculated column in the Library view?

Periodicals and the FFDL plugin (and maybe Quality Check) would greatly benefit from this kind of feature. Actually, if the necessary hooks already exist and nobody else's on it, I'd be interested in coding a plugin...

[kovidgoyal]

Book matching on the device is a many layered thing. First for books that calibre sends, it records the book uuid in metadata.calibre. If that does not exists, then metadata based matching is performed. You can run calibre ind ebug mode in which case the in library column shows how the matching was performed. You can also manually tell calibre a book on the device matches a book in the library by right clicking ont he bookint he device view.

[unboggling]

@davidfor & eschwartz, thanks, that clarified. @aleyx, thanks for commenting.

@kovid, thanks, that clarified. I'd never noticed in debug mode the uuid in the field inLibrary. And didn't understand what the menu command "match book" meant. But now it makes more sense to me.

@anyone, please evaluate these statements, are they true?:

In library view, Size (size) represents filesize of the largest format at the time book was added to library. Not editable. Adding or removing formats within calibre automatically recalculates it. But if user edits the format outside of calibre, size may not be accurate until the next time formats are added or removed in calibre or user resizes through Quality Check plugin. Usually accurate, but accuracy dependent on user.

In device view, Size (size) represents filesize of the format on device, whether sent there by calibre or not. Not editable by user. User could edit format on device (remote possibility). If calibre originally sent format to device, calibre matches by uuid and uses the filesize calibre remembers from before, not knowing about any device-side change of size; in that case, probably accurate unless user edited format outside of calibre. If calibre did not originally send format to device, filesize comes from the device; in that case, highly accurate. Accuracy depends on various factors, so interpret as approximation.

In library view, Date (date) represents when book was added to library. Accurate at that point. Editable by user. Subsequent accuracy depends on user.

In device view, Date (timestamp) represents when book was added to device, whether sent there by calibre or not. Not editable by user. Highly accurate.

(Not knowing exactly which fields are involved with the match book command in device view, I ignored those possibilities above. And I want to avoid parsing various conditional qualifiers out to infinite minute possibilities.)

[unboggling]

Originally Posted by kovidgoyal

Book matching on the device is a many layered thing. First for books that calibre sends, it records the book uuid in metadata.calibre. If that does not exists, then metadata based matching is performed. You can run calibre ind ebug mode in which case the in library column shows how the matching was performed. You can also manually tell calibre a book on the device matches a book in the library by right clicking ont he bookint he device view.

I completed missed that "metadata-based matching" step earlier. So, if uuids differ, then in metadata matching: if sizes are different, does calibre choose the device's size value or the library's size value?

[eschwartz]

I assume they always draw on the device size value, drawn from the actual file.

What point would there be in using the size in calibre?

[unboggling]

Originally Posted by eschwartz

I assume they always draw on the device size value, drawn from the actual file.

What point would there be in using the size in calibre?

I agree. I had originally assumed that too. But the goal is to check assumptions against reality and get rid of wrong assumptions. Assuming an assumption is really true often eventually gets me into trouble, if it's actually not true.

[kovidgoyal]

metadata matching is only performed on title and author, size plays no role. Since calibre updates metadata when sending to device, size is almost never going to match.

[unboggling]

Originally Posted by kovidgoyal

metadata matching is only performed on title and author, size plays no role. Since calibre updates metadata when sending to device, size is almost never going to match.

Ah. Thank you.

I'll just say in my metadata definitions that the size in Device View shows the size of the format on the device, to be interpreted as approximate.

Besides checking assumptions, I was curious and too lazy to go read the code through all calls and nooks and crannies, with limited understanding.

[unboggling]

Thank you davidfor, eschwartz, aleyx, and Kovid for explaining about Device View columns. I finally woke up late one night with lightbulb realization that all of the Device View columns are automatically calculated representations for those fields pertaining to books on device, presented after automatic calibre decision making per various contingencies. Collections being the only potential user-editable column in Device View, under certain conditions with some specific devices. Confused by the lookup names, I had been seeing the trees and not the forest.