Modification field on OPDS Feed

[pdurrant]

I'm using calibre server with justRead.app, which is mostly working great.

However, justRead.app uses the Modification field in the OPDS feed to tell whether a book file has been updated and needs to be re-downloaded.

Unfortunately the calibre server's OPDS Modification field is taken from the database entry Modification field, which is changed by ANY change to the metadata of the book, and sometimes even by purely display changes. E.g. Just changing the display formatting of a Progress column to include a % sign at the end of the number results in ALL books in calibre having an updated Modification date.

Downloading several thousand books again just because I've changed the way a column is displayed in calibre is not optimal.

It would be really nice if the OPDS feed provided some way for clients to tell whether the book file that would be downloaded has changed from the last time the client downloaded the book.

I don't know about the internal details of the OPDS format, but would it be possible, for instance, to add in a hash of the book that's available for download?

[kovidgoyal]

IIRC calibre OPDS acquisition feeds include fie length and mtime as attributes.
For example:

<link type="application/x-mobi8-ebook" href="/get/azw3/1356/test_library" rel="http://opds-spec.org/acquisition" length="3354704" mtime="2023-11-18T02:16:51+00:00"/>
    <link type="application/vnd.openxmlformats-officedocument.wordprocessingml.document" href="/get/docx/1356/test_library" rel="http://opds-spec.org/acquisition" length="3039862" mtime="2023-11-18T02:16:51+00:00"/>
    <link type="application/epub+zip" href="/get/epub/1356/test_library" rel="http://opds-spec.org/acquisition" length="2657477" mtime="2023-11-18T02:16:51+00:00"/>

[pdurrant]

I think I've misunderstood which mechanism justRead.app uses. It uses "calibre specific rest API in JSON format."

[petrjahoda]

Hi Kovid, justRead's developer here.

One correction to Paul: this isn't about accessing Calibre via OPDS, justRead connects through the Calibre Content Server API.

To explain what's happening: justRead reads each book's last_modified field from /ajax/books/{library} and compares it against the value stored from the previous sync. That comparison drives a per-book status that lets the user decide what to transfer:
- New: no stored last_modified on the justRead side yet
- Changed: the stored last_modified differs from the server's current value
- Unchanged: both values match (these are not displayed at all, no need to)

The Calibre action Paul described above rewrites last_modified on every book and from that point, every book's timestamp differs from what justRead stored, so they all get marked as Changed, even though their contents didn't actually change.

[DNSB]

I ran into a similar problem a while back and found my solution in the Job Spy plugin where the only function I still use is under GUI tools that are metadata related => Set last-modified date to date-added.

[kovidgoyal]

last_modified is not about format modification times, never has been nor ever will be. If you want format modification times to drive modification status you will need to actually load those and read them, not last_modified.

[kovidgoyal]

And to make it easier I have added mtimes for individual formats to book metadata https://github.com/kovidgoyal/calibr...19be0e5b878391

[petrjahoda]

Hi Kovid, thank you for it, so I will use mtime for this. I thought last_modified was for last modified time, by the name of the parameter.

And now I am curious: if last_modified is not last modified datetime, what datetime is it then? What is its purpose in the data?

[kovidgoyal]

Its the last time *any metadata* associated with the book was modified.

[petrjahoda]

Thank you very much, will use mtimes and last_modified as backup.