LibrarianSync: Kindle collections builder

[royr719]

So, is this compatible with jailbreak PW3 5.8.1 firmware?

[eschwartz]

LibrarianSync manipulates the database directly (unlike, say, Collections Manager which hooks into the java-based collections UI) so it is proof against anything but an incompatible database schema change.

Which there is no reason for, especially in a firmware upgrade to an existing device (since that would mess up existing collections).


tl;dr
It should work just fine, yes.

[royr719]

Originally Posted by eschwartz

LibrarianSync manipulates the database directly (unlike, say, Collections Manager which hooks into the java-based collections UI) so it is proof against anything but an incompatible database schema change.

Which there is no reason for, especially in a firmware upgrade to an existing device (since that would mess up existing collections).


tl;dr
It should work just fine, yes.

Thanks!

[homeceo43]

I just wanted to say thank you to all you wonderful, smart, and very talented people for offering this tool!

I have a KOA 5.8.1 and I use use KOReader to read epubs and PDFs. I use Calibre and already have a #bookshelf column that I use when sending books to my Kobo H20. In Calibre I went ahead and also added a bookshelf to each of my series books (a lot of them) and have modified my send-to-device template for the Kindle to save each book in its "bookshelf" folder ({#bookshelf}/{title} - {authors}) in the documents folder on the Kindle. I'm sending only epubs and PDFs to my Kindle. I found that it is much easier to locate books in the KOReader file manager with the books organized by "bookshelf".

But, I really prefer using KPVBooklet to open the epubs and PDFs directly from the Kindle home menu instead of through KUAL -> KOReader. With about 1200 books in about 200 "bookshelf" folders, I am not manually creating 200 collections. I would just do without .... until I came across this tool.

I installed Librarian Sync and selected "Rebuild collections (from folders)" and it ran and created 1 collection and it was created from the only folder I had put PDFs. I have about 1200 epubs/PDFs mix and none of the epubs were put in collections.

So....did a little digging and found that I could edit the kindle_contents.py file that is in the librariansync folder. I added .epub to the spot where supported extensions are listed.

I then ran Librarian Sync "Rebuild collections (from folders)" again.... and what do you know! It took only 40 seconds to create some 200 collections. All of my epubs and PDFs are now in their appropriate collection.

Just awesome!

[knc1]

To do that many entries in that amount of time seems to indicate that the KOA devices are over-powered brutes on the inside.
And that is writing to a database stored on flash (which is relatively slow).

PS: There has been evidence found that Lab126 has been playing with Android on the KOA.
No idea if there is a model in the future or if they where just evaluating that setup.
But that might explain the fast hardware on a simple e-book reader.

[NiLuJe]

@homeceo43: Thanks for actually testing that and confirming that it works, the issue came up recently, and that very solution was proposed, but I hadn't had time to test it myself .

I'll add that in the next batch of fixes of my next PR so that everyone can benefit .

[homeceo43]

Originally Posted by knc1

To do that many entries in that amount of time seems to indicate that the KOA devices are over-powered brutes on the inside.
And that is writing to a database stored on flash (which is relatively slow).

PS: There has been evidence found that Lab126 has been playing with Android on the KOA.
No idea if there is a model in the future or if they where just evaluating that setup.
But that might explain the fast hardware on a simple e-book reader.

knc1,

I was so excited to see the collections appear, I think I saw that number wrong.

Did an experiment.....

I deleted the collections (through Librarian Sync) and removed all the books off both KOA (5.8.1) and KV (5.8.1).

Then I added the same 1,145 books (about 50 are PDF, the rest are epubs) back to each device.

Then I ran Librarian Sync "rebuild collections (from folders)" on each device.

KOA -> 25 seconds to build 233 collections
KV - > 26 seconds to build 233 collections

And all appears to be in order.

Too cool.

[homeceo43]

Originally Posted by NiLuJe

@homeceo43: Thanks for actually testing that and confirming that it works, the issue came up recently, and that very solution was proposed, but I hadn't had time to test it myself .

I'll add that in the next batch of fixes of my next PR so that everyone can benefit .

You are very welcome!

[eschwartz]

@knc1,

Collections have always been much much faster to create when using LS vs the builtin collections UI or even the CM kindlet.

[picree]

Tried installing LS from .bin file in mrpackages folder using MrPI via the menu entry in the booklet version of KUAL on a Kindle Oasis. The MrPI log file:

[2016-08-15 @ 21:50:57 -0400] :: [MRPI r13359] - Beginning the processing of package 'Update_librariansync_1.10_kindle5.bin' (librariansync 1.10 kindle5) . . .


Package 'Update_librariansync_1.10_kindle5.bin' (librariansync 1.10 kindle5) is not targeting your device [0GC vs. 11 0F 10 24 1B 1C 1D 1F 20 D4 5A D5 D6 D7 D8 F2 17 60 62 5F C6 13 54 53], skipping . . .

Not sure if LS or KUAL needs to have the Oasis added as a supported device...also, the Kindle Collections Manager in Calibre picked up the Oasis as "unknown". I changed that to Voyager so I could transfer the collections json file. New territory I guess...

[NiLuJe]

@picree: The package in the first post hasn't been updated in a while.

The code on github is up-to-date, though, so just clone/tarball that and install it manually. It's just a KUAL extension, after all, you just need to make sure it's in the right place .

As for the Calibre plugin, current versions should detect the Oasis properly. Although autodetection is possibly broken on Windows 10 right now. I haven't had time to look into it yet, and probably won't for a while, since Win10 AU is being a whiny bitch on my desktop, and magnificently BSODs without logging...

[picree]

OK-so that was easy peasy. Thanks. I learned something today

agreed: Win10 is a whiny (and chatty) little thing...

[aholmes5]

Made a bunch of collections on calibre and used Kindle Collections to sort them on my Paperwhite 3. Used librariansync to create the collections and boom worked great.

Then I decided I wanted to rename my collections from (Series) to (Author - Series). So I deleted all collections in Librariansync - confirmed all were gone. Went back to Calibre worked the new magic with a custom column and resynced to the PW3. When I ran a rebuild collections (calibre), I now get all the new collections (Author - Series) and all the old collections (Series) that I deleted. I was in airplane mode when I did all of this if that matters.

Any idea whats going on?

[knc1]

It just did as documented for what you told it to do.

If you want the existing collection(s) REPLACED use the 'build from directory structure' method.

IIRC - that is the only CM method that does a full replace rather than just an update (addition).

Read the CM document(s) / readme to be certain.

[NiLuJe]

If you've setup the plugin to do incremental updates, you'll probably have to do an export after wiping everything from LS.

That should let things start from scratch.

Alternatively, just disable the incremental update mode in the plugin for that first sync.

Or clear the json backups done by the plugin to handle incremental mode.

(You get the idea: incremental mode is probably trying to be smart, and that backfired ^^).