Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Amazon Kindle > Kindle Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 03-04-2012, 10:43 PM   #1
sumpin
Enthusiast
sumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to behold
 
Posts: 29
Karma: 19952
Join Date: Sep 2011
Location: USA & UK
Device: down to just a PW4
[Kindle Touch] Calibre Kindle Collections Plug-in Hooks


This extension converts the collections in the Content Catalog (/var/local/cc.db) on the Kindle Touch to a JSON file (/mnt/us/system/collections.json) that can be read and updated by the Calibre Kindle Collections Plug-in. It then inserts/updates the collections managed by Calibre in Content Catalog. The script is compatible with collections and content that only exist on the Kindle and not in Calibre per the plug-in and books may exist in multiple collections.

Requirements:
1. Jailbreak - https://www.mobileread.com/forums/sho...d.php?t=160454
2. Launcher - https://www.mobileread.com/forums/sho...d.php?t=161263
3. Calibre - http://calibre-ebook.com/download
4. Calibre managed data stored on the Kindle in documents/calibre (i.e., under Calibre Preferences > Sending books to devices, Save template starts with: calibre)
5. Sqlite3 built with loadable extension support to run on the Kindle (included)

Install:
1. Extract both tar files to /mnt/us/extensions (If your zip file extractor has a setting like "TAR file smart CR/LF conversion", make sure it is disabled or the scripts cannot run)
2. Restart Kindle

To Use:
1. Select “Create input for Calibre” to export the existing collections
2. Connect Kindle to computer, run Calibre Kindle Collections Plug-in, eject Kindle
3. Select “Import data from Calibre and reboot” to import the new/modified collections

Note:
By default, when the Calibre data is loaded, the access times of all books is updated (avoid by changing –uat to +uat in menu.json) and the database is cleaned up by deleting the Collections Journal and by issuing an SQL VACUUM command (avoid by changing –vacuum to +vacuum in menu.json). To disable removing collections with no entries on import, add +rec to the command line of json2db.sh in menu.json.

By default, 5 copies of cc.db are maintained in /mnt/us/system. This can be changed by modifying the command line params of the json2db.sh script in menu.json.

If you run the import procedure too soon after loading data via Calibre, you may find that some books that are in a collection show up as not in a collection. Simply run the import again.

This supersedes my input to https://www.mobileread.com/forums/sho...d.php?t=160855 as that thread has taken the directory structure driven approach of CollectionSync https://www.mobileread.com/forums/sho...1&postcount=16.

I've split the release up into two parts. One tar file contains the sqlite3 binary and loadable extension needed and the other contains the Launcher hooks and scripts that do all the work.

I added a "progress bar" to the non-verbose import/export screens to give feedback that it's ticking along when importing large databases and I added verbose import/export to the menu to aid in trouble shooting and boredom reduction.

Updated for firmware 5.1.0 (use dynamic launcher menu, eips command line changes). Add menu item to update access times and remove this as default behavior.

Attachment d2j2d.tar.gz last updated on April 26th, 2012.
Attached Files
File Type: gz d2j2d-sql.tar.gz (884.8 KB, 1093 views)
File Type: gz d2j2d.tar.gz (11.3 KB, 974 views)

Last edited by sumpin; 04-26-2012 at 03:41 PM.
sumpin is offline   Reply With Quote
Old 03-05-2012, 12:29 AM   #2
dave2008
Addict
dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.dave2008 can program the VCR without an owner's manual.
 
Posts: 251
Karma: 183457
Join Date: Jan 2012
Device: k3G, KDXG, AuraHD
nice hack :-)
dave2008 is offline   Reply With Quote
Old 03-05-2012, 06:12 AM   #3
JustAMan
Groupie
JustAMan doesn't litterJustAMan doesn't litter
 
JustAMan's Avatar
 
Posts: 153
Karma: 113
Join Date: Jan 2012
Location: Russia
Device: Kindle Touch
Small advice - it would be better to store resulting .db somewhere else (or at least store copies not in /var/local as it easily fills up and thus breaks KT).
JustAMan is offline   Reply With Quote
Old 03-05-2012, 07:21 AM   #4
sumpin
Enthusiast
sumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to behold
 
Posts: 29
Karma: 19952
Join Date: Sep 2011
Location: USA & UK
Device: down to just a PW4
Good point. I updated the script to store the cc.db backups in /mnt/us/system as well.
sumpin is offline   Reply With Quote
Old 03-05-2012, 08:49 AM   #5
JustAMan
Groupie
JustAMan doesn't litterJustAMan doesn't litter
 
JustAMan's Avatar
 
Posts: 153
Karma: 113
Join Date: Jan 2012
Location: Russia
Device: Kindle Touch
And another idea - for those who don't install SSH it would be impossible to restore backup... Maybe add some menu item that would do exact that?
JustAMan is offline   Reply With Quote
Old 03-05-2012, 11:55 AM   #6
nynaevelan
eBook Junkie
nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.
 
nynaevelan's Avatar
 
Posts: 1,526
Karma: 1464018
Join Date: May 2010
Location: USA
Device: Kindle Fire 2020, Kindle PW2
I've tried running this but it does not seem to be working to put the books into collections. Perhaps I am misunderstanding something. If I put new books on the kindle, then run the kindle collections plugin, then run your script and reboot. Shouldn't it put the new books into the collections based on what the kindles plugin did?
nynaevelan is offline   Reply With Quote
Old 03-05-2012, 12:56 PM   #7
sumpin
Enthusiast
sumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to behold
 
Posts: 29
Karma: 19952
Join Date: Sep 2011
Location: USA & UK
Device: down to just a PW4
Quote:
Originally Posted by nynaevelan View Post
I've tried running this but it does not seem to be working to put the books into collections. Perhaps I am misunderstanding something. If I put new books on the kindle, then run the kindle collections plugin, then run your script and reboot. Shouldn't it put the new books into the collections based on what the kindles plugin did?
This is exactly what should happen. I've noticed that every time a new book gets added to cc.db, indexing and who knows what else kicks off. I've noticed that if I import the collections.json from calibre into cc.db before the cpu settles down, some of the books don't show up as in collections. I just re-run the import and all is dandy.

Also, if you're not storing your calibre content in documents/calibre, the script will skip those collections.

Just to prove what a masochist I am, I deleted my cc.db file. A zero length file was immediately created and it took a *long* time before all the content was re-added and re-indexed. When the indexing was done, I ran the export/calibre/import and got all my collections back.

Since stdout and stderr go to /dev/null it's hard to determine what's going wrong. The best bet is to ssh into the Kindle and run json2db.sh -v to see the output.
sumpin is offline   Reply With Quote
Old 03-06-2012, 04:37 AM   #8
wuschel83
Member
wuschel83 began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Feb 2012
Device: Kindle Touch
doesn´t work for me, all my folders are invisible now . how can i get them back to my screen???
wuschel83 is offline   Reply With Quote
Old 03-06-2012, 04:45 AM   #9
wuschel83
Member
wuschel83 began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Feb 2012
Device: Kindle Touch
i have delete the cc.db, after this i could start new:
rm /var/local/cc.db
wuschel83 is offline   Reply With Quote
Old 03-06-2012, 08:44 AM   #10
sumpin
Enthusiast
sumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to beholdsumpin is a splendid one to behold
 
Posts: 29
Karma: 19952
Join Date: Sep 2011
Location: USA & UK
Device: down to just a PW4
Quote:
Originally Posted by JustAMan View Post
And another idea - for those who don't install SSH it would be impossible to restore backup... Maybe add some menu item that would do exact that?
Done.
sumpin is offline   Reply With Quote
Old 03-23-2012, 05:18 PM   #11
nynaevelan
eBook Junkie
nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.nynaevelan ought to be getting tired of karma fortunes by now.
 
nynaevelan's Avatar
 
Posts: 1,526
Karma: 1464018
Join Date: May 2010
Location: USA
Device: Kindle Fire 2020, Kindle PW2
I took some time away from this and now I am back. I have run the import process twice and still the books are not showing up in collections. The wheel is not turning, how do I know if it doing anything?
nynaevelan is offline   Reply With Quote
Old 03-29-2012, 02:30 AM   #12
pookakitten
Connoisseur
pookakitten began at the beginning.
 
Posts: 55
Karma: 10
Join Date: Dec 2010
Device: K5
I'm not sure if this is caused by this or the Kindle Collections plugin, but it won't delete empty collections. The plugin reports less collections, but they never disappear when I reboot the kindle. The only way to remove the collections is to do a factory reset which is a pain, to say the least.

For now, I'm going to stick to creating tags based on tags which is genre for most books and fandom for fanfiction.
pookakitten is offline   Reply With Quote
Old 04-04-2012, 05:36 AM   #13
lsi
Junior Member
lsi began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Apr 2012
Device: Kindle Touch
I tried to implement this, but I cannot connect using SSH.

The first to steps were OK:
1. Jailbreak - https://www.mobileread.com/forums/sho...d.php?t=160454
2. Launcher - https://www.mobileread.com/forums/sho...d.php?t=161263
I was able to set up the USBnetworking, and my computer (Linux Mint) sees the network and connects. I use static IP 192.168.15.1.

Then when I run "sudo ssh root@192.168.15.244", I get "ssh: connect to host 192.168.15.244 port 22: Connection refused".

What else could I try?
lsi is offline   Reply With Quote
Old 04-04-2012, 08:02 AM   #14
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Quote:
Originally Posted by lsi View Post
I tried to implement this, but I cannot connect using SSH.

The first to steps were OK:
1. Jailbreak - https://www.mobileread.com/forums/sho...d.php?t=160454
2. Launcher - https://www.mobileread.com/forums/sho...d.php?t=161263
I was able to set up the USBnetworking, and my computer (Linux Mint) sees the network and connects. I use static IP 192.168.15.1.

Then when I run "sudo ssh root@192.168.15.244", I get "ssh: connect to host 192.168.15.244 port 22: Connection refused".

What else could I try?
nmap

Let it search for the address of the Kindle usbnet, which may not have recognized your entry of the address 192.168.15.244 in its configuration.

nmap 192.168.15.0/24
will report any addresses in that 256 address subnet range actively in-use and if port 22 is open on any of them.

"man nmap" for lots more details.
knc1 is offline   Reply With Quote
Old 04-04-2012, 10:53 AM   #15
lsi
Junior Member
lsi began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Apr 2012
Device: Kindle Touch
Quote:
Originally Posted by knc1 View Post
nmap

Let it search for the address of the Kindle usbnet, which may not have recognized your entry of the address 192.168.15.244 in its configuration.
Hi knc1! Thanks for your suggestion. This is what nmap says:
nmap 192.168.15.0/24

Starting Nmap 5.21 ( http://nmap.org ) at 2012-04-04 16:46 CEST
Nmap scan report for 192.168.15.1
Host is up (0.00022s latency).
Not shown: 997 closed ports
PORT STATE SERVICE
22/tcp open ssh
139/tcp open netbios-ssn
445/tcp open microsoft-ds

Nmap scan report for 192.168.15.244
Host is up (0.015s latency).
All 1000 scanned ports on 192.168.15.244 are closed

How should I word my ssh?
lsi is offline   Reply With Quote
Reply

Tags
calibre, collections, kindle touch, sqlite3

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Manage Collections in Kindle Touch eas187 Amazon Kindle 51 11-15-2013 09:37 AM
Kindle Touch Collections Database Fizzblizz Kindle Developer's Corner 22 02-28-2012 07:56 AM
Can I set up collections on the Kindle Touch in Windows? guspasho Amazon Kindle 10 12-13-2011 03:18 PM
Kindle Touch sorting by collections question tjsoundguy Amazon Kindle 4 12-01-2011 08:06 AM
Kindle Collections on Kindle 4 (non-touch) patrickmullin Plugins 0 11-23-2011 06:39 AM


All times are GMT -4. The time now is 05:25 AM.


MobileRead.com is a privately owned, operated and funded community.