[GUI Plugin] Kobo Utilities - Page 67

lambstone · 09-07-2015, 11:32 AM

With the manage series function, It seems that after I click on manage series and manually enter series name, upon restart of my kobo, the series name is shown underneath the book title in grey.

I've did the calibre settings to automatically add "shelves/collection" but it does not add the series in grey. Any idea?

davidfor · 09-07-2015, 06:58 PM

Quote:

Originally Posted by lambstone

With the manage series function, It seems that after I click on manage series and manually enter series name, upon restart of my kobo, the series name is shown underneath the book title in grey.

Yes, that is the expected behaviour.

Quote:

I've did the calibre settings to automatically add "shelves/collection" but it does not add the series in grey. Any idea?

I'm not sure what you mean here. If you have configured the KoboTouch driver to manage collections, they are created on the device and will be shown in the collections list. This has no affect on the main library list.

Leonatus · 10-06-2015, 03:15 PM

Dear David! The last few times, backing up the device database has not been possible. I receive this notice:

Code:

calibre, version 2.40.0 (win32, isfrozen: True)
Failed to backup device database: Fehlgeschlagen: Backing up Kobo device database


wrong # of entries in index sqlite_autoindex_Event_1

Traceback (most recent call last):
  File "site-packages\calibre\gui2\device.py", line 87, in run
  File "calibre_plugins.koboutilities.jobs", line 199, in do_device_database_backup
Exception: 
wrong # of entries in index sqlite_autoindex_Event_1

I do not understand. Can you give me some hints, please!

PeterT · 10-06-2015, 05:24 PM

Simply put, there is some corruption in the database, and due to this the plugin will not back it up.

davidfor · 10-06-2015, 06:58 PM

As PeterT said, the database is corrupt. The backup process copies the database to the local drive and then runs a check on it to make sure it is OK. If it isn't OK, you will get an error something like you are seeing (which I really should add some more explanation top). When this happens, the backed up file is renamed to include the word "CORRUPT".

Because of what is corrupt, an autoindex on the Event table, you probably won't notice any problems. If anything goes wrong, it will be something like not keeping the reading stats properly. And when you connect and sync books with calibre, it won't complain as the table with the error is not one the driver uses. Luckily, this type of error can usually be fixed. If you use the utilities plugin function to do a compress (really an SQLite VACUUM), it should rebuild the index.

My suggestion is:
- Check when the last successful backup was.
- Run the compress.
- Run a database check.
- If there are no errors, eject the device and restart the device.
- If there are errors, restore the last successful backup.

That should clear the error when the backup is done.

Leonatus · 10-07-2015, 02:11 AM

Thanks, David and Peter, I'll try your suggestion. Up to now, I had no problems with the reader, and despite the corrupt database, the update to FW 3.18 that arrived yesterday, was completed without issue.

BTW: Glorious idea the possibility to automatically copy the error message!

Leonatus · 10-07-2015, 11:29 AM

Glad to notice it worked!
But, beg your pardon for my ignorance: I simply replaced the squlite file in order to restore the database. The "backup" functionality, however, collects multiple files of the reader in a .zip folder. Would it be purposeful to replace them all? - I suppose, in this case, it would be no good idea to simply copy the whole .zip folder to the .kobo directory and unpack it there, or?

davidfor · 10-07-2015, 09:27 PM

Quote:

Originally Posted by Leonatus

Glad to notice it worked!
But, beg your pardon for my ignorance: I simply replaced the squlite file in order to restore the database. The "backup" functionality, however, collects multiple files of the reader in a .zip folder. Would it be purposeful to replace them all? - I suppose, in this case, it would be no good idea to simply copy the whole .zip folder to the .kobo directory and unpack it there, or?

If the database is corrupt, just restore the database file. The others are there in case of a factory reset or signing out of the Kobo account. And not all the files go into the .kobo directory.

Leonatus · 10-08-2015, 02:23 AM

O.k. Now I see. Thank you, once more!

Stevex · 11-27-2015, 06:04 AM

Kobo Utilities Book Sorting Problem

As I understand the functionality, the addin creates the shelf sort and then adds a timecode in increasing time from the highest series number, in order to "dummy" the sort order on the device. It also sets the sort to "date added".

As far as I can see the addin is indeed functioning as expected but the kobo shows the books in random order. For example (I anonymised the names):

DEBUG: 166.3 _order_series_shelves - shelf=The Anonymous Series, count=4
DEBUG: 166.3 _order_series_shelves - shelf_data: (u'The Anonymous Series',)
DEBUG: 166.3 _order_series_shelves - row: 0 The Anonymous Series file:///mnt/onboard/Author Name/Series Name, The/001-CoS, The.epub The Anonymous Series 1
DEBUG: 166.3 _order_series_shelves - series_index= 1.0
DEBUG: 166.3 _order_series_shelves - row: 1 The Anonymous Series file:///mnt/onboard/Author Name/Series Name, The/002-NS, The.epub The Anonymous Series 2
DEBUG: 166.3 _order_series_shelves - series_index= 2.0
DEBUG: 166.3 _order_series_shelves - row: 2 The Anonymous Series file:///mnt/onboard/Author Name/Series Name, The/003-RoD, The.epub The Anonymous Series 3
DEBUG: 166.3 _order_series_shelves - series_index= 3.0
DEBUG: 166.3 _order_series_shelves - row: 3 The Anonymous Series file:///mnt/onboard/Author Name/Series Name, The/004-RaE.epub The Anonymous Series 4
DEBUG: 166.3 _order_series_shelves - series_index= 4.0
DEBUG: 166.3 _order_series_shelves - shelf_dict: {(u'The Anonymous Series', 2.0, u'The Naked Sun'): [u'file:///mnt/onboard/Author Name/Series Name, The/002-NS, The.epub'], (u'The Anonymous Series', 1.0, u'The CoS'): [u'file:///mnt/onboard/Author Name/Series Name, The/001-CoS, The.epub'], (u'The Anonymous Series', 4.0, u'RaE'): [u'file:///mnt/onboard/Author Name/Series Name, The/004-RaE.epub'], (u'The Anonymous Series', 3.0, u'The RoD'): [u'file:///mnt/onboard/Author Name/Series Name, The/003-RoD, The.epub']}
DEBUG: 166.3 _order_series_shelves - sorted shelf_dict: [(u'The Anonymous Series', 1.0, u'The CoS'), (u'The Anonymous Series', 2.0, u'The NS'), (u'The Anonymous Series', 3.0, u'The RoD'), (u'The Anonymous Series', 4.0, u'RaE')]
DEBUG: 166.3 _order_series_shelves - lastModifiedTime= 2015-11-23 12:38:05 timeDiff: 0:00:01
DEBUG: 166.3 _order_series_shelves - sort_key: (u'The Anonymous Series', 4.0, u'RaE') update_data: (u'2015-11-23T12:38:05Z', u'The Anonymous Series', u'file:///mnt/onboard/Author Name/Series Name, The/004-RaE.epub')
DEBUG: 166.3 _order_series_shelves - sort_key: (u'The Anonymous Series', 3.0, u'The RoD') update_data: (u'2015-11-23T12:38:06Z', u'The Anonymous Series', u'file:///mnt/onboard/Author Name/Series Name, The/003-RoD, The.epub')
DEBUG: 166.3 _order_series_shelves - sort_key: (u'The Anonymous Series', 2.0, u'The NS') update_data: (u'2015-11-23T12:38:07Z', u'The Anonymous Series', u'file:///mnt/onboard/Author Name/Series Name, The/002-NS, The.epub')
DEBUG: 166.3 _order_series_shelves - sort_key: (u'The Anonymous Series', 1.0, u'The CoS') update_data: (u'2015-11-23T12:38:08Z', u'The Anonymous Series', u'file:///mnt/onboard/Author Name/Series Name, The/001-CoS, The.epub')

So you can see that the timestamp does indeed go 05,06,07,08 in its last element which is I assume how it's meant to work.

Further, if I look at the database on the device itself, there is a table called ShelfContent which shows for the relevant shelf that the date modified does indeed reflect the above data, so that the update has worked (and there are no other date fields on the table so there can't be ambiguity).

However, on the device itself, although the Sort shows correctly as "Date Added" (and I can see elsewhere in the debug log that this is being set so as requested), the books are displayed in order 2,4,1,3

Many thanks for any help.

davidfor · 11-27-2015, 06:51 AM

Unfortunately, there is a bug in the firmware. The Date Added sort doesn't work when the dates have been read from the database. And it hasn't for a long time.

Stevex · 11-27-2015, 07:25 AM

Quote:

Originally Posted by davidfor

Unfortunately, there is a bug in the firmware. The Date Added sort doesn't work when the dates have been read from the database. And it hasn't for a long time.

Oh well that explains that then! Long question, short answer ;-)

Is there any workaround? It's obviously not a killer but with long series searching for 3-4-5 screens is definitely an annoyance. Is the bug reported to Kobo?

Many thanks for the reply. Steve

PeterT · 11-27-2015, 07:43 AM

The approach I've used for years is to create a collection for each series.

I use series as one of my columns in the device driver to create a collection per series. Additionally I use a metadata plugboard to set the book's title ob my Kobo;

Code:

Format: ePub
Device: KOBOTOUCH
Source Template: {series:re(([^\s])[^\s]+(\s|$),\1)}{series_index:0>2s| - | - }{title}
Destination Field: title

This makes finding books in series very easy for me.

I can just sort by title and see all books in a series in the correct order.

I should add that the plugboard prepends an abbreviated version of the Series and the series-index to the books title; so for instance Harry Potter and the Philosopher's Stone appears on my Kobo in the collection Harry Potter, with the title HP - 01 - Harry Potter and the Philosopher's Stone.

davidfor · 11-27-2015, 09:16 AM

Quote:

Originally Posted by Stevex

Oh well that explains that then! Long question, short answer ;-)

Is there any workaround? It's obviously not a killer but with long series searching for 3-4-5 screens is definitely an annoyance. Is the bug reported to Kobo?

Other than PeterT's suggestion of using a plugboard, I don't know of a way. I don't use that myself, but I don't have many long series.

And yes, Kobo definitely know about it.

Stevex · 11-27-2015, 09:38 AM

Thanks everyone! Yes the series=> collection thing I was already doing. I get the plugboard => file name prefix idea also (I have a few series where I use 3.5, 3.8 and suchlike for intervening shorts, so I might have to think on that...)

The odd thing is, I can't make ANY sense of the order actually shown. I've wasted a couple of hours now trying various combinations of stuff like:

select content.series, content.seriesnumber, content.dateadded, content.DateLastREad, content.datecreated, shelfcontent.datemodified from ShelfContent, Content where ShelfName="The xxx Series" AND ShelfContent.ContentID = content.ContentID ORDER BY content.dateadded, content.DateLastREad, shelfcontent.datemodified

Sometimes it seems to make sense, but more often I can't work it out at all. It makes me wonder if there is a bug in whatever database runtime they are using on the device?

10-07-2015, 02:11 AM	#996
Leonatus Wizard Posts: 1,060 Karma: 11391181 Join Date: Mar 2013 Location: Guben, Brandenburg, Germany Device: Kobo Clara 2E, Tolino Shine 3	Thanks, David and Peter, I'll try your suggestion. Up to now, I had no problems with the reader, and despite the corrupt database, the update to FW 3.18 that arrived yesterday, was completed without issue. BTW: Glorious idea the possibility to automatically copy the error message! Last edited by Leonatus; 10-07-2015 at 02:15 AM.

11-27-2015, 07:43 AM	#1003
PeterT Grand Sorcerer Posts: 13,563 Karma: 79436716 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	The approach I've used for years is to create a collection for each series. I use series as one of my columns in the device driver to create a collection per series. Additionally I use a metadata plugboard to set the book's title ob my Kobo; Code: Format: ePub Device: KOBOTOUCH Source Template: {series:re(([^\s])[^\s]+(\s\|$),\1)}{series_index:0>2s\| - \| - }{title} Destination Field: title This makes finding books in series very easy for me. I can just sort by title and see all books in a series in the correct order. I should add that the plugboard prepends an abbreviated version of the Series and the series-index to the books title; so for instance Harry Potter and the Philosopher's Stone appears on my Kobo in the collection Harry Potter, with the title HP - 01 - Harry Potter and the Philosopher's Stone. Last edited by PeterT; 11-27-2015 at 09:30 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Manga plugin	mastertea	Plugins	6	01-06-2022 02:43 AM
[GUI Plugin] Save Virtual Libraries To Column (GUI)	chaley	Plugins	14	04-04-2021 05:25 AM
Kobo Utilities Plugin Question	nikev	Kobo Reader	10	09-25-2018 11:55 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

09-07-2015, 11:32 AM	#991
lambstone Enthusiast Posts: 25 Karma: 10 Join Date: Apr 2014 Device: Kobo Glo	With the manage series function, It seems that after I click on manage series and manually enter series name, upon restart of my kobo, the series name is shown underneath the book title in grey. I've did the calibre settings to automatically add "shelves/collection" but it does not add the series in grey. Any idea?

10-06-2015, 05:24 PM	#994
PeterT Grand Sorcerer Posts: 13,563 Karma: 79436716 Join Date: Nov 2007 Location: Toronto Device: Libra H2O, Libra Colour	Simply put, there is some corruption in the database, and due to this the plugin will not back it up.

10-06-2015, 06:58 PM	#995
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	As PeterT said, the database is corrupt. The backup process copies the database to the local drive and then runs a check on it to make sure it is OK. If it isn't OK, you will get an error something like you are seeing (which I really should add some more explanation top). When this happens, the backed up file is renamed to include the word "CORRUPT". Because of what is corrupt, an autoindex on the Event table, you probably won't notice any problems. If anything goes wrong, it will be something like not keeping the reading stats properly. And when you connect and sync books with calibre, it won't complain as the table with the error is not one the driver uses. Luckily, this type of error can usually be fixed. If you use the utilities plugin function to do a compress (really an SQLite VACUUM), it should rebuild the index. My suggestion is: - Check when the last successful backup was. - Run the compress. - Run a database check. - If there are no errors, eject the device and restart the device. - If there are errors, restore the last successful backup. That should clear the error when the backup is done.

10-07-2015, 11:29 AM	#997
Leonatus Wizard Posts: 1,060 Karma: 11391181 Join Date: Mar 2013 Location: Guben, Brandenburg, Germany Device: Kobo Clara 2E, Tolino Shine 3	Glad to notice it worked! But, beg your pardon for my ignorance: I simply replaced the squlite file in order to restore the database. The "backup" functionality, however, collects multiple files of the reader in a .zip folder. Would it be purposeful to replace them all? - I suppose, in this case, it would be no good idea to simply copy the whole .zip folder to the .kobo directory and unpack it there, or?

10-08-2015, 02:23 AM	#999
Leonatus Wizard Posts: 1,060 Karma: 11391181 Join Date: Mar 2013 Location: Guben, Brandenburg, Germany Device: Kobo Clara 2E, Tolino Shine 3	O.k. Now I see. Thank you, once more!

11-27-2015, 06:04 AM	#1000
Stevex Addict Posts: 225 Karma: 15318 Join Date: Nov 2015 Device: Kobo Glo HD	Kobo Utilities Book Sorting Problem As I understand the functionality, the addin creates the shelf sort and then adds a timecode in increasing time from the highest series number, in order to "dummy" the sort order on the device. It also sets the sort to "date added". As far as I can see the addin is indeed functioning as expected but the kobo shows the books in random order. For example (I anonymised the names): DEBUG: 166.3 _order_series_shelves - shelf=The Anonymous Series, count=4 DEBUG: 166.3 _order_series_shelves - shelf_data: (u'The Anonymous Series',) DEBUG: 166.3 _order_series_shelves - row: 0 The Anonymous Series file:///mnt/onboard/Author Name/Series Name, The/001-CoS, The.epub The Anonymous Series 1 DEBUG: 166.3 _order_series_shelves - series_index= 1.0 DEBUG: 166.3 _order_series_shelves - row: 1 The Anonymous Series file:///mnt/onboard/Author Name/Series Name, The/002-NS, The.epub The Anonymous Series 2 DEBUG: 166.3 _order_series_shelves - series_index= 2.0 DEBUG: 166.3 _order_series_shelves - row: 2 The Anonymous Series file:///mnt/onboard/Author Name/Series Name, The/003-RoD, The.epub The Anonymous Series 3 DEBUG: 166.3 _order_series_shelves - series_index= 3.0 DEBUG: 166.3 _order_series_shelves - row: 3 The Anonymous Series file:///mnt/onboard/Author Name/Series Name, The/004-RaE.epub The Anonymous Series 4 DEBUG: 166.3 _order_series_shelves - series_index= 4.0 DEBUG: 166.3 _order_series_shelves - shelf_dict: {(u'The Anonymous Series', 2.0, u'The Naked Sun'): [u'file:///mnt/onboard/Author Name/Series Name, The/002-NS, The.epub'], (u'The Anonymous Series', 1.0, u'The CoS'): [u'file:///mnt/onboard/Author Name/Series Name, The/001-CoS, The.epub'], (u'The Anonymous Series', 4.0, u'RaE'): [u'file:///mnt/onboard/Author Name/Series Name, The/004-RaE.epub'], (u'The Anonymous Series', 3.0, u'The RoD'): [u'file:///mnt/onboard/Author Name/Series Name, The/003-RoD, The.epub']} DEBUG: 166.3 _order_series_shelves - sorted shelf_dict: [(u'The Anonymous Series', 1.0, u'The CoS'), (u'The Anonymous Series', 2.0, u'The NS'), (u'The Anonymous Series', 3.0, u'The RoD'), (u'The Anonymous Series', 4.0, u'RaE')] DEBUG: 166.3 _order_series_shelves - lastModifiedTime= 2015-11-23 12:38:05 timeDiff: 0:00:01 DEBUG: 166.3 _order_series_shelves - sort_key: (u'The Anonymous Series', 4.0, u'RaE') update_data: (u'2015-11-23T12:38:05Z', u'The Anonymous Series', u'file:///mnt/onboard/Author Name/Series Name, The/004-RaE.epub') DEBUG: 166.3 _order_series_shelves - sort_key: (u'The Anonymous Series', 3.0, u'The RoD') update_data: (u'2015-11-23T12:38:06Z', u'The Anonymous Series', u'file:///mnt/onboard/Author Name/Series Name, The/003-RoD, The.epub') DEBUG: 166.3 _order_series_shelves - sort_key: (u'The Anonymous Series', 2.0, u'The NS') update_data: (u'2015-11-23T12:38:07Z', u'The Anonymous Series', u'file:///mnt/onboard/Author Name/Series Name, The/002-NS, The.epub') DEBUG: 166.3 _order_series_shelves - sort_key: (u'The Anonymous Series', 1.0, u'The CoS') update_data: (u'2015-11-23T12:38:08Z', u'The Anonymous Series', u'file:///mnt/onboard/Author Name/Series Name, The/001-CoS, The.epub') So you can see that the timestamp does indeed go 05,06,07,08 in its last element which is I assume how it's meant to work. Further, if I look at the database on the device itself, there is a table called ShelfContent which shows for the relevant shelf that the date modified does indeed reflect the above data, so that the update has worked (and there are no other date fields on the table so there can't be ambiguity). However, on the device itself, although the Sort shows correctly as "Date Added" (and I can see elsewhere in the debug log that this is being set so as requested), the books are displayed in order 2,4,1,3 Many thanks for any help.

11-27-2015, 06:51 AM	#1001
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Unfortunately, there is a bug in the firmware. The Date Added sort doesn't work when the dates have been read from the database. And it hasn't for a long time.

11-27-2015, 09:38 AM	#1005
Stevex Addict Posts: 225 Karma: 15318 Join Date: Nov 2015 Device: Kobo Glo HD	Thanks everyone! Yes the series=> collection thing I was already doing. I get the plugboard => file name prefix idea also (I have a few series where I use 3.5, 3.8 and suchlike for intervening shorts, so I might have to think on that...) The odd thing is, I can't make ANY sense of the order actually shown. I've wasted a couple of hours now trying various combinations of stuff like: select content.series, content.seriesnumber, content.dateadded, content.DateLastREad, content.datecreated, shelfcontent.datemodified from ShelfContent, Content where ShelfName="The xxx Series" AND ShelfContent.ContentID = content.ContentID ORDER BY content.dateadded, content.DateLastREad, shelfcontent.datemodified Sometimes it seems to make sense, but more often I can't work it out at all. It makes me wonder if there is a bug in whatever database runtime they are using on the device?