[GUI Plugin] Kobo Utilities - Page 12

davidfor · 07-07-2013, 11:28 PM

Quote:

Originally Posted by PeterT

Ugh; that's kind of ugly. I suppose naive me had assumed that the volume-id / serial # / some unique value would also be stored, allowing one to easily swap between multiple SD cards without the need to re-scan them each time.

So many items seem to have a good initial thought to them, and then some poor implementation choices.

There is an id for the card stored in the ExternalId column in the content table. It is an 8 character hex value and I have no idea where it comes from. I have seen different values in the column. I could match one value to a card, but not the other. I assumed the cleanup when the new card had been inserted failed.

Also, the hex id is in the "Kobo eReader.conf". It's that "90164EF8=true" line. I have seen a couple of these in the file at the same time, but only one of them was used in the database.

davidfor · 07-10-2013, 01:28 AM

Quote:

Originally Posted by davidfor

Something I've been meaning to add for a while is a check of the images directory for extra image files. So, a new option that will scan the images directory and check for anything that looks like a cover and doesn't match an ImageID in the database. When started, there is a prompt whether to delete the extra images. The clean is run as a background job. When finished, it shows the number of extra image files found. The list of files is available by pressing the "Show details" button. The clean is done for the main memory and the SD card if it is installed.

This will probably be release next week as version 1.2.0.

As always, if you see a bug, tell me so I can squash it.

Hmm, no-one noticed that there wasn't a file attached to this post? I'm not sure what that says.

Anyway, here is another beta. Two changes:

- Fixed an error in the previous beta when running store as background job.
- Added "compress device database". This runs "vacuum" against the database to clean it and reduce the size. This might improve performance after adding a lot of books to the device.

It's still "next week", so I have a few more days before to release it.

BobC · 07-10-2013, 05:16 AM

Quote:

Originally Posted by davidfor

Hmm, no-one noticed that there wasn't a file attached to this post? I'm not sure what that says.

However I read the post I don't see anything that would suggest there should be a file attachment. It's not lack of interest that you didn't get a storm of protest about the (non)file, just eager anticipation .

BobC

davidfor · 07-10-2013, 06:24 AM

Quote:

Originally Posted by BobC

However I read the post I don't see anything that would suggest there should be a file attachment. It's not lack of interest that you didn't get a storm of protest about the (non)file, just eager anticipation .

Well, the post title was "Another beta". But rereading it, I can see that the text could be taken either way.

PeterT · 07-10-2013, 10:01 AM

Quote:

Originally Posted by davidfor

Hmm, no-one noticed that there wasn't a file attached to this post? I'm not sure what that says.

Anyway, here is another beta. Two changes:

- Fixed an error in the previous beta when running store as background job.
- Added "compress device database". This runs "vacuum" against the database to clean it and reduce the size. This might improve performance after adding a lot of books to the device.

It's still "next week", so I have a few more days before to release it.

I'd just assumed you were talking about what you were working on. I will admit I did try the old HTTP URL I had for the Plugin and saw no change there lol.

PeterT · 07-10-2013, 10:07 AM

No wonder I have so many issues with my database!

Tried a compress and got this

Quote:

calibre, version 0.9.38
Kobo Utilities - Compress Device Database: The database on the device has been compressed.
Original size = 10.252GB
Compressed size = 10.092GB

I had NO idea it was that large. MB maybe, NOT GB!

davidfor · 07-10-2013, 07:49 PM

Quote:

Originally Posted by PeterT

No wonder I have so many issues with my database!

Tried a compress and got this

I had NO idea it was that large. MB maybe, NOT GB!

That's a deliberate error to see if anyone is paying attention. And as you can tell by my not attaching the beta the last time, it's me not paying attention.

auspex · 07-10-2013, 08:29 PM

Quote:

Originally Posted by PeterT

Seeing the exact same sort options are available everywhere why are you surprised.

I'm not surprised, but clearly nobody who decided to ignore numbering bothered to actually look at normal use-cases for the shelves. When you're looking at Titles, it's easy to miss the importance of the third line of the display, but when you look at a shelf consisting of a series of three books, and the first line is always identical, and the third line shows "SERIES - 3" followed by "SERIES - 1" and then "... - 2" it becomes pretty obvious that you're missing something important.

davidfor · 07-10-2013, 09:15 PM

Quote:

Originally Posted by auspex

I'm not surprised, but clearly nobody who decided to ignore numbering bothered to actually look at normal use-cases for the shelves. When you're looking at Titles, it's easy to miss the importance of the third line of the display, but when you look at a shelf consisting of a series of three books, and the first line is always identical, and the third line shows "SERIES - 3" followed by "SERIES - 1" and then "... - 2" it becomes pretty obvious that you're missing something important.

Your right, but I disagree with you. Shelves are just shelves and have no smarts associated with them. They do have an extra sort sequences which is by date added. The fact that you have shelves that only contain books of one series just means you decided to set them up that way. Of course, Kobo could add function to do this with a shelf. But I hope they don't.

They don't need to do this as the series info is elsewhere. I would like them to present a series list that looks and works like the shelf list but is automatically built from the series info. As much as I would like to see this, I don't expect it to happen. But, I do hope to see a series sort option added in the appropriate places. Adding it as a sort for a shelve would be natural.

While I mention my wishes for a list, the other I want is an Author list. Again, it would look like the shelf list, but with an entry for each author and a count of books by that author. Again, I don't expect to see it, but I can hope.

DNSB · 07-11-2013, 12:40 AM

Quote:

Originally Posted by davidfor

That's a deliberate error to see if anyone is paying attention. And as you can tell by my not attaching the beta the last time, it's me not paying attention.

You mean I didn't have to go half crazy trying to figure out how Kobo fitted a 140GB database on a 16GB card? BTW, it shrank down to 70MB. This was after adding a 1400 books to do some testing and then deleting them.

Regards,
David

PeterT · 07-11-2013, 12:47 AM

/me hears the Hoover busy vacuuming the database

davidfor · 07-11-2013, 12:57 AM

Quote:

Originally Posted by DNSB

You mean I didn't have to go half crazy trying to figure out how Kobo fitted a 140GB database on a 16GB card?

Good compression?

Quote:

BTW, it shrank down to 70GB. This was after adding a 1400 books to do some testing and then deleting them.

So how many books are in the database now? 70whateverB still sounds pretty big.

DNSB · 07-11-2013, 03:59 PM

Quote:

Originally Posted by davidfor

Good compression?

So how many books are in the database now? 70whateverB still sounds pretty big.

70MB for around 1800 books. I think that the database should be much smaller for the number of books on the device. I can't see much use for quite a bit of the information stored in the database short of giving the users an SQL query interface possibly with precanned queries such as return the number of books with 17 to 19 table of content entries.

Regards,
David

auspex · 07-11-2013, 10:33 PM

I agree that there seems to be a lot of unnecessary info in the DB, but as a long time DB guy, I love to see too much data, rather than too little.

davidfor · 07-11-2013, 11:38 PM

In this case, I think they are storing to much data. The TOC rows in the content table fell like overkill. They are probably carried over from the older devices. The TOC can be read from the NCX file when the book is opened. They have a little more use for kepubs as when the book is opened the word count for each chapter is calculated and stored. Splitting the TOC entries off to another table would have cut the size by a lot. And it would improve performance for searches and the various lists as the main content table would be a lot smaller.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Manga plugin	mastertea	Plugins	6	01-06-2022 02:43 AM
[GUI Plugin] Save Virtual Libraries To Column (GUI)	chaley	Plugins	14	04-04-2021 05:25 AM
Kobo Utilities Plugin Question	nikev	Kobo Reader	10	09-25-2018 11:55 PM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

07-11-2013, 12:47 AM	#176
PeterT Grand Sorcerer Posts: 12,162 Karma: 73448616 Join Date: Nov 2007 Location: Toronto Device: Nexus 7, Clara, Touch, Tolino EPOS	/me hears the Hoover busy vacuuming the database

07-11-2013, 10:33 PM	#179
auspex Groupie Posts: 199 Karma: 1071756 Join Date: Sep 2012 Location: Nova Scotia Device: Kobo Aura, Nexus 5x	I agree that there seems to be a lot of unnecessary info in the DB, but as a long time DB guy, I love to see too much data, rather than too little.

07-11-2013, 11:38 PM	#180
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	In this case, I think they are storing to much data. The TOC rows in the content table fell like overkill. They are probably carried over from the older devices. The TOC can be read from the NCX file when the book is opened. They have a little more use for kepubs as when the book is opened the word count for each chapter is calculated and stored. Splitting the TOC entries off to another table would have cut the size by a lot. And it would improve performance for searches and the various lists as the main content table would be a lot smaller.