Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 07-08-2013, 12:28 AM   #166
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
Quote:
Originally Posted by PeterT View Post
Ugh; that's kind of ugly. I suppose naive me had assumed that the volume-id / serial # / some unique value would also be stored, allowing one to easily swap between multiple SD cards without the need to re-scan them each time.

So many items seem to have a good initial thought to them, and then some poor implementation choices.
There is an id for the card stored in the ExternalId column in the content table. It is an 8 character hex value and I have no idea where it comes from. I have seen different values in the column. I could match one value to a card, but not the other. I assumed the cleanup when the new card had been inserted failed.

Also, the hex id is in the "Kobo eReader.conf". It's that "90164EF8=true" line. I have seen a couple of these in the file at the same time, but only one of them was used in the database.
davidfor is offline   Reply With Quote
Old 07-10-2013, 02:28 AM   #167
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
Beta 1.1.3 - Compress device database

Quote:
Originally Posted by davidfor View Post
Something I've been meaning to add for a while is a check of the images directory for extra image files. So, a new option that will scan the images directory and check for anything that looks like a cover and doesn't match an ImageID in the database. When started, there is a prompt whether to delete the extra images. The clean is run as a background job. When finished, it shows the number of extra image files found. The list of files is available by pressing the "Show details" button. The clean is done for the main memory and the SD card if it is installed.

This will probably be release next week as version 1.2.0.

As always, if you see a bug, tell me so I can squash it.
Hmm, no-one noticed that there wasn't a file attached to this post? I'm not sure what that says.

Anyway, here is another beta. Two changes:

- Fixed an error in the previous beta when running store as background job.
- Added "compress device database". This runs "vacuum" against the database to clean it and reduce the size. This might improve performance after adding a lot of books to the device.

It's still "next week", so I have a few more days before to release it.
Attached Files
File Type: zip KoboUtilities-1.1.3.zip (102.1 KB, 37 views)

Last edited by davidfor; 07-10-2013 at 07:23 AM.
davidfor is offline   Reply With Quote
Old 07-10-2013, 06:16 AM   #168
BobC
Addict
BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.BobC ought to be getting tired of karma fortunes by now.
 
Posts: 354
Karma: 245756
Join Date: Dec 2008
Location: Lancashire, U.K.
Device: BeBook 1, BeBook Pure, Kobo Glo, Various Android Apps
Quote:
Originally Posted by davidfor View Post
Hmm, no-one noticed that there wasn't a file attached to this post? I'm not sure what that says.
However I read the post I don't see anything that would suggest there should be a file attachment. It's not lack of interest that you didn't get a storm of protest about the (non)file, just eager anticipation .

BobC
BobC is offline   Reply With Quote
Old 07-10-2013, 07:24 AM   #169
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
Quote:
Originally Posted by BobC View Post
However I read the post I don't see anything that would suggest there should be a file attachment. It's not lack of interest that you didn't get a storm of protest about the (non)file, just eager anticipation .
Well, the post title was "Another beta". But rereading it, I can see that the text could be taken either way.
davidfor is offline   Reply With Quote
Old 07-10-2013, 11:01 AM   #170
PeterT
Taking a break; Fed up
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 7,370
Karma: 46288753
Join Date: Nov 2007
Location: Toronto
Device: Wife: Touch, Arc, Vox Me: Nexus 7, Glo
Quote:
Originally Posted by davidfor View Post
Hmm, no-one noticed that there wasn't a file attached to this post? I'm not sure what that says.

Anyway, here is another beta. Two changes:

- Fixed an error in the previous beta when running store as background job.
- Added "compress device database". This runs "vacuum" against the database to clean it and reduce the size. This might improve performance after adding a lot of books to the device.

It's still "next week", so I have a few more days before to release it.
I'd just assumed you were talking about what you were working on. I will admit I did try the old HTTP URL I had for the Plugin and saw no change there lol.
PeterT is offline   Reply With Quote
Old 07-10-2013, 11:07 AM   #171
PeterT
Taking a break; Fed up
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 7,370
Karma: 46288753
Join Date: Nov 2007
Location: Toronto
Device: Wife: Touch, Arc, Vox Me: Nexus 7, Glo
No wonder I have so many issues with my database!

Tried a compress and got this
Quote:
calibre, version 0.9.38
Kobo Utilities - Compress Device Database: The database on the device has been compressed.
Original size = 10.252GB
Compressed size = 10.092GB
I had NO idea it was that large. MB maybe, NOT GB!
PeterT is offline   Reply With Quote
Old 07-10-2013, 08:49 PM   #172
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
Quote:
Originally Posted by PeterT View Post
No wonder I have so many issues with my database!

Tried a compress and got this


I had NO idea it was that large. MB maybe, NOT GB!
That's a deliberate error to see if anyone is paying attention. And as you can tell by my not attaching the beta the last time, it's me not paying attention.
davidfor is offline   Reply With Quote
Old 07-10-2013, 09:29 PM   #173
auspex
Groupie
auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.
 
auspex's Avatar
 
Posts: 162
Karma: 1066430
Join Date: Sep 2012
Location: Nova Scotia
Device: Sony PRS-T2, Samsung Galaxy Ace IIx
Quote:
Originally Posted by PeterT View Post
Seeing the exact same sort options are available everywhere why are you surprised.
I'm not surprised, but clearly nobody who decided to ignore numbering bothered to actually look at normal use-cases for the shelves. When you're looking at Titles, it's easy to miss the importance of the third line of the display, but when you look at a shelf consisting of a series of three books, and the first line is always identical, and the third line shows "SERIES - 3" followed by "SERIES - 1" and then "... - 2" it becomes pretty obvious that you're missing something important.
auspex is offline   Reply With Quote
Old 07-10-2013, 10:15 PM   #174
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
Quote:
Originally Posted by auspex View Post
I'm not surprised, but clearly nobody who decided to ignore numbering bothered to actually look at normal use-cases for the shelves. When you're looking at Titles, it's easy to miss the importance of the third line of the display, but when you look at a shelf consisting of a series of three books, and the first line is always identical, and the third line shows "SERIES - 3" followed by "SERIES - 1" and then "... - 2" it becomes pretty obvious that you're missing something important.
Your right, but I disagree with you. Shelves are just shelves and have no smarts associated with them. They do have an extra sort sequences which is by date added. The fact that you have shelves that only contain books of one series just means you decided to set them up that way. Of course, Kobo could add function to do this with a shelf. But I hope they don't.

They don't need to do this as the series info is elsewhere. I would like them to present a series list that looks and works like the shelf list but is automatically built from the series info. As much as I would like to see this, I don't expect it to happen. But, I do hope to see a series sort option added in the appropriate places. Adding it as a sort for a shelve would be natural.

While I mention my wishes for a list, the other I want is an Author list. Again, it would look like the shelf list, but with an entry for each author and a count of books by that author. Again, I don't expect to see it, but I can hope.
davidfor is offline   Reply With Quote
Old 07-11-2013, 01:40 AM   #175
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 2,337
Karma: 6272177
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Aura HD, Glo, Nexus 7 HD, iPad Air
Quote:
Originally Posted by davidfor View Post
That's a deliberate error to see if anyone is paying attention. And as you can tell by my not attaching the beta the last time, it's me not paying attention.
You mean I didn't have to go half crazy trying to figure out how Kobo fitted a 140GB database on a 16GB card? BTW, it shrank down to 70MB. This was after adding a 1400 books to do some testing and then deleting them.

Regards,
David

Last edited by DNSB; 07-12-2013 at 03:21 AM. Reason: Fat fingers cause typos as does turning off brain before typing...
DNSB is offline   Reply With Quote
Old 07-11-2013, 01:47 AM   #176
PeterT
Taking a break; Fed up
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
PeterT's Avatar
 
Posts: 7,370
Karma: 46288753
Join Date: Nov 2007
Location: Toronto
Device: Wife: Touch, Arc, Vox Me: Nexus 7, Glo
/me hears the Hoover busy vacuuming the database
PeterT is offline   Reply With Quote
Old 07-11-2013, 01:57 AM   #177
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
Quote:
Originally Posted by DNSB View Post
You mean I didn't have to go half crazy trying to figure out how Kobo fitted a 140GB database on a 16GB card?
Good compression?
Quote:
BTW, it shrank down to 70GB. This was after adding a 1400 books to do some testing and then deleting them.
So how many books are in the database now? 70whateverB still sounds pretty big.
davidfor is offline   Reply With Quote
Old 07-11-2013, 04:59 PM   #178
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 2,337
Karma: 6272177
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Aura HD, Glo, Nexus 7 HD, iPad Air
Quote:
Originally Posted by davidfor View Post
Good compression?


So how many books are in the database now? 70whateverB still sounds pretty big.
70MB for around 1800 books. I think that the database should be much smaller for the number of books on the device. I can't see much use for quite a bit of the information stored in the database short of giving the users an SQL query interface possibly with precanned queries such as return the number of books with 17 to 19 table of content entries.

Regards,
David
DNSB is offline   Reply With Quote
Old 07-11-2013, 11:33 PM   #179
auspex
Groupie
auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.auspex ought to be getting tired of karma fortunes by now.
 
auspex's Avatar
 
Posts: 162
Karma: 1066430
Join Date: Sep 2012
Location: Nova Scotia
Device: Sony PRS-T2, Samsung Galaxy Ace IIx
I agree that there seems to be a lot of unnecessary info in the DB, but as a long time DB guy, I love to see too much data, rather than too little.
auspex is offline   Reply With Quote
Old 07-12-2013, 12:38 AM   #180
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 6,090
Karma: 6238033
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo Touch, Kobo Glo
In this case, I think they are storing to much data. The TOC rows in the content table fell like overkill. They are probably carried over from the older devices. The TOC can be read from the NCX file when the book is opened. They have a little more use for kepubs as when the book is opened the word count for each chapter is calculated and stored. Splitting the TOC entries off to another table would have cut the size by a lot. And it would improve performance for searches and the various lists as the main content table would be a lot smaller.
davidfor is offline   Reply With Quote
Reply

Tags
kobo

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Hyphenate This! SauliusP. Plugins 209 12-07-2014 02:44 PM
[GUI Plugin] EpubSplit JimmXinu Plugins 151 11-07-2014 12:06 PM
[GUI Plugin] KindleUnpack - The Plugin DiapDealer Plugins 286 10-30-2014 10:25 AM
[GUI Plugin] Wordpress frescogamba Plugins 8 05-20-2014 01:22 PM
[GUI Plugin] Plugin Updater **Deprecated** kiwidude Plugins 159 06-19-2011 01:27 PM


All times are GMT -4. The time now is 06:25 AM.


MobileRead.com is a privately owned, operated and funded community.