Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > More E-Book Readers > iRex > iRex Developer's Corner

Notices

Reply
 
Thread Tools Search this Thread
Old 04-15-2011, 10:22 AM   #31
dima_tr
Groupie
dima_tr has learned how to read e-booksdima_tr has learned how to read e-booksdima_tr has learned how to read e-booksdima_tr has learned how to read e-booksdima_tr has learned how to read e-booksdima_tr has learned how to read e-booksdima_tr has learned how to read e-books
 
Posts: 175
Karma: 863
Join Date: Oct 2009
Location: Germany, Karlsruhe
Device: Iliad v2, DR800S (inlove)
Or trigger vacuuming as an additional script after each disconnect from PC, just after updating global.db. I agree, autovacuuming might not be a good idea.
dima_tr is offline   Reply With Quote
Old 04-15-2011, 11:43 AM   #32
Iņigo
Guru
Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.
 
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
Quote:
Originally Posted by Mackx View Post
Something related:
I recently examined a global.db file from someone else. It seems that in the past a lot of documents where put onto it and removed again. When I used the VACUUM-command (using the free SqLite2009Pro program) the size of global.db was reduced from ~2.5MB to ~0.5MB

Maybe reducing the size also improves the speed?
Hi there, back from a travel.

@mackx: I do vacuum often when I mount the SD on my linux main computer. So this is not related to the slowness I suffer.
I recently discovered other problems with my DB, maybe related: it looks like some files in my SD make the DB a bit inconsistent. If I delete global.db to regenerate it, the indexing stops after 1.2MB of file size. I haven't been able to discover the file, but as the backup global.db file works is not a big issue for me.
A theory: mdbindex binary calls code in ctb, which uses metadata gathered from uds.
As I'm using dr1000's uds in a dr800 (dr800+ in fact ;-) perhaps there is some kind in incompatibility.

@dima_tr: auto-vacuum is a bad idea, it could last lots of time.


Starting on sunday I'll take a week off, visiting family and friends, with no good internet (well, only on the mobile)... but I'll carry some interesting ideas I'm currently thinking about... ;-)

Last edited by Iņigo; 04-15-2011 at 11:45 AM.
Iņigo is offline   Reply With Quote
Advert
Old 04-16-2011, 04:25 AM   #33
Mackx
Guru
Mackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to behold
 
Posts: 999
Karma: 19985
Join Date: Dec 2008
Location: Netherlands
Device: iRex DR1000S
Quote:
Originally Posted by Iņigo View Post
I recently discovered other problems with my DB, maybe related: it looks like some files in my SD make the DB a bit inconsistent. If I delete global.db to regenerate it, the indexing stops after 1.2MB of file size. I haven't been able to discover the file, but as the backup global.db file works is not a big issue for me.
A theory: mdbindex binary calls code in ctb, which uses metadata gathered from uds.
As I'm using dr1000's uds in a dr800 (dr800+ in fact ;-) perhaps there is some kind in incompatibility.
1) I assume you have thumbnail generation (during mdbindex) enabled, I wonder if the performance is also dependent on the number of thumbnails in the global.db file?
2) you can run mdbindex from the command line (although I only have done this in the emulator) first argument is the location of global.db, 2nd parameter determines if thumbnails will be generated. When there is a 2nd parameter no thumbnails are generated. By adding a few print you should be able to determine the file that is causing the problems.

(Have a nice holiday.:-)
Mackx is offline   Reply With Quote
Old 04-16-2011, 04:38 AM   #34
Iņigo
Guru
Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.
 
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
Quote:
Originally Posted by Mackx View Post
1) I assume you have thumbnail generation (during mdbindex) enabled, I wonder if the performance is also dependent on the number of thumbnails in the global.db file?
2) you can run mdbindex from the command line (although I only have done this in the emulator) first argument is the location of global.db, 2nd parameter determines if thumbnails will be generated. When there is a 2nd parameter no thumbnails are generated. By adding a few print you should be able to determine the file that is causing the problems.

(Have a nice holiday.:-)
Haven't tried running a clean indexing without thumbnails...
Regarding 2), I often run mdbindex from usbnet, inside a ssh session (btw, this is a marvellous way to detect bugs that don't appear on the emulator).

In fact some last weekend I recompiled ctb with some prints to discover the file causing the problem, but I suppose I put then in wrong places because I couldn't find out anything useful. It was a quict & dirty editing as I had to work last weekend. Then, travel so it will have to wait some days.

Iņigo
Iņigo is offline   Reply With Quote
Old 04-16-2011, 04:45 AM   #35
Mackx
Guru
Mackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to beholdMackx is a splendid one to behold
 
Posts: 999
Karma: 19985
Join Date: Dec 2008
Location: Netherlands
Device: iRex DR1000S
Quote:
Originally Posted by dima_tr View Post
Or trigger vacuuming as an additional script after each disconnect from PC, just after updating global.db. I agree, autovacuuming might not be a good idea.
I do not like the unpredictable duration of vacuum, how long does it take to vacuum a multi MB database file? So (at this moment) I am not in favour of doing it automatically.
Maybe some code in a Database-Settings-dialog that would be able to check the state of global.db and propose a vacuum to the user. An alternative would be to have such a functionality integrated in ctb.
On the other hand, we do not know which problem a vacuum would solve... does it improve performance or does it only cleanup some diskspace?
Mackx is offline   Reply With Quote
Advert
Old 04-16-2011, 05:55 AM   #36
rvs
Groupie
rvs has a complete set of Star Wars action figures.rvs has a complete set of Star Wars action figures.rvs has a complete set of Star Wars action figures.rvs has a complete set of Star Wars action figures.
 
Posts: 181
Karma: 364
Join Date: Oct 2009
Device: DR1000s, Hanlin V5
With about 1700 books and no automatic thumbnail generation it takes about 2 seconds to load the Books view on my device, so there's a good chance that thumbnails are the main culprit.

BTW there's an added advantage when not using thumbnail generation on indexing. It's an easy way to see if a book has been opened before or not, because it will generate a thumbnail when opening a book.
rvs is offline   Reply With Quote
Old 04-16-2011, 06:11 AM   #37
Iņigo
Guru
Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.
 
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
Quote:
Originally Posted by Mackx View Post
I do not like the unpredictable duration of vacuum, how long does it take to vacuum a multi MB database file? So (at this moment) I am not in favour of doing it automatically.
Maybe some code in a Database-Settings-dialog that would be able to check the state of global.db and propose a vacuum to the user. An alternative would be to have such a functionality integrated in ctb.
On the other hand, we do not know which problem a vacuum would solve... does it improve performance or does it only cleanup some diskspace?
Another solution could be to perform the vaccum after the indexing, that would add no more than 2-3 seconds to the process, but as the user is already waiting it wouldn't be a problem.

When deleting rows in a sqlite db, it creates void room inside the db. Vacuum-ing basically compacts the file, making (sometimes) the file smaller and improving the performance (a bit).

Anyway, for a typical DR use, there is not much books deletion so vacuum is usually not needed. As much, once each 3 months if you change your contents often.

Iņigo
Iņigo is offline   Reply With Quote
Old 04-16-2011, 06:16 AM   #38
Iņigo
Guru
Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.
 
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
Quote:
Originally Posted by rvs View Post
With about 1700 books and no automatic thumbnail generation it takes about 2 seconds to load the Books view on my device, so there's a good chance that thumbnails are the main culprit.
uhmmm, interesting. I'll do some tests after my return.

Quote:
Originally Posted by rvs View Post
BTW there's an added advantage when not using thumbnail generation on indexing. It's an easy way to see if a book has been opened before or not, because it will generate a thumbnail when opening a book.
But those thumbnails are so nice... (flowers, colours, springtime is here ;-)
ok, ok, only for the north hemisphere...
Iņigo is offline   Reply With Quote
Old 06-23-2011, 10:36 AM   #39
Iņigo
Guru
Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.Iņigo did not drink the Kool Aid.
 
Posts: 730
Karma: 72743
Join Date: Feb 2008
Location: Here or there
Device: iRex iLiad, iRex DR800S. K4NT. Kobo Aura, Aura One, Libra 2.
This thread somehow continue in https://www.mobileread.com/forums/sho...d.php?t=139330
Iņigo is offline   Reply With Quote
Reply

Tags
secret, suprise, testing

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Strange things happening w/PDF ebook Writernan Astak EZReader 5 04-05-2010 06:36 PM
Best firmware for Hanlin V5 aka Bebook Mini aka Papyre 5.1? javimm HanLin eBook 1 01-11-2010 05:37 PM
Strange things happening at Fictionwise... captaingeorges Bookeen 5 03-08-2009 05:20 AM
The Secret of Lost Things Taylor514ce Reading Recommendations 7 05-15-2008 09:06 AM
Strange things happening after Kindle update tsgreer Amazon Kindle 15 02-22-2008 05:34 PM


All times are GMT -4. The time now is 08:17 AM.


MobileRead.com is a privately owned, operated and funded community.