Edit Metadata Not Responding

[dumining]

Hi all, please excuse my 'needy' newbie post but hope to help others as we go!

With Calibre I am going thru the new library using the Edit Metadata box, after downloading metadata etc I then click on 'next' and first it sits there for 15s doing nothing, then the title bar shows not responding and then some 60s or more later it finally flicks on the next book - thus it is taking a long time to go thru the library cleaning things up

I originally posted a ticket in the developers bug system, got a really fast response from kovid (thankyou!) to switch off antivirus for calibre locations and also switch off the tag browser - the ticket was then marked invalid

However the problem remains (unchanged) - I have switched off book details as well, no cover browser etc but still -

Any ideas from the community please?
(version 8.37, I am working on a w7 64 bit i5 SSD tablet, have the 12GB 1500 book library on a 32GB removable drive which is a class 10 sdhc)

[theducks]

SD cards are slow (compared to HD,SSHD)

Changing metadata, causes quite a few places/files to be updated.

One thing that might help, is to exclude the (Calibre) Library folder from active A/V scans .

[ilovejedd]

Originally Posted by dumining

With Calibre I am going thru the new library using the Edit Metadata box, after downloading metadata etc I then click on 'next' and first it sits there for 15s doing nothing, then the title bar shows not responding and then some 60s or more later it finally flicks on the next book - thus it is taking a long time to go thru the library cleaning things up

However the problem remains (unchanged) - I have switched off book details as well, no cover browser etc but still -

Any ideas from the community please?
(version 8.37, I am working on a w7 64 bit i5 SSD tablet, have the 12GB 1500 book library on a 32GB removable drive which is a class 10 sdhc)

Originally Posted by theducks

SD cards are slow (compared to HD,SSHD)

Changing metadata, causes quite a few places/files to be updated.

As theducks mentioned, SD cards have pretty slow write speeds (Class 10 is just 10 MB/s). My HDD has an average sustained write speed of 80 MB/s (min ~60 MB/s, max ~100 MB/s) and my SATA II SSD can do over 150 MB/s. That's just talking about sustained write.

Calibre database operations are probably more akin to random read/write performance in which case, a Class 4 or 6 card from a good manufacturer (e.g. Sandisk) might work better than a Class 10 card from another.

[dumining]

Thanks all (I did the anti-virus exclusion early on but no change)

AS SSD Benchmark 1.6.4237.30508
(from here http://www.overclock.net/t/754763/as...nchmark-thread
------------------------------
Name: Multiple Card Reader USB Device
Firmware: 1.00
Controller:
Offset: 4096 K - OK
Size: 29.88 GB
Date: 30/01/2012 4:02:12 PM
------------------------------
Sequential:
------------------------------
Read: 16.41 MB/s
Write: 8.30 MB/s
------------------------------
Access Times:
------------------------------
Read: 3.692 ms
Write: 17.971 ms

OK it is no rocket ship compared to a SSD but would like to be able to get more out of removable storage with Calibre - (but probably would require the database to be on the SSD only needing to point to files which dont change on the SDHC - which as I understand is not how Calibre works?)

It may be coincidence but changing in device manager - disk drives - multiple card reader USB device - properties - policies - select better performance (enables write caching) changed the 'move on to the next book in edit metadata' time from 60s+ to a consistent 25s.

[ilovejedd]

Originally Posted by dumining

------------------------------
Sequential:
------------------------------
Read: 16.41 MB/s
Write: 8.30 MB/s
------------------------------
Access Times:
------------------------------
Read: 3.692 ms
Write: 17.971 ms

OK it is no rocket ship compared to a SSD but would like to be able to get more out of removable storage with Calibre - (but probably would require the database to be on the SSD only needing to point to files which dont change on the SDHC - which as I understand is not how Calibre works?)

Those write access times look pretty bad. It might even be worse than mechanical hard drives. It's possible to get that kind of behavior (metadata in a different location from the actual library) by setting the CALIBRE_OVERRIDE_DATABASE_PATH variable but you would lose the ability to use multiple libraries. Still need to write to the SDHC for the metadata.opf, though.

Originally Posted by dumining

It may be coincidence but changing in device manager - disk drives - multiple card reader USB device - properties - policies - select better performance (enables write caching) changed the 'move on to the next book in edit metadata' time from 60s+ to a consistent 25s.

That's to be expected. Write caching uses RAM. If this solves your problem and you don't need to be able to quickly remove the SD card from your PC, you might just stick to this.

[theducks]

Originally Posted by dumining

It may be coincidence but changing in device manager - disk drives - multiple card reader USB device - properties - policies - select better performance (enables write caching) changed the 'move on to the next book in edit metadata' time from 60s+ to a consistent 25s.

I Do NOT recommend doing this. (long experience with multi-file Desktop database: Paradox DOS. A sure-fire way to damage things: Cached writes)

Caching can get things horribly out of sync.
Calibre is transactional. It writes into the DB, and it writes book files (not sure which order). If the queue has 2 operations on a book before the cache flushes those items to disk

[dumining]

thanks theducks, I have taken off the write caching to be safe, still have the time at 25s so it must have just been coincidence that I tried the change and thought it was the reason for the improvement

thanks ilovejedd - from the info you gave me I found this http://manual.calibre-ebook.com/portable.html - will try to make sense of it and work out how to get CALIBRE_OVERRIDE_DATABASE_PATH working - looks to be a way thru and I am not fussed with multiple library support

[dumining]

OK I see how it should work but struggling with the .bat file

Can someone lend a hand please with what the .bat file needs to say?
(common I would think for tablet users relying on removable storage for capacity)

the program is running from:
C:\Program Files (x86)\Calibre2
the library is in the removable storage
D:\000 library

many thanks

[ilovejedd]

Originally Posted by dumining

the program is running from:
C:\Program Files (x86)\Calibre2
the library is in the removable storage
D:\000 library

The following should work (store batch file in C:\Program Files (x86)\Calibre2):

SET CALIBRE_OVERRIDE_DATABASE_PATH="C:\Users\<username>\Calibre\metadata.db"
SET CALIBRE_LIBRARY_DIRECTORY="D:\000 library"
START /belownormal Calibre --with-library "%CALIBRE_LIBRARY_DIRECTORY%"

Just change the CALIBRE_OVERRIDE_DATABASE_PATH to wherever you want to store the Calibre metadata (on your SSD). I suggest not storing it in the Program Files directory.