MobileRead Forums
Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre
Reload this Page 6.11 problem with adding large book collection

Notices

Reply
Page 1 of 2 1 2
 
Thread Tools Search this Thread
Old 09-08-2009, 07:46 AM   #1
acekyoto
Member
acekyoto began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Sep 2009
Device: PRS-600
6.11 problem with adding large book collection
Here is my problem. I have a large number of books on my hard drive sorted in folders alphabetically. For example in my 'A' folder I have 1.8GB worth of books. My total book collection (not in calibre) right now is more than 50GB

When I add a folder of books to Calibre I select the add books section and the third selection down which is to search folders for multiple books and add them to the library.

The first bar goes by very fast as it scan all the books in the folder. The second bar that pops up called 'adding' moves extremely slowly. Maybe 1% every 10-15 minutes and sometimes longer. I don't think I've ever seen it reach the end because I usually go to bed. When I wake up I check it I see a calibre message about a problem adding the files to the library.

I added folders in 6.10 just fine and it didn't take long at all so I don't think its anything with my system. I need 6.11 for the prs-600 support.

Any idea what the problem is?

Some info

I have a PRS-600 so that's my chosen setting.
Win Vista x64 8gb of ram and core2duo CPU.
My calibre library and my book collection are both on an external 500GB HD
AVG antivirus
acekyoto is offline   Reply With Quote
Old 09-08-2009, 11:10 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,826
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
First thing to try is to turn off your antivirus temporarily and see if it speeds things up. Next thing to try is to run calibre from a comamnd prompt as


calibre-debug -g

it should post some debug output to the terminal as its adding books, post that here.

Third thing to try is go into preferences and raise the process priority from Normal to High.
kovidgoyal is offline   Reply With Quote
Advert
Old 09-08-2009, 06:12 PM   #3
acekyoto
Member
acekyoto began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Sep 2009
Device: PRS-600
This is one part of the debug but it just repeats as different books are added

Found files...
HTMLFile:0:a:H:\Books\A\Axler, James\Axler, James - Dea
thlands 22 - Rider, Reaper - Laurence James 1 0b.html
Parsing Axler%2c%20James%20-%20Deathlands%2022%20-%20Rider%2c%20Reaper%2
0-%20Laurence%20James%201%200b.html ...
Input debug saved to: c:\users\allan\appdata\local\temp\calibre_0.6.11_z
egbsn_plugin_html2zip\input
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Helvetica-Bold', subst is 'Helvetica'
Error: Couldn't find a font for 'Helvetica-Bold'
Error: Couldn't find a font for 'Helvetica-Bold', subst is 'Helvetica'
Error: Couldn't find a font for 'Helvetica-Bold'
Error: Couldn't find a font for 'Helvetica-BoldOblique', subst is 'Helve
tica'
Error: Couldn't find a font for 'Helvetica-BoldOblique'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Helvetica', subst is 'Helvetica'
Error: Couldn't find a font for 'Helvetica'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Bold', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Bold'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Helvetica-Bold', subst is 'Helvetica'
Error: Couldn't find a font for 'Helvetica-Bold'
Error: Couldn't find a font for 'Helvetica-BoldOblique', subst is 'Helve
tica'
Error: Couldn't find a font for 'Helvetica-BoldOblique'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Helvetica', subst is 'Helvetica'
Error: Couldn't find a font for 'Helvetica'
InputFormatPlugin: HTML Input running on H:\Books\A\Axler, James\Axler,
James - Deathlands 29 - Bloodlines - Laurence James 1 0b.html
Language not specified
Creator not specified
Building file list...
Found files...
HTMLFile:0:a:H:\Books\A\Axler, James\Axler, James - Dea
thlands 29 - Bloodlines - Laurence James 1 0b.html
Parsing Axler%2c%20James%20-%20Deathlands%2029%20-%20Bloodlines%20-%20La
urence%20James%201%200b.html ...
Input debug saved to: c:\users\allan\appdata\local\temp\calibre_0.6.11_f
d22cr_plugin_html2zip\input
Error: Couldn't find a font for 'Times-Bold', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Bold'
Error: Couldn't find a font for 'Times-Bold', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Bold'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Bold', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Bold'
Error: Couldn't find a font for 'Times-Bold', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Bold'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
Error: Couldn't find a font for 'Times-Roman', subst is 'Helvetica'
Error: Couldn't find a font for 'Times-Roman'
InputFormatPlugin: HTML Input running on H:\Books\A\Axler, James\Axler,
James - Deathlands 33 - Eclipse at Noon.html
Language not specified
Creator not specified
Building file list...
acekyoto is offline   Reply With Quote
Old 09-08-2009, 07:31 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,826
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
What format are these books in?
kovidgoyal is offline   Reply With Quote
Old 09-08-2009, 09:14 PM   #5
acekyoto
Member
acekyoto began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Sep 2009
Device: PRS-600
A whole bunch of different formats. Some are PDF. Many are .lit. Some are .mobi...some are .txt. ...basically any and all formats.

Why would that matter when just adding them to library? Wouldn't it matter when converting them?
acekyoto is offline   Reply With Quote
Advert
Old 09-09-2009, 03:40 AM   #6
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,552
Karma: 950151
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
You might want to check that your TEMP folder has not filled up with Calibre related files that have not been removed. I find that on my system I have to do a periodic houskeeping of this folder to avoid a large build-up of files at this location. Whether that is affecting performance I am not sure, but I feel it must when the number of files here gets very large.
Last edited by itimpi; 09-09-2009 at 03:42 AM.
itimpi is offline   Reply With Quote
Old 09-09-2009, 06:09 AM   #7
acekyoto
Member
acekyoto began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Sep 2009
Device: PRS-600
Ok I'll try that as soon as i get home from the office.
acekyoto is offline   Reply With Quote
Old 09-09-2009, 09:01 AM   #8
acekyoto
Member
acekyoto began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Sep 2009
Device: PRS-600
Hmm...Ok. I went to my temp folder and found 42 Calibre folders in there. There is nothing in almost all of the folders and I assume its safe to delete them since they are in the temp folder. Deleting those...and then trying a big book add again...
acekyoto is offline   Reply With Quote
Old 09-09-2009, 09:27 AM   #9
acekyoto
Member
acekyoto began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Sep 2009
Device: PRS-600
Well that didn't work...deleted the temp folder and still have the same problem....changed the priority to 'high' and same problem. Disabled the virus scanner...same problem. Hmmm...it must just be my system since no one else is reporting this. I wish I could put my finger on what it is though.

**I wonder if its because my book and library are on an external drive. Calibre is installed to my c:\ drive and my temp folder is also on C:\ The Calibre library is on H:\ and so are all my books. Could there be a problem with that somehow?
Last edited by acekyoto; 09-09-2009 at 09:55 AM. Reason: addtional info
acekyoto is offline   Reply With Quote
Old 09-09-2009, 10:36 AM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,826
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The format of the books matters, because some formats, like html have to be processed before import. HTML is parsed, all the link followed and collected into a zip file. For most other formats though, it shouldn't make a difference. As for being on an external drive, that also shouldn't matter as long as disk access speeds are high on that drive.
kovidgoyal is offline   Reply With Quote
Old 09-09-2009, 05:13 PM   #11
acekyoto
Member
acekyoto began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Sep 2009
Device: PRS-600
Ahh...I see...so the problem is with .html files....I'm sure there are lots of those. Is there anything I can do to speed up the process? Is there any way Calibre could skip .html parsing when doing a library add?

**I went to the advanced/plugin section and disabled .html parsing to a zip file just to see if that was causing things to slow down but it didn't seem to make a difference. Would .doc files cause a problem because of parsing?
Last edited by acekyoto; 09-09-2009 at 05:39 PM. Reason: adding information
acekyoto is offline   Reply With Quote
Old 09-09-2009, 05:28 PM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,826
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Go to Preferences -> Plugins and disable the HTML2Zip plugin
kovidgoyal is offline   Reply With Quote
Old 09-09-2009, 08:59 PM   #13
acekyoto
Member
acekyoto began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Sep 2009
Device: PRS-600
Yeah I disabled but no difference. I just installed the 3.1 64bit version of Python on my system. Previously I was running the 2.5 version. I wonder if that will make a difference
acekyoto is offline   Reply With Quote
Old 09-09-2009, 09:02 PM   #14
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,826
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
No, it wont. Python 3.1 is actually slower than python 2.x and in anycase calibre comes with its own embedded python interpreter
kovidgoyal is offline   Reply With Quote
Old 09-10-2009, 12:01 AM   #15
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,552
Karma: 950151
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
As I understand it at the moment Calibre assumes the Python 2.6 series (with the current latest release being 2.6.2). There are significant language changes between the 2.6 and 3.x series, so I would not be sure that Calibre would compile correctly with 3.1 (although I guess I could be wrong).

Also as Kovid mentions, at the moment the 3.x series is measurably slower than the 2.6 series of python.
itimpi is offline   Reply With Quote
Reply
Page 1 of 2 1 2

« Previous Thread | Next Thread »
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Tips for managing a large collection? ficbot Reading and Management 23 03-13-2010 02:11 AM
Adding books to a large database Student1 Calibre 31 04-07-2009 06:43 PM
Large free ebook collection tech_au Deals and Resources (No Self-Promotion or Affiliate Links) 1 12-05-2008 08:47 AM
Problem with BookDesigner on large book Megatron-UK Sony Reader 6 04-07-2008 02:52 PM


All times are GMT -4. The time now is 04:25 AM.

Contact Us - FAQ - Guidelines - Privacy - Top

MobileRead.com is a privately owned, operated and funded community.