Reconverted Entire Library

DoctorOhh · 09-13-2011, 02:59 AM

Because of my older eyes, I use larger fonts on my reader. Large fonts plus fully justified text is ugly. So I chose to take the time tonight to do a epub to epub conversion of my entire library. I added left justify and changed the top and bottom paragraph margins from .5em to .3em.

First, in calibre, I changed the number of processes able to run to take advantage of the new machine I just setup at work. I ended up with 8 conversions running at any time. Next I exited Dropbox, on rare occasions Dropbox seems to get in the way of a lot of reading/writing. So to be safe I exited Dropbox for the duration of the conversions.

I did the conversion in batches.

100 at first to see if my settings were correct.
900 Some more spot checking.
1000
2031
5055 This batch took exactly 2 hours.

When I finished I had 8 books that would not convert this time even though they had converted fine in the past. 1 book was corrupted beyond salvage and the other 7 all had the same UnicodeDecodeError. The error pointed to a themedata file left over from the books beginning as a Word document. I used tweak Epub to open each file and delete the offending file and converted these 7 like I had the rest. The 1 corrupted file I retrieved from an external drive and converted that to my liking too.

Now let's see how long it takes Dropbox to upload 9000+ new files. Fortunately my upload speed here at work is better than that at home (where my home machine is busy downloading the new updates.)

Kovid, thanks for a great program, it was fun to see all the cores of the cpu working their butts off.

speakingtohe · 09-13-2011, 03:51 AM

Just curious: Do you have the original_ file tweak on? If so did your library double in size?
Helen

DoctorOhh · 09-13-2011, 04:11 AM

Quote:

Originally Posted by speakingtohe

Just curious: Do you have the original_ file tweak on? If so did your library double in size?
Helen

No, I turned that off the day it was implemented.

Thanks for mentioning it, even though I am at the maximum free size of 10.25 gigs having that turned on under these circumstances definitely would have busted my Dropbox.

kovidgoyal · 09-13-2011, 11:35 AM

Yeah about the only times all 8 cores on my machine are really exercised is when I am running a bulk convert in calibre, or compiling software

speakingtohe · 09-13-2011, 12:52 PM

Now I want 8 cores

Helen

theducks · 09-13-2011, 01:17 PM

I feel left out with my P4 with 1 core (hyper-threaded)

09-13-2011, 02:59 AM	#1
DoctorOhh US Navy, Retired Posts: 9,889 Karma: 13806776 Join Date: Feb 2009 Location: North Carolina Device: Icarus Illumina XL HD, Kindle PaperWhite SE 11th Gen	Reconverted Entire Library Because of my older eyes, I use larger fonts on my reader. Large fonts plus fully justified text is ugly. So I chose to take the time tonight to do a epub to epub conversion of my entire library. I added left justify and changed the top and bottom paragraph margins from .5em to .3em. First, in calibre, I changed the number of processes able to run to take advantage of the new machine I just setup at work. I ended up with 8 conversions running at any time. Next I exited Dropbox, on rare occasions Dropbox seems to get in the way of a lot of reading/writing. So to be safe I exited Dropbox for the duration of the conversions. I did the conversion in batches. 100 at first to see if my settings were correct. 900 Some more spot checking. 1000 2031 5055 This batch took exactly 2 hours. When I finished I had 8 books that would not convert this time even though they had converted fine in the past. 1 book was corrupted beyond salvage and the other 7 all had the same UnicodeDecodeError. The error pointed to a themedata file left over from the books beginning as a Word document. I used tweak Epub to open each file and delete the offending file and converted these 7 like I had the rest. The 1 corrupted file I retrieved from an external drive and converted that to my liking too. Now let's see how long it takes Dropbox to upload 9000+ new files. Fortunately my upload speed here at work is better than that at home (where my home machine is busy downloading the new updates.) Kovid, thanks for a great program, it was fun to see all the cores of the cpu working their butts off. Last edited by DoctorOhh; 09-13-2011 at 03:02 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Check formats for entire library??	slayda	Library Management	6	08-02-2011 02:11 PM
Covers won't change without being reconverted	crossbred900	Calibre	3	10-09-2010 11:15 PM
Please Help - My Entire Calibre Library Disappeared	Marcy	Calibre	17	08-14-2010 09:56 AM
E-reader can hold the entire Library	honey	General Discussions	16	06-25-2010 02:58 AM
Translating entire pages	RickyMaveety	Lounge	3	02-10-2009 12:31 PM

09-13-2011, 03:51 AM	#2
speakingtohe Wizard Posts: 4,812 Karma: 26912940 Join Date: Apr 2010 Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet	Just curious: Do you have the original_ file tweak on? If so did your library double in size? Helen

09-13-2011, 11:35 AM	#4
kovidgoyal creator of calibre Posts: 45,170 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Yeah about the only times all 8 cores on my machine are really exercised is when I am running a bulk convert in calibre, or compiling software

09-13-2011, 12:52 PM	#5
speakingtohe Wizard Posts: 4,812 Karma: 26912940 Join Date: Apr 2010 Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet	Now I want 8 cores Helen

09-13-2011, 01:17 PM	#6
theducks Well trained by Cats Posts: 30,891 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I feel left out with my P4 with 1 core (hyper-threaded)