Converting epub to mobi within app 7x slower than from command line???

andrewburt · 11-25-2015, 03:09 PM

Hi, all, I noticed that converting the same .epub file to a .mobi inside the app (win10 PC) took 7 minutes. I then converted a copy of the same .epub using "ebook-convert" via the command line (same PC, in a separate, empty directory, so no involvement with the original files)-- and it took 1 minute. (Same results if I do it in the reverse order, so it isn't a caching kind of thing.)

I compared the job log outputs (using the -v -v -v option on the command line), and they are essentially the same. (The semi-random output order of the internal xml files are displayed in two different random orders, and some options that say 'None' show as '' empty, or 0.0 vs 0, or for search_replace, '[]' vs None, but I can't imagine those being meaningful.) All the settings are essentially the same. There's no smoking gun showing.

I've repeated this on different books, same result: In app takes waaaaay longer than command line.

This is for version 2.44.1.

When watching the in-app log, it has particularly slow points (e.g. ~1+ minutes each) at "Flattening css and remapping font sizes", "Parsing manglecase.css", "converting xhtml to mobipocket markup", and "compressing markup content." When running ebook-convert from the command line, it doesn't pause there nearly as long.

???

Is there something running as interpreted code that's compiled in the command line version? Or...???

Thanks...

itimpi · 11-25-2015, 06:24 PM

As far as I know they actually both end up using the same code for conversion.

You might want to check what priority the conversion runs as when run from the GUI? I have a feeling it might run at a reduced priority to reduce the impact on system performance.

You also might want to try the conversions in different orders in case some sort of caching is coming into play.

BetterRed · 11-25-2015, 06:50 PM

^-^

The default Priority is Normal (I have it set to Low) its in Preferences->Behaviour, also check the settings re simultaneous jobs in Preferences->Miscellaneous.

My experience is that if you over-commit resources overall performance (i.e throughput, productivity, effective work) diminishes - at least as it applies to computers, people, cows (less milk) and sheep (less twins).

BR

andrewburt · 11-26-2015, 10:53 PM

Thanks for the suggestions... though no dice. Still took way longer to convert in-app than via command line. (And when speedily running via command line, it doesn't impact system performance in any noticeable way, so it's not like it's in some cpu-sucking super-high priority state. Indeed, checking, it's the same, normal cpu priority as most other processes.)

I also looked to see what processes are running via the app -- and the app does not call ebook-convert.exe, it shows instead as calibre-parallel.exe. So it's not the same identical software (even if they share some common code). Whatever the difference is, the in-app approach is dramatically slower.

Hmm!?!

kovidgoyal · 11-26-2015, 11:15 PM

You have an antivirus program interfering. And yes, it is the exact same code. The name of the executable is irrelevant. Executables in calibre are simply entry point to underlying python code.

andrewburt · 11-28-2015, 12:05 PM

Kovid, thanks for the info! I did more testing (disabling antivirus, etc.) -- and it turns out there was indeed a small but important difference in the job log output I hadn't noticed. I just saw that the hex addresses of the output profile were different, and didn't notice that, in fact, the output profile name itself was different.

The upshot is that on the 7x slower cases, it was using an output profile of ipad3. Using the default output profile -- 7x faster!

(Just documenting this for others who might stumble on the issue. I played around with some other output profiles, and found some had no impact on speed, some others had the massive slowdown [such as ipad, nook HD+]. I looked at both the "ipad3" and "default" resulting .mobi output on an ipad, and saw no visual difference, in text or images, at least to my eye, meaning I personally don't see any benefit from the 7x slowness, so won't be using that.)

Thanks everyone for all the help. Hopefully this info can help others avoid possibly needlessly slow conversions.

11-25-2015, 03:09 PM	#1
andrewburt Science Fiction Writer Posts: 229 Karma: 124188 Join Date: Dec 2010 Location: Golden, CO Device: several	Converting epub to mobi within app 7x slower than from command line??? Hi, all, I noticed that converting the same .epub file to a .mobi inside the app (win10 PC) took 7 minutes. I then converted a copy of the same .epub using "ebook-convert" via the command line (same PC, in a separate, empty directory, so no involvement with the original files)-- and it took 1 minute. (Same results if I do it in the reverse order, so it isn't a caching kind of thing.) I compared the job log outputs (using the -v -v -v option on the command line), and they are essentially the same. (The semi-random output order of the internal xml files are displayed in two different random orders, and some options that say 'None' show as '' empty, or 0.0 vs 0, or for search_replace, '[]' vs None, but I can't imagine those being meaningful.) All the settings are essentially the same. There's no smoking gun showing. I've repeated this on different books, same result: In app takes waaaaay longer than command line. This is for version 2.44.1. When watching the in-app log, it has particularly slow points (e.g. ~1+ minutes each) at "Flattening css and remapping font sizes", "Parsing manglecase.css", "converting xhtml to mobipocket markup", and "compressing markup content." When running ebook-convert from the command line, it doesn't pause there nearly as long. ??? Is there something running as interpreted code that's compiled in the command line version? Or...??? Thanks...

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Formatting when converting ePub to Mobi on ipad Kindle app	jetmaster	Apple Devices	9	05-20-2013 05:32 PM
Ebook conversion (EPUB to MOBI) via command line	ebooklab	Conversion	4	09-16-2011 09:34 PM
Command line convert ePub->MOBI mangles cover	cscotts	Calibre	7	12-14-2010 12:21 AM
bullets add a new line converting ePub to MOBI	jomaweb	Calibre	4	06-22-2010 11:13 AM

11-25-2015, 06:24 PM	#2
itimpi Wizard Posts: 4,552 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	As far as I know they actually both end up using the same code for conversion. You might want to check what priority the conversion runs as when run from the GUI? I have a feeling it might run at a reduced priority to reduce the impact on system performance. You also might want to try the conversions in different orders in case some sort of caching is coming into play.

11-25-2015, 06:50 PM	#3
BetterRed null operator (he/him) Posts: 20,570 Karma: 26954694 Join Date: Mar 2012 Location: Sydney Australia Device: none	^-^ The default Priority is Normal (I have it set to Low) its in Preferences->Behaviour, also check the settings re simultaneous jobs in Preferences->Miscellaneous. My experience is that if you over-commit resources overall performance (i.e throughput, productivity, effective work) diminishes - at least as it applies to computers, people, cows (less milk) and sheep (less twins). BR

11-26-2015, 10:53 PM	#4
andrewburt Science Fiction Writer Posts: 229 Karma: 124188 Join Date: Dec 2010 Location: Golden, CO Device: several	Thanks for the suggestions... though no dice. Still took way longer to convert in-app than via command line. (And when speedily running via command line, it doesn't impact system performance in any noticeable way, so it's not like it's in some cpu-sucking super-high priority state. Indeed, checking, it's the same, normal cpu priority as most other processes.) I also looked to see what processes are running via the app -- and the app does not call ebook-convert.exe, it shows instead as calibre-parallel.exe. So it's not the same identical software (even if they share some common code). Whatever the difference is, the in-app approach is dramatically slower. Hmm!?!

11-26-2015, 11:15 PM	#5
kovidgoyal creator of calibre Posts: 43,858 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You have an antivirus program interfering. And yes, it is the exact same code. The name of the executable is irrelevant. Executables in calibre are simply entry point to underlying python code.

11-28-2015, 12:05 PM	#6
andrewburt Science Fiction Writer Posts: 229 Karma: 124188 Join Date: Dec 2010 Location: Golden, CO Device: several	Kovid, thanks for the info! I did more testing (disabling antivirus, etc.) -- and it turns out there was indeed a small but important difference in the job log output I hadn't noticed. I just saw that the hex addresses of the output profile were different, and didn't notice that, in fact, the output profile name itself was different. The upshot is that on the 7x slower cases, it was using an output profile of ipad3. Using the default output profile -- 7x faster! (Just documenting this for others who might stumble on the issue. I played around with some other output profiles, and found some had no impact on speed, some others had the massive slowdown [such as ipad, nook HD+]. I looked at both the "ipad3" and "default" resulting .mobi output on an ipad, and saw no visual difference, in text or images, at least to my eye, meaning I personally don't see any benefit from the 7x slowness, so won't be using that.) Thanks everyone for all the help. Hopefully this info can help others avoid possibly needlessly slow conversions.

Advert

Advert