ebook-convert: reduce memory usage

Loteckal · 04-12-2015, 08:28 AM

Hi there

I'm running calibre-server (2.24) on a shared hosting enviroment (centos). Main drawback is that I can only use a limited amount of memory and sure I'm not root, but can compile applications. The normal "calibre-server" consume < 100MB (RES), for the ebook-convert I would like to stay under 200MB memory.

I also wanted to fetch some recipes and send them to my kindle. but the memory usage (ebook-convert) is above my limit. It goes up to 300-400MB easily and for the "New York Times" recipe ebook-convert uses around 1'141MB.

I guess this is not fixable (without rewrite the ebook-convert itself)- but would like to get some feedback from others.

bye

theducks · 04-12-2015, 10:33 AM

Stop converting complex sources

News recipe processing is intensive

Buy more RAM or
Switch to a lighter-weight Linux (just reduces the base use, leaving more of what you have for the user aps)

Seriously,
My mid range W7hp Laptop came with 3G RAM, My W7pro had 8G.
I have 2G in my 32bit XP system

Calibre is not some light-weight text editor. It is also made from common building blocks, that are not optimized for a single OS . Calibre is made to run on 3 OS

kovidgoyal · 04-16-2015, 12:52 AM

Conversion in calibre is designed to maintain in-memory parsed representations of everything in the book. That avoids the overhead of re-parsing repeatedly at different stages of the conversion process. This is not going to change. Your best bet is to just install enough swap in your server to handle the load, that means your server will slow to a crawl suring large conversions, but that should be relatively rare.

04-12-2015, 08:28 AM	#1
Loteckal Junior Member Posts: 1 Karma: 10 Join Date: Apr 2015 Location: Switzerland Device: Kindle	ebook-convert: reduce memory usage Hi there I'm running calibre-server (2.24) on a shared hosting enviroment (centos). Main drawback is that I can only use a limited amount of memory and sure I'm not root, but can compile applications. The normal "calibre-server" consume < 100MB (RES), for the ebook-convert I would like to stay under 200MB memory. I also wanted to fetch some recipes and send them to my kindle. but the memory usage (ebook-convert) is above my limit. It goes up to 300-400MB easily and for the "New York Times" recipe ebook-convert uses around 1'141MB. I guess this is not fixable (without rewrite the ebook-convert itself)- but would like to get some feedback from others. bye

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
n2a card memory usage	jvik	Android Devices	1	04-10-2015 02:28 PM
Memory usage	jessie102	Calibre	4	01-29-2015 12:44 PM
Memory Usage - 650 MB and Constantly Growing!	TomTanderson	Calibre	33	09-20-2012 09:10 PM
Extreme memory usage	MJBoa	Calibre	1	02-12-2010 11:37 AM
Memory Usage	BCCISProf	Sony Reader	15	05-16-2007 08:11 PM

04-12-2015, 10:33 AM	#2
theducks Well trained by Cats Posts: 29,799 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Stop converting complex sources News recipe processing is intensive Buy more RAM or Switch to a lighter-weight Linux (just reduces the base use, leaving more of what you have for the user aps) Seriously, My mid range W7hp Laptop came with 3G RAM, My W7pro had 8G. I have 2G in my 32bit XP system Calibre is not some light-weight text editor. It is also made from common building blocks, that are not optimized for a single OS . Calibre is made to run on 3 OS

04-16-2015, 12:52 AM	#3
kovidgoyal creator of calibre Posts: 43,852 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Conversion in calibre is designed to maintain in-memory parsed representations of everything in the book. That avoids the overhead of re-parsing repeatedly at different stages of the conversion process. This is not going to change. Your best bet is to just install enough swap in your server to handle the load, that means your server will slow to a crawl suring large conversions, but that should be relatively rare.