03-05-2016, 09:32 PM | #1 |
Member
Posts: 11
Karma: 10
Join Date: Apr 2010
Device: None
|
Calibre memory usage - Linux
I'm currently in the process of editing metadata in my newly created Calibre Library (approximately 1500 books) and I noticed the memory utilisation seemed to be increasing gradually.
Upon loading Calibre the initial memory used is: Code:
22504 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 681200 python2 /usr/bin/calibre --detach Code:
22504 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 70620 python2 /usr/bin/calibre-parallel 71468 python2 /usr/bin/calibre-parallel 827796 python2 /usr/bin/calibre --detach Code:
22504 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 1056120 python2 /usr/bin/calibre --detach So far I have used the copy of Calibre in the repos (Arch based Manjaro) and the binary install from the Calibre web site. I was thinking about building from git but I thought I'd check here first. Is this a memory leak? |
03-05-2016, 10:33 PM | #2 |
creator of calibre
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
No it isn't. Dynamic languages like python keep memory around in memory pools rather than always returning it to the OS immediately, for performance reasons.
|
Advert | |
|
03-05-2016, 10:39 PM | #3 |
Member
Posts: 11
Karma: 10
Join Date: Apr 2010
Device: None
|
|
03-05-2016, 10:43 PM | #4 |
creator of calibre
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
According to the figures you posted the memory increase is from
700MB to 1GB that is normal. If you are saying that memory usage keeps increasing unboundedly then that is not normal and is almost certainly something specific to your system. Start by disabling any third party plugins you have installed. |
03-05-2016, 10:50 PM | #5 | |
Member
Posts: 11
Karma: 10
Join Date: Apr 2010
Device: None
|
Quote:
|
|
Advert | |
|
03-05-2016, 11:02 PM | #6 |
creator of calibre
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
What would be the point of running in it in a vm with the same system? Use a different system.
Also what are you doing in calibre to cause the memory to increase? For instance if I start calibre memory usage is ~ 100MB then I use the edit metadata dialog for many books, memory usage climbs to 200MB and stays constant there no matter how many books I edit the metadata for. Since most tasks in calibre happen in short lived worker processes, I dont see how they could be leaking memory. The only other things I can think of that happen in the same process are running the calibre content server and connecting to a device. Two things you can do to decrease coupling to the system is use the binary calibre install and in Preferences->Look & Feel change the theme from system default to calibre. |
03-05-2016, 11:24 PM | #7 | ||||
Member
Posts: 11
Karma: 10
Join Date: Apr 2010
Device: None
|
Quote:
Quote:
Quote:
Quote:
For what it's worth, with a clean install of the OS and a clean install of Calibre - no library defined - the memory is: Code:
16592 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 254932 python2 /usr/bin/calibre --detach |
||||
03-05-2016, 11:50 PM | #8 |
Member
Posts: 11
Karma: 10
Join Date: Apr 2010
Device: None
|
It appears to be something on my main system causing this memory climb as importing my library into the new VM based system only causes a minimal change (lower?) in memory:
Code:
16468 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 249796 python2 /usr/bin/calibre Last edited by Dapper; 03-05-2016 at 11:53 PM. |
03-05-2016, 11:53 PM | #9 |
creator of calibre
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Pure python libraries are not. Python libraries compiled to native code are shared, just like other native libraries.
|
03-05-2016, 11:59 PM | #10 |
Member
Posts: 11
Karma: 10
Join Date: Apr 2010
Device: None
|
Ok, thanks. I'll investigate and maybe reinstall.
|
03-07-2016, 10:51 PM | #11 |
Member
Posts: 11
Karma: 10
Join Date: Apr 2010
Device: None
|
I found a cause for the massive increase in initial memory, changing the icons. In all honesty, in my earlier post, I'd forgotten I'd changed the icons, it wasn't until I started playing around in a VM that I noticed.
From what I can see, Circle UI and Square UI are the main culprits, both of which increase initial memory by several hundred megabytes. Initial memory - default icons Code:
22480 9684 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 246860 9656 python2 /usr/bin/calibre Code:
22292 9434 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 672080 9406 python2 /usr/bin/calibre Code:
22288 9790 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 251532 9758 python2 /usr/bin/calibre Code:
22292 9887 python2 /usr/bin/calibre-parallel --pipe-worker from calibre.utils.ipc.pool import run_main, worker_main; run_main(worker_main) 708424 9851 python2 /usr/bin/calibre |
03-07-2016, 11:03 PM | #12 |
creator of calibre
Posts: 43,776
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Those two icon themes presumably use very high resolution images as icons. You might want to post in the threads for the icon themes asking their creator to reduce the resolution of the images.
|
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
ebook-convert: reduce memory usage | Loteckal | Related Tools | 2 | 04-16-2015 12:52 AM |
n2a card memory usage | jvik | Android Devices | 1 | 04-10-2015 02:28 PM |
Memory usage | jessie102 | Calibre | 4 | 01-29-2015 12:44 PM |
Extreme memory usage | MJBoa | Calibre | 1 | 02-12-2010 11:37 AM |
Memory Usage | BCCISProf | Sony Reader | 15 | 05-16-2007 08:11 PM |