![]() |
#1 |
Junior Member
![]() Posts: 4
Karma: 10
Join Date: Jun 2014
Device: Kindle
|
Memory leak in calibre-server?
Possibly there is a memory leak in calibre-server.
calibre-server runs on an unused netbook with 64-bit Linux, dual core, 2GB RAM and 1GB swap space. X is not running. Server load is very low (a few ebook readers). After some days calibre-server consumes all available memory and Linux begins to trash between RAM and swap. The machine is unusable. Restarting calibre-server solves this problem immediately. Last edited by mondai; 06-24-2014 at 04:04 AM. |
![]() |
![]() |
![]() |
#2 |
Grand Sorcerer
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 13,472
Karma: 78880114
Join Date: Nov 2007
Location: Toronto
Device: Libra H2O, Libra Colour
|
What version of calibre and is it installed from the official calibre site or from your distribution's repository?
|
![]() |
![]() |
Advert | |
|
![]() |
#3 | |
Junior Member
![]() Posts: 4
Karma: 10
Join Date: Jun 2014
Device: Kindle
|
Quote:
OS is latest Lubuntu 14.04 64-bit. |
|
![]() |
![]() |
![]() |
#4 |
creator of calibre
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 45,322
Karma: 27111242
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
I cannot reproduce this. You will need to figure out what is leaking exactly. Monitor the memory usage of the process and make some requests from the server, see which requests are causing the increase in memory consumption, and note that the server is written in python which automatically manages memory so monitoring memory is not completely straightforward as python will not always relinquish memory to the OS completely, so to detect a leak you need to make the same request repeatedly and check that the memory usage climbs by approximately the same amount each time.
|
![]() |
![]() |
![]() |
#5 |
Junior Member
![]() Posts: 4
Karma: 10
Join Date: Jun 2014
Device: Kindle
|
The virtual memory size of calibre-server is very high: 2 GB. That is equal to the complete physical memory. Take a look at the output of memstat below.
Could it be that there is a problem with the CPU and the OS? The CPU is an Intel N570 Atom Processor. "Max. Memory Size" is 2 GB. Therefore it makes no sense to use additional swap space (except for hibernation). Intel Atom Processor N570 Another strange behavior is loading a lot of X-related libraries, but X is not running. Code:
2394708k: PID 2047 (/opt/calibre-1.41/bin/calibre-server) 39168k( 0k): /opt/calibre-1.41/lib/libicudata.so.49 2047 2053 2047 2053 2047 2053 29324k( 21988k): /opt/calibre-1.41/lib/libQtWebKit.so.4 2047 2053 2047 2053 2047 2053 15616k( 10944k): /opt/calibre-1.41/lib/libQtGui.so.4 2047 2053 2047 2053 2047 2053 2047 2053 12996k( 6620k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtGui.so 2047 2053 2047 2053 2047 2053 8940k( 4348k): /opt/calibre-1.41/lib/libQtXmlPatterns.so.4 2047 2053 2047 2053 2047 2053 2047 2053 7140k( 2948k): /opt/calibre-1.41/lib/libQtCore.so.4 2047 2053 2047 2053 2047 2053 7016k( 2136k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtCore.so 2047 2053 2047 2053 2047 2053 2047 2053 6972k( 2036k): /opt/calibre-1.41/lib/libMagickCore.so.5 2047 2053 2047 2053 2047 2053 6320k( 2120k): /opt/calibre-1.41/lib/libicui18n.so.49 2047 2053 2047 2053 2047 2053 2047 2053 6080k( 1688k): /opt/calibre-1.41/lib/libcrypto.so.1.0.0 2047 2053 2047 2053 2047 2053 2047 2053 6080k( 1480k): /opt/calibre-1.41/lib/libpython2.7.so.1.0 2047 2053 2047 2053 2047 2053 5740k( 1220k): /opt/calibre-1.41/lib/python2.7/site-packages/lxml/etree.so 2047 2053 2047 2053 2047 2053 5700k( 1460k): /opt/calibre-1.41/lib/libicuuc.so.49 2047 2053 2047 2053 2047 2053 5476k( 1316k): /opt/calibre-1.41/lib/libQtNetwork.so.4 2047 2053 2047 2053 2047 2053 5472k( 1304k): /opt/calibre-1.41/lib/libxml2.so.2 2047 2053 2047 2053 2047 2053 2047 2053 5248k( 1120k): /opt/calibre-1.41/lib/libMagickWand.so.5 2047 2053 2047 2053 2047 2053 5240k( 1128k): /opt/calibre-1.41/lib/libglib-2.0.so.0 2047 2053 2047 2053 2047 2053 5136k( 968k): /opt/calibre-1.41/lib/libstdc++.so.6 2047 2053 2047 2053 2047 2053 4940k( 612k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtNetwork.so 2047 2053 2047 2053 2047 2053 4848k( 600k): /opt/calibre-1.41/lib/python2.7/lib-dynload/unicodedata.so 2047 2053 2047 2053 2047 2053 4824k( 696k): /opt/calibre-1.41/lib/libsqlite3.so.0 2047 2053 2047 2053 2047 2053 2047 2053 4616k( 504k): /opt/calibre-1.41/lib/libQtDBus.so.4 2047 2053 2047 2053 2047 2053 2047 2053 4612k( 380k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtWebKit.so 2047 2053 2047 2053 2047 2053 2047 2053 4588k( 460k): /opt/calibre-1.41/lib/libgcrypt.so.11 2047 2053 2047 2053 2047 2053 4560k( 432k): /opt/calibre-1.41/lib/libtiff.so.5 2047 2053 2047 2053 2047 2053 4552k( 384k): /opt/calibre-1.41/lib/libssl.so.1.0.0 2047 2053 2047 2053 2047 2053 2047 2053 4480k( 360k): /opt/calibre-1.41/lib/libQtSvg.so.4 2047 2053 2047 2053 2047 2053 4480k( 272k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtXml.so 2047 2053 2047 2053 2047 2053 4476k( 276k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtDesigner.so 2047 2053 2047 2053 2047 2053 4428k( 244k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtOpenGL.so 2047 2053 2047 2053 2047 2053 4404k( 276k): /opt/calibre-1.41/lib/liblcms2.so.2 2047 2053 2047 2053 2047 2053 4404k( 268k): /opt/calibre-1.41/lib/python2.7/site-packages/apsw.so 2047 2053 2047 2053 2047 2053 4396k( 276k): /opt/calibre-1.41/lib/libQtXml.so.4 2047 2053 2047 2053 2047 2053 4364k( 188k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtDBus.so 2047 2053 2047 2053 2047 2053 4360k( 248k): /opt/calibre-1.41/lib/libjpeg.so.8 2047 2053 2047 2053 2047 2053 4348k( 236k): /opt/calibre-1.41/lib/libdbus-1.so.3 2047 2053 2047 2053 2047 2053 4328k( 224k): /opt/calibre-1.41/lib/libxslt.so.1 2047 2053 2047 2053 2047 2053 4296k( 144k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtXmlPatterns.so 2047 2053 2047 2053 2047 2053 4296k( 112k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_io.so 2047 2053 2047 2053 2047 2053 4272k( 152k): /opt/calibre-1.41/lib/libexpat.so.1 2047 2053 2047 2053 2047 2053 4256k( 152k): /opt/calibre-1.41/lib/libpng14.so.14 2047 2053 2047 2053 2047 2053 4240k( 104k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_ctypes.so 2047 2053 2047 2053 2047 2053 4224k( 112k): /opt/calibre-1.41/lib/python2.7/site-packages/sip.so 2047 2053 2047 2053 2047 2053 4200k( 80k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtSvg.so 2047 2053 2047 2053 2047 2053 4196k( 68k): /opt/calibre-1.41/lib/python2.7/lib-dynload/datetime.so 2047 2053 2047 2053 2047 2053 4192k( 80k): /opt/calibre-1.41/lib/libz.so.1 2047 2053 2047 2053 2047 2053 4192k( 56k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_socket.so 2047 2053 2047 2053 2047 2053 4188k( 76k): /opt/calibre-1.41/lib/libexslt.so.0 2047 2053 2047 2053 2047 2053 4176k( 72k): /opt/calibre-1.41/lib/python2.7/lib-dynload/cPickle.so 2047 2053 2047 2053 2047 2053 4176k( 40k): /opt/calibre-1.41/lib/python2.7/lib-dynload/itertools.so 2047 2053 2047 2053 2047 2053 4164k( 60k): /opt/calibre-1.41/lib/libbz2.so.1 2047 2053 2047 2053 2047 2053 4160k( 48k): /opt/calibre-1.41/lib/libicuio.so.49 2047 2053 2047 2053 2047 2053 2047 2053 4160k( 40k): /opt/calibre-1.41/lib/python2.7/lib-dynload/pyexpat.so 2047 2053 2047 2053 2047 2053 4156k( 52k): /opt/calibre-1.41/lib/python2.7/site-packages/calibre/plugins/magick.so 2047 2053 2047 2053 2047 2053 4144k( 40k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_elementtree.so 2047 2053 2047 2053 2047 2053 4144k( 32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/operator.so 2047 2053 2047 2053 2047 2053 4144k( 32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/math.so 2047 2053 2047 2053 2047 2053 4144k( 32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/array.so 2047 2053 2047 2053 2047 2053 4140k( 28k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtTest.so 2047 2053 2047 2053 2047 2053 4140k( 28k): /opt/calibre-1.41/lib/python2.7/site-packages/calibre/plugins/icu.so 2047 2053 2047 2053 2047 2053 4140k( 28k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_struct.so 2047 2053 2047 2053 2047 2053 4140k( 20k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_csv.so 2047 2053 2047 2053 2047 2053 4136k( 32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_ssl.so 2047 2053 2047 2053 2047 2053 4136k( 32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_json.so 2047 2053 2047 2053 2047 2053 4136k( 24k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_collections.so 2047 2053 2047 2053 2047 2053 4132k( 28k): /opt/calibre-1.41/lib/libffi.so.4 2047 2053 2047 2053 2047 2053 4132k( 20k): /opt/calibre-1.41/lib/python2.7/lib-dynload/zlib.so 2047 2053 2047 2053 2047 2053 4132k( 20k): /opt/calibre-1.41/lib/python2.7/lib-dynload/strop.so 2047 2053 2047 2053 2047 2053 4132k( 20k): /opt/calibre-1.41/lib/python2.7/lib-dynload/select.so 2047 2053 2047 2053 2047 2053 4128k( 16k): /opt/calibre-1.41/lib/python2.7/lib-dynload/time.so 2047 2053 2047 2053 2047 2053 4128k( 16k): /opt/calibre-1.41/lib/python2.7/lib-dynload/cStringIO.so 2047 2053 2047 2053 2047 2053 4124k( 12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_heapq.so 2047 2053 2047 2053 2047 2053 4120k( 16k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_locale.so 2047 2053 2047 2053 2047 2053 4120k( 16k): /opt/calibre-1.41/lib/python2.7/lib-dynload/binascii.so 2047 2053 2047 2053 2047 2053 4116k( 12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_random.so 2047 2053 2047 2053 2047 2053 4116k( 12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_hashlib.so 2047 2053 2047 2053 2047 2053 4116k( 12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_functools.so 2047 2053 2047 2053 2047 2053 4116k( 12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/fcntl.so 2047 2053 2047 2053 2047 2053 4116k( 12k): /opt/calibre-1.41/lib/libgpg-error.so.0 2047 2053 2047 2053 2047 2053 4116k( 12k): /opt/calibre-1.41/lib/libcalibre-launcher.so 2047 2053 2047 2053 2047 2053 4112k( 8k): /opt/calibre-1.41/lib/python2.7/site-packages/calibre/plugins/speedup.so 2047 2053 2047 2053 2047 2053 4112k( 8k): /opt/calibre-1.41/lib/python2.7/lib-dynload/grp.so 2047 2053 2047 2053 2047 2053 4112k( 8k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_bisect.so 2047 2053 2047 2053 2047 2053 4108k( 4k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/Qt.so 2047 2053 2047 2053 2047 2053 4108k( 4k): /opt/calibre-1.41/lib/python2.7/lib-dynload/future_builtins.so 2047 2053 2047 2053 2047 2053 2096k( 40k): /opt/calibre-1.41/lib/ImageMagick-6.7.6/modules-Q16/coders/jpeg.so 2047 20k( 4k): /opt/calibre-1.41/bin/calibre-server 2047 2053 2047 2053 2047 2053 |
![]() |
![]() |
Advert | |
|
![]() |
#6 |
creator of calibre
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() Posts: 45,322
Karma: 27111242
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
memory usage and memory leaks are very different things. Which one are you asking about?
|
![]() |
![]() |
![]() |
#7 |
Junior Member
![]() Posts: 4
Karma: 10
Join Date: Jun 2014
Device: Kindle
|
Hmm. I'm not a developer. I only noticed that my machine runs out of memory caused by a simple webserver. "Memory leak" or "extremely memory consumption"? Both is not OK.
My workaround is restarting calibre-server each morning by a cron job. That's crazy but it works... |
![]() |
![]() |
![]() |
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
calibre-server OPDS catalog - manual move to web server | HaakonME | Related Tools | 5 | 09-21-2012 03:11 AM |
500 Internal Server Error accessing content server Calibre 0.8.8 | DaddyO57 | Calibre | 1 | 07-20-2012 06:08 PM |
Memory leak in 0.8.28? | kongzifan | Calibre | 6 | 12-02-2011 09:46 AM |
Memory leak? | jhay777 | Calibre | 45 | 10-06-2011 10:18 AM |
[Old Thread] Memory leak, Worker procees closeing | theducks | Calibre | 14 | 03-30-2011 09:06 PM |