Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 06-24-2014, 03:55 AM   #1
mondai
Junior Member
mondai began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jun 2014
Device: Kindle
Memory leak in calibre-server?

Possibly there is a memory leak in calibre-server.

calibre-server runs on an unused netbook with 64-bit Linux, dual core, 2GB RAM and 1GB swap space. X is not running. Server load is very low (a few ebook readers).

After some days calibre-server consumes all available memory and Linux begins to trash between RAM and swap. The machine is unusable. Restarting calibre-server solves this problem immediately.

Last edited by mondai; 06-24-2014 at 04:04 AM.
mondai is offline   Reply With Quote
Old 06-24-2014, 07:10 AM   #2
PeterT
Grand Sorcerer
PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.PeterT ought to be getting tired of karma fortunes by now.
 
Posts: 13,472
Karma: 78880114
Join Date: Nov 2007
Location: Toronto
Device: Libra H2O, Libra Colour
What version of calibre and is it installed from the official calibre site or from your distribution's repository?
PeterT is offline   Reply With Quote
Advert
Old 06-24-2014, 08:30 AM   #3
mondai
Junior Member
mondai began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jun 2014
Device: Kindle
Quote:
Originally Posted by PeterT View Post
What version of calibre and is it installed from the official calibre site or from your distribution's repository?
It's the latest version 1.41.0 "Linux 64-bit binary" directly from http://download.calibre-ebook.com.

OS is latest Lubuntu 14.04 64-bit.
mondai is offline   Reply With Quote
Old 06-24-2014, 08:58 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 45,322
Karma: 27111242
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I cannot reproduce this. You will need to figure out what is leaking exactly. Monitor the memory usage of the process and make some requests from the server, see which requests are causing the increase in memory consumption, and note that the server is written in python which automatically manages memory so monitoring memory is not completely straightforward as python will not always relinquish memory to the OS completely, so to detect a leak you need to make the same request repeatedly and check that the memory usage climbs by approximately the same amount each time.
kovidgoyal is offline   Reply With Quote
Old 06-24-2014, 10:08 AM   #5
mondai
Junior Member
mondai began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jun 2014
Device: Kindle
The virtual memory size of calibre-server is very high: 2 GB. That is equal to the complete physical memory. Take a look at the output of memstat below.

Could it be that there is a problem with the CPU and the OS? The CPU is an Intel N570 Atom Processor. "Max. Memory Size" is 2 GB. Therefore it makes no sense to use additional swap space (except for hibernation).
Intel Atom Processor N570


Another strange behavior is loading a lot of X-related libraries, but X is not running.

Code:
2394708k: PID  2047 (/opt/calibre-1.41/bin/calibre-server)
  39168k(      0k): /opt/calibre-1.41/lib/libicudata.so.49 2047 2053 2047 2053 2047 2053
  29324k(  21988k): /opt/calibre-1.41/lib/libQtWebKit.so.4 2047 2053 2047 2053 2047 2053
  15616k(  10944k): /opt/calibre-1.41/lib/libQtGui.so.4 2047 2053 2047 2053 2047 2053 2047 2053
  12996k(   6620k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtGui.so 2047 2053 2047 2053 2047 2053
   8940k(   4348k): /opt/calibre-1.41/lib/libQtXmlPatterns.so.4 2047 2053 2047 2053 2047 2053 2047 2053
   7140k(   2948k): /opt/calibre-1.41/lib/libQtCore.so.4 2047 2053 2047 2053 2047 2053
   7016k(   2136k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtCore.so 2047 2053 2047 2053 2047 2053 2047 2053
   6972k(   2036k): /opt/calibre-1.41/lib/libMagickCore.so.5 2047 2053 2047 2053 2047 2053
   6320k(   2120k): /opt/calibre-1.41/lib/libicui18n.so.49 2047 2053 2047 2053 2047 2053 2047 2053
   6080k(   1688k): /opt/calibre-1.41/lib/libcrypto.so.1.0.0 2047 2053 2047 2053 2047 2053 2047 2053
   6080k(   1480k): /opt/calibre-1.41/lib/libpython2.7.so.1.0 2047 2053 2047 2053 2047 2053
   5740k(   1220k): /opt/calibre-1.41/lib/python2.7/site-packages/lxml/etree.so 2047 2053 2047 2053 2047 2053
   5700k(   1460k): /opt/calibre-1.41/lib/libicuuc.so.49 2047 2053 2047 2053 2047 2053
   5476k(   1316k): /opt/calibre-1.41/lib/libQtNetwork.so.4 2047 2053 2047 2053 2047 2053
   5472k(   1304k): /opt/calibre-1.41/lib/libxml2.so.2 2047 2053 2047 2053 2047 2053 2047 2053
   5248k(   1120k): /opt/calibre-1.41/lib/libMagickWand.so.5 2047 2053 2047 2053 2047 2053
   5240k(   1128k): /opt/calibre-1.41/lib/libglib-2.0.so.0 2047 2053 2047 2053 2047 2053
   5136k(    968k): /opt/calibre-1.41/lib/libstdc++.so.6 2047 2053 2047 2053 2047 2053
   4940k(    612k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtNetwork.so 2047 2053 2047 2053 2047 2053
   4848k(    600k): /opt/calibre-1.41/lib/python2.7/lib-dynload/unicodedata.so 2047 2053 2047 2053 2047 2053
   4824k(    696k): /opt/calibre-1.41/lib/libsqlite3.so.0 2047 2053 2047 2053 2047 2053 2047 2053
   4616k(    504k): /opt/calibre-1.41/lib/libQtDBus.so.4 2047 2053 2047 2053 2047 2053 2047 2053
   4612k(    380k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtWebKit.so 2047 2053 2047 2053 2047 2053 2047 2053
   4588k(    460k): /opt/calibre-1.41/lib/libgcrypt.so.11 2047 2053 2047 2053 2047 2053
   4560k(    432k): /opt/calibre-1.41/lib/libtiff.so.5 2047 2053 2047 2053 2047 2053
   4552k(    384k): /opt/calibre-1.41/lib/libssl.so.1.0.0 2047 2053 2047 2053 2047 2053 2047 2053
   4480k(    360k): /opt/calibre-1.41/lib/libQtSvg.so.4 2047 2053 2047 2053 2047 2053
   4480k(    272k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtXml.so 2047 2053 2047 2053 2047 2053
   4476k(    276k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtDesigner.so 2047 2053 2047 2053 2047 2053
   4428k(    244k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtOpenGL.so 2047 2053 2047 2053 2047 2053
   4404k(    276k): /opt/calibre-1.41/lib/liblcms2.so.2 2047 2053 2047 2053 2047 2053
   4404k(    268k): /opt/calibre-1.41/lib/python2.7/site-packages/apsw.so 2047 2053 2047 2053 2047 2053
   4396k(    276k): /opt/calibre-1.41/lib/libQtXml.so.4 2047 2053 2047 2053 2047 2053
   4364k(    188k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtDBus.so 2047 2053 2047 2053 2047 2053
   4360k(    248k): /opt/calibre-1.41/lib/libjpeg.so.8 2047 2053 2047 2053 2047 2053
   4348k(    236k): /opt/calibre-1.41/lib/libdbus-1.so.3 2047 2053 2047 2053 2047 2053
   4328k(    224k): /opt/calibre-1.41/lib/libxslt.so.1 2047 2053 2047 2053 2047 2053
   4296k(    144k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtXmlPatterns.so 2047 2053 2047 2053 2047 2053
   4296k(    112k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_io.so 2047 2053 2047 2053 2047 2053
   4272k(    152k): /opt/calibre-1.41/lib/libexpat.so.1 2047 2053 2047 2053 2047 2053
   4256k(    152k): /opt/calibre-1.41/lib/libpng14.so.14 2047 2053 2047 2053 2047 2053
   4240k(    104k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_ctypes.so 2047 2053 2047 2053 2047 2053
   4224k(    112k): /opt/calibre-1.41/lib/python2.7/site-packages/sip.so 2047 2053 2047 2053 2047 2053
   4200k(     80k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtSvg.so 2047 2053 2047 2053 2047 2053
   4196k(     68k): /opt/calibre-1.41/lib/python2.7/lib-dynload/datetime.so 2047 2053 2047 2053 2047 2053
   4192k(     80k): /opt/calibre-1.41/lib/libz.so.1 2047 2053 2047 2053 2047 2053
   4192k(     56k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_socket.so 2047 2053 2047 2053 2047 2053
   4188k(     76k): /opt/calibre-1.41/lib/libexslt.so.0 2047 2053 2047 2053 2047 2053
   4176k(     72k): /opt/calibre-1.41/lib/python2.7/lib-dynload/cPickle.so 2047 2053 2047 2053 2047 2053
   4176k(     40k): /opt/calibre-1.41/lib/python2.7/lib-dynload/itertools.so 2047 2053 2047 2053 2047 2053
   4164k(     60k): /opt/calibre-1.41/lib/libbz2.so.1 2047 2053 2047 2053 2047 2053
   4160k(     48k): /opt/calibre-1.41/lib/libicuio.so.49 2047 2053 2047 2053 2047 2053 2047 2053
   4160k(     40k): /opt/calibre-1.41/lib/python2.7/lib-dynload/pyexpat.so 2047 2053 2047 2053 2047 2053
   4156k(     52k): /opt/calibre-1.41/lib/python2.7/site-packages/calibre/plugins/magick.so 2047 2053 2047 2053 2047 2053
   4144k(     40k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_elementtree.so 2047 2053 2047 2053 2047 2053
   4144k(     32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/operator.so 2047 2053 2047 2053 2047 2053
   4144k(     32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/math.so 2047 2053 2047 2053 2047 2053
   4144k(     32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/array.so 2047 2053 2047 2053 2047 2053
   4140k(     28k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/QtTest.so 2047 2053 2047 2053 2047 2053
   4140k(     28k): /opt/calibre-1.41/lib/python2.7/site-packages/calibre/plugins/icu.so 2047 2053 2047 2053 2047 2053
   4140k(     28k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_struct.so 2047 2053 2047 2053 2047 2053
   4140k(     20k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_csv.so 2047 2053 2047 2053 2047 2053
   4136k(     32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_ssl.so 2047 2053 2047 2053 2047 2053
   4136k(     32k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_json.so 2047 2053 2047 2053 2047 2053
   4136k(     24k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_collections.so 2047 2053 2047 2053 2047 2053
   4132k(     28k): /opt/calibre-1.41/lib/libffi.so.4 2047 2053 2047 2053 2047 2053
   4132k(     20k): /opt/calibre-1.41/lib/python2.7/lib-dynload/zlib.so 2047 2053 2047 2053 2047 2053
   4132k(     20k): /opt/calibre-1.41/lib/python2.7/lib-dynload/strop.so 2047 2053 2047 2053 2047 2053
   4132k(     20k): /opt/calibre-1.41/lib/python2.7/lib-dynload/select.so 2047 2053 2047 2053 2047 2053
   4128k(     16k): /opt/calibre-1.41/lib/python2.7/lib-dynload/time.so 2047 2053 2047 2053 2047 2053
   4128k(     16k): /opt/calibre-1.41/lib/python2.7/lib-dynload/cStringIO.so 2047 2053 2047 2053 2047 2053
   4124k(     12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_heapq.so 2047 2053 2047 2053 2047 2053
   4120k(     16k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_locale.so 2047 2053 2047 2053 2047 2053
   4120k(     16k): /opt/calibre-1.41/lib/python2.7/lib-dynload/binascii.so 2047 2053 2047 2053 2047 2053
   4116k(     12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_random.so 2047 2053 2047 2053 2047 2053
   4116k(     12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_hashlib.so 2047 2053 2047 2053 2047 2053
   4116k(     12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_functools.so 2047 2053 2047 2053 2047 2053
   4116k(     12k): /opt/calibre-1.41/lib/python2.7/lib-dynload/fcntl.so 2047 2053 2047 2053 2047 2053
   4116k(     12k): /opt/calibre-1.41/lib/libgpg-error.so.0 2047 2053 2047 2053 2047 2053
   4116k(     12k): /opt/calibre-1.41/lib/libcalibre-launcher.so 2047 2053 2047 2053 2047 2053
   4112k(      8k): /opt/calibre-1.41/lib/python2.7/site-packages/calibre/plugins/speedup.so 2047 2053 2047 2053 2047 2053
   4112k(      8k): /opt/calibre-1.41/lib/python2.7/lib-dynload/grp.so 2047 2053 2047 2053 2047 2053
   4112k(      8k): /opt/calibre-1.41/lib/python2.7/lib-dynload/_bisect.so 2047 2053 2047 2053 2047 2053
   4108k(      4k): /opt/calibre-1.41/lib/python2.7/site-packages/PyQt4/Qt.so 2047 2053 2047 2053 2047 2053
   4108k(      4k): /opt/calibre-1.41/lib/python2.7/lib-dynload/future_builtins.so 2047 2053 2047 2053 2047 2053
   2096k(     40k): /opt/calibre-1.41/lib/ImageMagick-6.7.6/modules-Q16/coders/jpeg.so 2047
     20k(      4k): /opt/calibre-1.41/bin/calibre-server 2047 2053 2047 2053 2047 2053
mondai is offline   Reply With Quote
Advert
Old 06-24-2014, 11:27 AM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 45,322
Karma: 27111242
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
memory usage and memory leaks are very different things. Which one are you asking about?
kovidgoyal is offline   Reply With Quote
Old 06-24-2014, 11:54 AM   #7
mondai
Junior Member
mondai began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jun 2014
Device: Kindle
Hmm. I'm not a developer. I only noticed that my machine runs out of memory caused by a simple webserver. "Memory leak" or "extremely memory consumption"? Both is not OK.

My workaround is restarting calibre-server each morning by a cron job. That's crazy but it works...
mondai is offline   Reply With Quote
Reply


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
calibre-server OPDS catalog - manual move to web server HaakonME Related Tools 5 09-21-2012 03:11 AM
500 Internal Server Error accessing content server Calibre 0.8.8 DaddyO57 Calibre 1 07-20-2012 06:08 PM
Memory leak in 0.8.28? kongzifan Calibre 6 12-02-2011 09:46 AM
Memory leak? jhay777 Calibre 45 10-06-2011 10:18 AM
[Old Thread] Memory leak, Worker procees closeing theducks Calibre 14 03-30-2011 09:06 PM


All times are GMT -4. The time now is 03:37 AM.


MobileRead.com is a privately owned, operated and funded community.