06-11-2011, 04:40 PM | #1 |
Groupie
Posts: 178
Karma: 134
Join Date: May 2010
Device: IREX DR1000
|
Out of memory error in the latest version
Hello,
I was renaming books. I have renamed about 100 of them and then an error popped up: calibre, version 0.8.5 ERROR: Unhandled exception: <b>MemoryError</b>ut of memory Traceback (most recent call last): File "site-packages\calibre\gui2\library\models.py", line 901, in setData File "site-packages\calibre\library\database2.py", line 1784, in set File "site-packages\calibre\library\database2.py", line 2083, in set_title File "site-packages\calibre\library\database2.py", line 611, in set_path File "shutil.py", line 51, in copyfileobj File "tempfile.py", line 595, in write MemoryError: out of memory |
06-11-2011, 04:59 PM | #2 |
creator of calibre
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
Were you renaming a book with a particularly large file?
|
06-11-2011, 05:04 PM | #3 |
Groupie
Posts: 178
Karma: 134
Join Date: May 2010
Device: IREX DR1000
|
Yes, the book wich crashed calibre was 84,6 MB in size.
|
06-12-2011, 10:37 AM | #4 |
creator of calibre
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
And how much RAM do you have on your computer?
|
06-12-2011, 11:00 AM | #5 |
Groupie
Posts: 178
Karma: 134
Join Date: May 2010
Device: IREX DR1000
|
It is a 64Bit machine Dual Processor 8GB
|
06-12-2011, 12:01 PM | #6 |
creator of calibre
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
That's weird. I cannot replicate this with a 90MB file on Win 7. And that traceback is indicating an out of memory error while copying from disk to memory, which cannot happen for only an 80MB file when you have several GB of RAM.
I'm at a loss. |
06-12-2011, 03:22 PM | #7 |
creator of calibre
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
I have reduced the max size stored in memory during the file copy, see if that makes a difference in the next release.
|
06-12-2011, 03:31 PM | #8 |
creator of calibre
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
I think I know what's going on...stay tuned.
|
06-12-2011, 04:07 PM | #9 |
creator of calibre
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
|
06-22-2011, 08:26 AM | #10 |
Member
Posts: 10
Karma: 10
Join Date: Jun 2011
Device: Kindle3
|
I am running into something similar. After downloading metadata for several books (as few as 6) Windows tells me that I need to close Calibre because I am running out of memory.
I watched the working set memory in TaskMan go up for the calibre.exe process each time I edited the metadata for a book (40-30mb each time). It seems to be more dramatic if I edit books rapidly, one after another. Could there be a memory leak here? System Config Windows 7 Enterprise (x86) 2GB RAM Calibre 0.8.6 I was hoping the debug log would gather more information. calibre Debug log calibre 0.8.6 Windows-7-6.1.7600 Windows ('Windows', '7', '6.1.7600') Python 2.7.1 Windows: ('7', '6.1.7600', '', 'Multiprocessor Free') Starting up... Started up in 18.0480000973 Thanks, BradV |
06-22-2011, 09:29 AM | #11 |
Grand Sorcerer
Posts: 11,741
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
I tried an experiment using Win7 Pro/32, 4GB, calibre 0.8.6, 2000-book test library on C:, system on C:, calibre on C:. Memory numbers are what windows reports as "Working set".
Mem at start: 104MB. Open edit metadata/single (default window layout): 113MB Hit next button 20 times (commits the dialog): 128MB Close edit metadata: 128MB E/M a book and hit escape 20 times: 132MB E/M a book, hit escape, hit down-arrow 20 times: 132MB Select 20 books, EM Bulk, press OK: 132MB At the end of this, Peak Working Set was 161MB. There was a report that DEP (Data Execution Protection) made a difference. Mine is off. I turned it on and ran a few experiments, and saw no significant difference in memory use. It feels subjectively slightly faster without DEP, but that could be my imagination. Bottom line: I see no evidence of a significant memory leak. Are you doing something very different from what I did? |
06-22-2011, 04:31 PM | #12 |
Member
Posts: 10
Karma: 10
Join Date: Jun 2011
Device: Kindle3
|
Thanks for trying this Chaley. It looks like you were seeing a small leak as well. You went from 104mb to 132mb of working set. I think the issue is related to the number of books in my library. I was able to repro the issue on a second computer (using the same database).
Calibre 0.8.6 9268 book library on E: OS on C:, Calibre on C: Win7 Enterprise x86 (32-bit), 2gb RAM Numbers are the “Working Set (Memory)” as reported by Task Manager Repro Steps Start Calibre (229,260k) Select book, Edit Metadata (245,132k) Download Metadata (257,778k) Select 1st entry, Next Select current cover (256,320k) Click OK (285,672k) Select book, edit metadata (296,580k) Download metadata (305,888k) Select 1st entry, Next Select current cover (306,004k) Click OK (336,568k) That is over 100mb of working set memory by editing (downloading metadata for) 2 books. When I attempted to repro the issue on a smaller book library (117 entries) I did not see anything this dramatic, but I did see a small amount of Working Set (Memory) increase with each edit. Thanks, BradV |
06-22-2011, 04:58 PM | #13 | ||||
Grand Sorcerer
Posts: 11,741
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Quote:
Quote:
Quote:
There is clearly something different with your edit metadata. In every one of my tests on my three librarys, the maximum increase was 4 MB, and was often zero. You are seeing 16MB. Are you using the default 2-tab edit metadata window? I don't think there is any difference, but one never knows. Are your covers particularly large (mine are all 500x500 or smaller)? If so, then memory must be allocated to display them, and this isn't an error. We have another difference. I never use download metadata. and my tests didn't use it. There could be a leak in there, but that experiment will need to wait for another developer who uses it. I also didn't change covers. which might also account for it. |
||||
06-22-2011, 09:00 PM | #14 |
Member
Posts: 10
Karma: 10
Join Date: Jun 2011
Device: Kindle3
|
Yes, I am using the default Edit Metadata window.
Some covers are large, some are small (less than 200x300). The drive is a USB hard drive (local). I will file a bug. Thanks! BradV |
06-22-2011, 09:08 PM | #15 |
creator of calibre
Posts: 43,856
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
No point filing a bug, without being able to replicate your memory leak, it is impossible to fix. I tried it on my Win 7 VM and after the initial jump of ~30MB the first time the dialog is opened and metadata downloaded, subsequent clicking of next or OK and opening again with/without metadata download or editing covers shows no change larger than a couple of MB in memory consumption.
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
How do I update Calibre to the latest version? | nsivakr | Calibre | 18 | 02-22-2017 11:07 AM |
Kindle 3G 6" Latest gen. memory is 256MB or 2GB? | dagero | Kindle Developer's Corner | 8 | 11-11-2010 04:20 PM |
is there virus in the latest version | ichthyosaur | Calibre | 1 | 01-19-2010 03:58 PM |
latest version for Ubuntu | Lago | Calibre | 7 | 01-15-2010 01:54 PM |
Guardian Recipe messed up in latest version?? | pars_andy | Calibre | 6 | 11-14-2009 03:50 PM |