Run Away Memory or Expected Results? - Page 2

kovidgoyal · 01-22-2011, 01:30 PM

Quote:

Originally Posted by paulfiera

Kovid: maybe we could improve CALIBRE_TEMP_DIR to place all the temp files in the directory defined by this env variable?

Not going to happen, those other temp files are created by libraries calibre uses, not calibre code.

paulfiera · 01-22-2011, 01:38 PM

Quote:

Originally Posted by kovidgoyal

Not going to happen, those other temp files are created by libraries calibre uses, not calibre code.

No problem at all.

Nothing you can't fix with a batch file after all

kovidgoyal · 01-22-2011, 04:00 PM

Hmm after spending all morning on this, I think I have it nailed down, after the first couple of switches, you should only see memory increases of 2-3MB per switch, which means that the in memory cache of metadata is not leaking any more. There are probably some still some small leaks left, but I don't care enough to fix those.

@GRiker: Could you test please as the last time I thought this was fixed, it wasn't. I've committed the fix.

You will see the mem increase by a lot the first couple of switches (depending on the relative sizes of the libraries), but after that it shouldn't increase by more than 2-3MB per switch.

Here is what I get switching between two similar sized libraries:

before: 805.37109375
after: 827.1015625
before: 827.1015625
after: 828.51171875
before: 828.55078125
after: 831.703125
before: 831.703125
after: 831.703125
before: 831.703125
after: 832.53125
before: 832.53125
after: 832.53125

Ignore the actual numbers (they are an artifact of how I'm measuring memory consumption), focus on the changes. Before is before the memory switch was started, after is after it was completed.

GRiker · 01-22-2011, 04:59 PM

After merging with the trunk ...
Lib 1: 235
Lib 2: 260
Lib 1: 261
Lib 2: 264
Lib 3: 266
Lib 1: 266
Lib 2: 267
Lib 3: 266
Lib 1: 268
Lib 2: 268

...

Looks pretty good.

G

DoctorOhh · 01-22-2011, 10:15 PM

Quote:

Originally Posted by kovidgoyal

You will see the mem increase by a lot the first couple of switches (depending on the relative sizes of the libraries), but after that it shouldn't increase by more than 2-3MB per switch.

Sounds a great.

kovidgoyal · 01-22-2011, 11:38 PM

Good, this is one squashed bug.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
No. of visitors expected	Manishsb	General Discussions	5	08-05-2010 11:20 AM
The Kindle Previewer expected soon	Sheikspeare	Amazon Kindle	2	07-04-2010 12:34 PM
Run Sony 505 software from memory stick?	Technophobe	Sony Reader	3	12-07-2008 03:51 PM

01-22-2011, 04:00 PM	#18
kovidgoyal creator of calibre Posts: 45,384 Karma: 27756918 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Hmm after spending all morning on this, I think I have it nailed down, after the first couple of switches, you should only see memory increases of 2-3MB per switch, which means that the in memory cache of metadata is not leaking any more. There are probably some still some small leaks left, but I don't care enough to fix those. @GRiker: Could you test please as the last time I thought this was fixed, it wasn't. I've committed the fix. You will see the mem increase by a lot the first couple of switches (depending on the relative sizes of the libraries), but after that it shouldn't increase by more than 2-3MB per switch. Here is what I get switching between two similar sized libraries: before: 805.37109375 after: 827.1015625 before: 827.1015625 after: 828.51171875 before: 828.55078125 after: 831.703125 before: 831.703125 after: 831.703125 before: 831.703125 after: 832.53125 before: 832.53125 after: 832.53125 Ignore the actual numbers (they are an artifact of how I'm measuring memory consumption), focus on the changes. Before is before the memory switch was started, after is after it was completed.

01-22-2011, 04:59 PM	#19
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	After merging with the trunk ... Lib 1: 235 Lib 2: 260 Lib 1: 261 Lib 2: 264 Lib 3: 266 Lib 1: 266 Lib 2: 267 Lib 3: 266 Lib 1: 268 Lib 2: 268 ... Looks pretty good. G

01-22-2011, 11:38 PM	#21
kovidgoyal creator of calibre Posts: 45,384 Karma: 27756918 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Good, this is one squashed bug.

Advert

Advert