New database backend - testers needed - Page 15

kovidgoyal · 08-10-2013, 11:13 PM

Quote:

Originally Posted by theducks

OK, it happened again and I tracked it down to one of 3 PI i had just used.
Generate Cover PI 1.5.12 does not seem to give it all back (even if you abort)

That's good to know, does it happen only with the new backend or with the old backend as well?

theducks · 08-11-2013, 02:29 AM

Quote:

Originally Posted by kovidgoyal

That's good to know, does it happen only with the new backend or with the old backend as well?

Yes, both
and I just noticed (regular B E) that Count Pages (posted the results to the DB) also has a leak, just smaller.

some days I must have hit the PI usage harder, so I just noticed the connection

kovidgoyal · 08-11-2013, 02:32 AM

You should probably let kiwidude know by posting in the appropriate plugin thread.

theducks · 08-11-2013, 02:47 AM

Quote:

Originally Posted by kovidgoyal

You should probably let kiwidude know by posting in the appropriate plugin thread.

You are probably right if it was not the linkage (?) that allows user PI's to run.
The bigger question: Why am I the only one noticing this?

kovidgoyal · 08-11-2013, 02:56 AM

This python. There is no linkage. Plugins run inexactly the same way as the rest of calibre's code base. As far as calibre is concerned there is no difference between a plugin and "base" calibre code once the plugin is loaded.

DoctorOhh · 08-11-2013, 03:11 AM

Quote:

Originally Posted by theducks

The bigger question: Why am I the only one noticing this?

Because you're special.

BetterRed · 08-11-2013, 03:33 AM

Quote:

Originally Posted by theducks

You are probably right if it was not the linkage (?) that allows user PI's to run.
The bigger question: Why am I the only one noticing this?

You're not, I've noticed memory leaks in calibre for a long time, never bothered me because I run it as a start/stop program. I only worry about memory leaks with programs I like to have running all the time.

BR

MartyTX · 08-11-2013, 02:31 PM

Is the "cover grid" in the thumbnail attached to Kovid's first post a feature in the new database backend or has it been around for a while?

I definitely like how it looks. How is it activated? Is it a Plugin?

Thanks,
Marty

Click image for larger version

Name: CoverGrid.jpg
Views: 276
Size: 37.9 KB
ID: 109262

Edit:
I just realized that this question has already been asked and answered. Sorry!

DoctorOhh · 08-11-2013, 10:09 PM

I would love an option to allow all covers to be cached immediately after start-up instead of having to manually scroll one step at a time to have the covers rendered and added to the cache.

It is nice to finally make use of the memory I have available. Caching my 9800+ covers increases the memory footprint to around 1.4 gigs, but it makes a seamless experience when scrolling through the cover grid.

kovidgoyal · 08-12-2013, 04:44 AM

The next build will have a disk cache for cover thumbnails. This persists between calibre restarts and stores the resized thumbnails for all covers.

This is fast enough that we can go back to rendering the covers live during scrolling, and there's no need to maintain a large in memory cache of the covers (though you still can if you want to).

EDIT: This means that the very first time you use the cover grid, it will be slow when scrolling. But after that it should be fast (of course if you change the cover size then it will be slow again the first time).

DoctorOhh · 08-12-2013, 05:16 AM

Quote:

Originally Posted by kovidgoyal

The next build will have a disk cache for cover thumbnails. This persists between calibre restarts and stores the resized thumbnails for all covers.

This is fast enough that we can go back to rendering the covers live during scrolling, and there's no need to maintain a large in memory cache of the covers (though you still can if you want to).

EDIT: This means that the very first time you use the cover grid, it will be slow when scrolling. But after that it should be fast (of course if you change the cover size then it will be slow again the first time).

Sounds good to me.

Adoby · 08-12-2013, 06:05 AM

Yes, a disk cache for the covers would be great.

I would expect it to be implemented as a subfolder inside the calibre library? Perhaps with images prescaled for the current grid size?

Since I have my calibre library on a relatively slow NAS (I know, not really recommended) with NFS, I suspect that I will immediately try to move the cover cache to my SSD, like I have done using a symlink for metadata.db.

kovidgoyal · 08-12-2013, 06:18 AM

The cache is not inside the library folder, it is in whatever folder your OS designates as being for caches, which is LOCAL_APPDATA on windows and it can be overriden via the env var CALIBRE_CACHE_DIRECTORY

And of course for calibre portable it is inside the portable directory.

DoctorOhh · 08-12-2013, 06:27 AM

The tool tip on the books includes Title and Author, could that be expanded to include series if the book is part of a series?

Adoby · 08-12-2013, 06:42 AM

Quote:

Originally Posted by kovidgoyal

The cache is not inside the library folder, it is in whatever folder your OS designates as being for caches, which is LOCAL_APPDATA on windows and it can be overriden via the env var CALIBRE_CACHE_DIRECTORY

And of course for calibre portable it is inside the portable directory.

In Linux that typically means ~/.cache will be used if CALIBRE_CACHE_DIRECTORY or XDG_CACHE_HOME haven't been set. Seems like CALIBRE_CACHE_DIRECTORY will be useful for me...

08-11-2013, 02:31 PM	#218
MartyTX Dedicated Posts: 441 Karma: 11279376 Join Date: Jun 2012 Location: Amarillo, TX Device: iPad Mini 1 & 4, Nook ST, Dell 11-3000, iPhone 5s	Cover Grid? Is the "cover grid" in the thumbnail attached to Kovid's first post a feature in the new database backend or has it been around for a while? I definitely like how it looks. How is it activated? Is it a Plugin? Thanks, Marty Edit: I just realized that this question has already been asked and answered. Sorry! Last edited by MartyTX; 08-11-2013 at 02:38 PM. Reason: Already asked and answered ... sorry!

08-12-2013, 04:44 AM	#220
kovidgoyal creator of calibre Posts: 45,373 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The next build will have a disk cache for cover thumbnails. This persists between calibre restarts and stores the resized thumbnails for all covers. This is fast enough that we can go back to rendering the covers live during scrolling, and there's no need to maintain a large in memory cache of the covers (though you still can if you want to). EDIT: This means that the very first time you use the cover grid, it will be slow when scrolling. But after that it should be fast (of course if you change the cover size then it will be slow again the first time). Last edited by kovidgoyal; 08-12-2013 at 04:57 AM.

08-12-2013, 06:05 AM	#222
Adoby Handy Elephant Posts: 1,737 Karma: 26785684 Join Date: Dec 2009 Location: Southern Sweden, far out in the quiet woods Device: Samsung Galaxy Tab S8 Ultra	Yes, a disk cache for the covers would be great. I would expect it to be implemented as a subfolder inside the calibre library? Perhaps with images prescaled for the current grid size? Since I have my calibre library on a relatively slow NAS (I know, not really recommended) with NFS, I suspect that I will immediately try to move the cover cache to my SSD, like I have done using a symlink for metadata.db. Last edited by Adoby; 08-12-2013 at 06:09 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
MTP support: Testers needed	kovidgoyal	Calibre	68	09-21-2012 03:54 AM
shared backend database?	perler	Calibre	4	01-26-2012 05:37 AM
Windows testers needed	kovidgoyal	Calibre	51	11-21-2010 05:58 AM
OS X testers needed	kovidgoyal	Calibre	23	11-18-2010 12:11 PM
calibre ereader output testers needed	user_none	Other formats	24	05-11-2009 12:00 AM

08-11-2013, 02:32 AM	#213
kovidgoyal creator of calibre Posts: 45,373 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You should probably let kiwidude know by posting in the appropriate plugin thread.

08-11-2013, 02:56 AM	#215
kovidgoyal creator of calibre Posts: 45,373 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	This python. There is no linkage. Plugins run inexactly the same way as the rest of calibre's code base. As far as calibre is concerned there is no difference between a plugin and "base" calibre code once the plugin is loaded.

08-11-2013, 10:09 PM	#219
DoctorOhh US Navy, Retired Posts: 9,897 Karma: 13806776 Join Date: Feb 2009 Location: North Carolina Device: Icarus Illumina XL HD, Kindle PaperWhite SE 11th Gen	I would love an option to allow all covers to be cached immediately after start-up instead of having to manually scroll one step at a time to have the covers rendered and added to the cache. It is nice to finally make use of the memory I have available. Caching my 9800+ covers increases the memory footprint to around 1.4 gigs, but it makes a seamless experience when scrolling through the cover grid.

08-12-2013, 06:18 AM	#223
kovidgoyal creator of calibre Posts: 45,373 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The cache is not inside the library folder, it is in whatever folder your OS designates as being for caches, which is LOCAL_APPDATA on windows and it can be overriden via the env var CALIBRE_CACHE_DIRECTORY And of course for calibre portable it is inside the portable directory.

08-12-2013, 06:27 AM	#224
DoctorOhh US Navy, Retired Posts: 9,897 Karma: 13806776 Join Date: Feb 2009 Location: North Carolina Device: Icarus Illumina XL HD, Kindle PaperWhite SE 11th Gen	The tool tip on the books includes Title and Author, could that be expanded to include series if the book is part of a series?

Advert

Advert