Time to create a catalog in Calibre?

crich70 · 03-19-2013, 03:47 PM

Is there a connection between how many books are in Calibre and how long it takes to create a catalog of the books? I found after discounting some duplicates in my listing that I made in spreadsheet form that I had bought some 579 books from Smashwords during the ebook week and it took 351 minutes (or 5.85 hrs) for Calibre to make the catalog file for me. So I was wondering if there is a Formula for figuring out how long it takes to create one.

GRiker · 03-19-2013, 04:15 PM

You're saying it took 351 minutes to create a catalog for 579 books? That's a very long time.

There's no formula, as it depends on your machine specs and the options you select for the catalog.

Here's a simple test you can perform to isolate what's taking so long. Build a catalog with a smaller subset of books, say 100. Create a catalog with only the 'Authors' or 'Titles' sections enabled. Note that time.

With the same set of 100 books selected, add the 'Descriptions' section. Descriptions take longer than any of the other sections. I would guess that the time to build should go up by a factor of 3-4. So if it took a minute to catalog 100 books for Titles only, I'm guessing less than 5 minutes to do Titles and Descriptions.

That test will tell you a couple of things - how long per 100 books when cataloging Titles only, and how much longer it takes when adding Descriptions. That should give you a rough idea of how long n books should take.

Another thing that could be taking a lot of time is Tags. Try disabling Genres to see how much of a difference that makes. If that is a factor, clean up your tags, or designate a custom column to use for Genre cataloging.

G

theducks · 03-19-2013, 08:07 PM

Quote:

Originally Posted by crich70

Is there a connection between how many books are in Calibre and how long it takes to create a catalog of the books? I found after discounting some duplicates in my listing that I made in spreadsheet form that I had bought some 579 books from Smashwords during the ebook week and it took 351 minutes (or 5.85 hrs) for Calibre to make the catalog file for me. So I was wondering if there is a Formula for figuring out how long it takes to create one.

IIRC The first time you use a book title in the Catalog, It has to create a thumbnail <calibre config folder>\cache\catalog\thumbs.zip
How much RAM do you have?
Was there other programs running at the time (those use some of that RAM)?

GRiker · 03-20-2013, 09:11 AM

I've added some diagnostic timing information to the jobs log which will make it possible to see how much time is being spent in the various phases of catalog creation. It should be available in the next release.

G

03-19-2013, 03:47 PM	#1
crich70 Grand Sorcerer Posts: 11,305 Karma: 43993832 Join Date: Feb 2010 Location: Monroe Wisconsin Device: K3, Kindle Paperwhite, Calibre, and Mobipocket for Pc (netbook)	Time to create a catalog in Calibre? Is there a connection between how many books are in Calibre and how long it takes to create a catalog of the books? I found after discounting some duplicates in my listing that I made in spreadsheet form that I had bought some 579 books from Smashwords during the ebook week and it took 351 minutes (or 5.85 hrs) for Calibre to make the catalog file for me. So I was wondering if there is a Formula for figuring out how long it takes to create one.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Create Catalog Error	tibia	Library Management	4	06-20-2011 06:24 AM
Create Catalog - fantastic feature !	paulfiera	Library Management	5	01-12-2011 11:09 AM
Calibre2OPDS Catalog --> Dropbox Upload Time	Mickey330	Related Tools	6	10-20-2010 04:38 PM
Problems with Create Catalog	ballast	Calibre	2	09-14-2010 06:28 AM
Adobe Teams Up With Stanza to Create Open EBook Catalog Standard	Moejoe	News	2	04-11-2009 11:12 PM

03-19-2013, 04:15 PM	#2
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	You're saying it took 351 minutes to create a catalog for 579 books? That's a very long time. There's no formula, as it depends on your machine specs and the options you select for the catalog. Here's a simple test you can perform to isolate what's taking so long. Build a catalog with a smaller subset of books, say 100. Create a catalog with only the 'Authors' or 'Titles' sections enabled. Note that time. With the same set of 100 books selected, add the 'Descriptions' section. Descriptions take longer than any of the other sections. I would guess that the time to build should go up by a factor of 3-4. So if it took a minute to catalog 100 books for Titles only, I'm guessing less than 5 minutes to do Titles and Descriptions. That test will tell you a couple of things - how long per 100 books when cataloging Titles only, and how much longer it takes when adding Descriptions. That should give you a rough idea of how long n books should take. Another thing that could be taking a lot of time is Tags. Try disabling Genres to see how much of a difference that makes. If that is a factor, clean up your tags, or designate a custom column to use for Genre cataloging. G

03-20-2013, 09:11 AM	#4
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	I've added some diagnostic timing information to the jobs log which will make it possible to see how much time is being spent in the various phases of catalog creation. It should be available in the next release. G

Advert