Creating Catalogs (Archived)

[kovidgoyal]

and what template are you using?

[theducks]

Originally Posted by kovidgoyal

and what template are you using?

My current Save to disk template setting I use this for books with Series)

{series[0]}-series/{series}/{series_index:0>2s} {title}_{id}_{authors}

Saving a regular Series book to "A Single Directory" C:\Temp OK
Saving a regular NON-Series book to "A Single Directory" C:\Temp OK
Saving "My_Test" Catalog to "A Single Directory" C:\Temp WORKS OK
Doing a Normal "My_Test" Catalog Save To Disk C:\TEMP FAILS

[kovidgoyal]

open a ticket for it, chaley is currently away, when he gets back, he'll look at this.

[theducks]

Originally Posted by kovidgoyal

and what template are you using?

#7516 Created.
Thanks

[GRiker]

Some noteworthy enhancements to EPUB/MOBI catalog generation as of 0.7.29:

• The E-book options tab now includes a 'Wishlist' tag. Books tagged as wishlist items will be displayed with an 'x', similar to the way that read books are displayed with a check mark in the catalog.
  This feature should be useful to keep track of books you wish to add to your collection. Create an empty book using 'Add books', then add the wishlist tag (by default, 'Wishlist'). Populate the metadata as you would normally.
  Wishlist items are shown in generated catalogs with a leading 'x', indicating that they are missing from your collection.
• The description page layout has been updated. The cover thumbnail is about 10% larger, and the Publisher/Published/Rating layout has been simplified. Covers are rendered with a drop shadow (via CSS) on iDevices.
• I've implemented thumbnail caching, reducing the amount of time it takes to build a catalog. You won't notice the improvement until after the first time you've generated a catalog under 0.7.29+, which creates the initial cache. I'm observing improvements of about 35%. Your mileage may vary.

G

[pchrist7]

Originally Posted by GRiker

Some noteworthy enhancements to EPUB/MOBI catalog generation as of 0.7.29:

• The E-book options tab now includes a 'Wishlist' tag. Books tagged as wishlist items will be displayed with an 'x', similar to the way that read books are displayed with a check mark in the catalog.
  This feature should be useful to keep track of books you wish to add to your collection. Create an empty book using 'Add books', then add the wishlist tag (by default, 'Wishlist'). Populate the metadata as you would normally.
  Wishlist items are shown in generated catalogs with a leading 'x', indicating that they are missing from your collection.
• The description page layout has been updated. The cover thumbnail is about 10% larger, and the Publisher/Published/Rating layout has been simplified. Covers are rendered with a drop shadow (via CSS) on iDevices.
• I've implemented thumbnail caching, reducing the amount of time it takes to build a catalog. You won't notice the improvement until after the first time you've generated a catalog under 0.7.29+, which creates the initial cache. I'm observing improvements of about 35%. Your mileage may vary.

G

Can't wait to try it out !
Would be sometime Saturday i hope.
And it'll probably end with

[msr]

I'm getting Memory errors with large libraries (~15,000+ books) when trying to generate either epub or mobi catalogs. The end part of the output is:

<code>
Initial parse failed:
Parsing file 'content/book_15985.html' as HTML
Forcing content/book_15985.html into XHTML namespace
Stripping comments and meta tags from content/book_15985.html
Merging user specified metadata...
Detecting structure...
Flattening CSS and remapping font sizes...
Property: Unknown Property name. [1:1: mso-fareast-font-family]
Property: Unknown Property name. [1:1: mso-fareast-font-family]

[note: this is repeated a lot of times]

Property: Unknown Property name. [1:1: mso-fareast-font-family]
Property: Unknown Property name. [1:1: mso-bidi-font-style]
Property: Unknown Property name. [1:1: mso-bidi-font-style]
Exception MemoryError: in <generator object gen at 0x769E43E8> ignored
Exception MemoryError: in <generator object __iter__ at 0x23EC6A58> ignored
Python function terminated unexpectedly
(Error Code: 1)
Traceback (most recent call last):
File "site-packages\calibre\ebooks\oeb\transforms\flatcss.py" , line 122, in __
call__
File "site-packages\calibre\ebooks\oeb\stylizer.py", line 191, in __init__
File "site-packages\cssutils-0.9.7b4-py2.7.egg\cssutils\util.py", line 738, in
namespaces
MemoryError
</code>

This occurs whether I include tags or exclude all of them. I can create a csv catalog without problems. I've also created smaller catalogs without errors, including up to about 8000 books. I believe I've created subsets with every book in my library, so I don't believe it is a specific book causing the problem, rather simply the large overall number. Any other possibilities or anything I can try to create a single catalog?

[GRiker]

Originally Posted by msr

I'm getting Memory errors with large libraries (~15,000+ books) when trying to generate either epub or mobi catalogs. The end part of the output is:

<code>
Initial parse failed:
Parsing file 'content/book_15985.html' as HTML
...
Property: Unknown Property name. [1:1: mso-fareast-font-family]
Property: Unknown Property name. [1:1: mso-fareast-font-family]
...
</code>

It looks like one of your books has comments formatted in Microsoft Word; perhaps the entire text of the book is in the comments field.

Try this:

• Preferences|Interface|Add your own columns
• Click 'Add custom column'
  • Lookup name: cid
  • Column heading: Calibre ID
  • Column type: Column built from other columns
  • Template: {id:0>5s}
• Click OK, restart calibre
• Isolate the book where the problem is occurring by searching for the calibre ID in the search box:
  

  #cid:15985

• Edit/remove the comments for that book

[GRiker]

The method for marking books as read has been enhanced with 0.7.32. The E-book options tab of the Generate catalog dialog for EPUB/MOBI now features a dropdown list of custom columns from which a book's read status may be inferred.

The legacy method of using a tag of + is still supported as the default.

You now have additional options for marking a book as read by referencing custom columns. Eligible custom column types:

• Yes/No (checkmark)
• Column built from other columns
• Date
• Text, column shown in the tag browser

To create your specification marking books as read, select the custom column (or Tag) from the drop-down list, then type the matching text in the edit field.

Examples:

1. Tag: The default method using a tag of + marking a book as read still works.
   For those books you wish marked as read in your catalog, add the tag +.
   When you create your catalog, select Tag in the combo box, type + in the text edit field.
2. Text: Create a custom column of type Text, column shown in the tag browser, with a Lookup name of read, and a Column heading of Read. Exit calibre, restart to apply the custom column changes.
   For those books you wish marked as read in your catalog, add Yes in the Read custom column.
   When you create your catalog, select Read from the combo box, and type Yes in the text field.
3. Checkmark: Create a custom column of type Yes/No, with a lookup name of read, and a Column heading of Read. Exit calibre, restart to apply the custom column changes.
   For those books you wish marked as read in your catalog, mark the books with a checkmark (or type Y) in the Read custom column.
   When you create your catalog, select Read from the combo box, and type True in the text field.
4. Date: Create a custom column of type Date, with a lookup name of last_read, and a Column heading of Last Read. Exit calibre, restart to apply the custom column changes.
   For those books you wish marked as read in your catalog, add the date you finished reading the book in the Last Read custom column.
   When you create your catalog, select Last Read from the combo box, and type .* in the text field. This regex pattern recognizes any non-empty field as being read.
5. Composite: Create a custom column of type Column built from other columns, with a lookup name of read, and a Column heading of Read. The template describes how to infer a read book from another column's contents. For example, using the Last Read column from example #4, the template below marks the Read column with a check mark when it finds anything in #last_read.
   

   {#last_read:test(✓,)}

   Exit calibre, restart to apply the custom column changes.
   When you create your catalog, select Read from the combo box, and type .* in the text field. This regex pattern recognizes any non-empty field as being read.

G

[Malcolm]

@GRiker:
My Catalog creation is no longer being made available in 'Periodical' format for my KDX. I have the profiles set for Kindle DX and the catalog option set for Mobi but I can't seem to get the catalog to show in any way but alpha. I'm running calibre 7.35 on a Win7 PC.

While I only have about 200 ebooks it makes the catalog lookup very tedious in alpha only form. I know I must be doing something wrong but all the posts indicate I'm set as you have told us.

Thanks for your help.

[GRiker]

Originally Posted by Malcolm

@GRiker:
My Catalog creation is no longer being made available in 'Periodical' format for my KDX. I have the profiles set for Kindle DX and the catalog option set for Mobi but I can't seem to get the catalog to show in any way but alpha. I'm running calibre 7.35 on a Win7 PC.

While I only have about 200 ebooks it makes the catalog lookup very tedious in alpha only form. I know I must be doing something wrong but all the posts indicate I'm set as you have told us.

Thanks for your help.

You will need to create a ticket and attach a debug trace to it.

• Capturing a debug trace
  
  • Click the Jobs: icon in the lower right corner of calibre's main window
  • In the 'calibre - jobs' dialog window, click on the most recent (closest to the top) 'Generate catalog' job
  • Click the Show job details button
  • Select all of the text in the 'Generate catalog' window, and copy it to your clipboard. Save it to a file 'debug.txt'

• Creating a ticket
  Go to calibre's project management site to report a bug. Attach the 'debug.txt' file to the ticket. Set the component field to 'Catalogs'.

There's probably a book with a character in the title that's confusing the indexer, but I need to see the debug trace to be certain.

G

[Loosheesh]

I'm not sure if this has been asked and answered already (I didn't read all the posts ) but I'm wondering if I can include custom metadata in a catalog (CSV format). I created a column named 'Cost' and wanted that output in the Catalog - is it possible? Thanks in advance.

[GRiker]

Originally Posted by Loosheesh

I'm not sure if this has been asked and answered already (I didn't read all the posts ) but I'm wondering if I can include custom metadata in a catalog (CSV format). I created a column named 'Cost' and wanted that output in the Catalog - is it possible? Thanks in advance.

That feature will be included in the next release (0.7.36)

G

[Loosheesh]

Thanks GRiker! I'm looking forwarding to the new release then

[GRiker]

Major enhancements to Catalog generation in 0.7.36:

• CSV/XML output formats now support custom columns
• Shiny new UI for EPUB/MOBI options:
  
• New EPUB/MOBI output options:
  • A minimal catalog is now a 'Books by Author' section. All other sections are optional.
  • An additional option for excluding books based on custom column criteria has been added.
  • Added a thumbnail width preference. Changing thumb width invalidates the cover cache, slowing the first catalog built with the new width. Subsequent builds take advantage of the updated cache.
  • Deprecated support for the special '*' note tag, replaced with Description header note option, derived from custom column.
  • Added 'Merge with Comments' feature, non-destructively combining Comments with a custom column.
  • Moved the Description HTML into a template file which can be modified, along with the catalog's CSS.
  • Improved management of widows and orphans in various lists.

See the first post in this thread for a current overview of generating catalogs.

G