New Catalog features for generating EPUB/MOBI/AZW3 catalogs (0.8.64)

GRiker · 08-07-2012, 02:30 PM

Moderator Notice
For general information about calibre's Catalog feature, refer to the Creating catalogs: Start here sticky. This post specifically discusses changes to catalog generation in 0.8.64.

Catalog generation has been enhanced in 0.8.64 with the following new features:

AZW3 output is now supported.
Added a reset button to the 'Tags to exclude' edit box, restoring the default values (See callout #2)
Excluded books with multiple exclusion rules. Matching criteria can be a tag, or a custom column value from Text, Date, Yes/No, or Composite columns. All exclusion criteria are tested for each book. (See callout #3)
Prefix rules with user-selected prefix markers. Matching criteria can be a tag, or a custom column value from Text, Date, Yes/No, or Composite columns. The first matching prefix rule is used - only one prefix per book. (See callout #4)

Included sections (no changes)
- Books by Author - all books, sorted by author, presented in a list
- Books by Title - all books, sorted by title, presented in a list
- Books by Series - all series books, sorted by series, presented in a list
- Books by Genre - all enabled genres, with individual genres presented in a list sorted by Author and Series
- Recently Added - all included books sorted in reverse chronological order by date added to your library over the last 30 days, plus a month-by-month listing of added books. Presented in a list.
- Descriptions - detailed descriptions of each book, including a cover thumbnail and Comments. Description pages are sorted by author, with regular titles followed by series titles.
Excluded genres (added reset button)
A regex pattern describing tags to exclude as genres.
Tags are used to create the genre sections in your catalog. You may be using certain tags that you don't want included as genres in the catalog. For example, you might use a tag [Amazon freebie], but you don't want that tag included as a genre in your catalog. The default exclusion pattern \[.+]]|\+ excludes any tags of the form <tag> as genres, as well as excluding +, the default tag for read books.
The recycle button resets the regex pattern to the default value.
Excluded books (new feature)
Exclusion rules specify criteria for books that will not be included in the generated catalog.
The checkbox in the first column enables the rule.
Name is a rule name that you provide indicating the purpose of the rule. Field is either Tags or a custom column. Value is the content of Field to match. When an enabled rule is satisfied, the book will be excluded from the generated catalog.
Two exclusion rules have been specified in the example above:
- Rule #1, 'Catalogs' specifies that any book with a 'Catalog' tag will be excluded from the generated catalog.
- Rule #2, 'Archived books', specifies that any book with a value of 'Archived' in the custom column 'Status' will be excluded from the generated catalog.
All rules are evaluated when deciding to exclude books. Disabled or incomplete rules are ignored.
Prefixes (new feature)
Each rule specifies a matching criteria and prefix.
The checkbox in the first column enables the rule.
Name is a rule name that you provide indicating the purpose of the rule. Field is either Tags or a custom column. Value is the content of Field to match. When an enabled rule is satisfied, the book will be marked with the prefix from the Prefix column.
Three exclusion rules have been specified in the example above:
- Rule #1, 'Read book', specifies that a book with any date in a custom column 'Last read' will be prefixed with a checkmark.
- Rule #2, 'Wishlist item', specifies that any book with a 'Wishlist' tag will be prefixed with an X.
- Rule #3, 'Library books', specifies that any book with a value of True (or Yes) in a custom column 'Available in Library' will be prefixed with a double arrow.
If a book matches more than one prefix rule, the first matching rule supplies the prefix. Only one prefix is used. Disabled or incomplete rules are ignored.
Other options (no changes)
- Thumbnail width specifies a width preference for cover thumbnails included in the Descriptions section. The catalog generator caches these thumbnails for performance reasons; changing this value invalidates the cache. If you want to experiment with different widths, try building a catalog with just a few books until you've found your optimal width, then build your entire catalog. The first time a catalog is generated with a new thumbnail width will be slower, but subsequent builds will take advantage of the cache.
- Extra note specifies a custom column's contents to be inserted into the Description header, opposite the cover. For example, you might want to display the date you last read a book using a Last Read custom column described earlier. For advanced use of the Description note feature, see this post.
- Merge with Comments specifies a custom column whose content will be non-destructively merged with the Comments field during catalog generation. For example, you might have a custom colum Author Bio that you'd like to append to the Comments section. You can choose to insert the custom column contents before or after the Comments section, and optionally separate the appended content with a horizontal rule. Eligible custom column types include text, comments, and composite.

Questions, comments, feedback specific to the new features in 0.8.64 should be posted to this thread.

Tom SKP · 08-09-2012, 11:58 AM

Quote:

Originally Posted by GRiker

For general information about calibre's Catalog feature, refer to the Creating catalogs: Start here sticky. This post specifically discusses changes to catalog generation in 0.8.64.
Catalog generation has been enhanced in 0.8.64 with the following new features:.....

Wow, can't wait for 0.8.64 to come out... Thanks for the effort and all of these major changes.

GRiker · 08-09-2012, 12:18 PM

I think the changes will accomplish what you want to do. I'm going to be traveling next week, so I thought I'd dive in and get it out there while the changes were fresh in my head. I'll be monitoring the forum while traveling, so I'll keep an eye on the feedback but I won't be able to make any bug fixes until I return. Please try the 0.8.64 release as soon as it's available and you have time. If you find any problems I should be able to fix them before I leave.

G

GeekyGal · 08-10-2012, 10:59 AM

Brilliant, brilliant, brilliant! THANK YOU. 8.64's new catalog features simplified this for me so very significantly.

I can tell at a glance which books I have read, own but have not read, have a sample for (so I know that they are /were e-available) and which are not yet e-available -- and, best yet, I can do it with a custom "Status" column I was already maintaining. That allowed me to delete tags for the same information, which were duplicative additional work and created Genres I didn't want anyway. Thank you.

GRiker · 08-10-2012, 11:14 AM

Thanks for the feedback.

Sounds like you have some excellent use cases.

G

Tom SKP · 08-10-2012, 11:20 AM

Quote:

Originally Posted by GRiker

I think the changes will accomplish what you want to do. I'm going to be traveling next week, so I thought I'd dive in and get it out there while the changes were fresh in my head. I'll be monitoring the forum while traveling, so I'll keep an eye on the feedback but I won't be able to make any bug fixes until I return. Please try the 0.8.64 release as soon as it's available and you have time. If you find any problems I should be able to fix them before I leave.
G

I too am traveling (retired in an RV) and where I'm camping right now I am stuck with a very weak 1X cell signal so internet is VERY poor and intermittent. So downloading must wait for a couple of days...

GRiker · 08-10-2012, 11:22 AM

Quote:

Originally Posted by Tom SKP

I too am traveling (retired in an RV) and where I'm camping right now I am stuck with a very weak 1X cell signal so internet is VERY poor and intermittent. So downloading must wait for a couple of days...

Ah, an Escapee!

G

Tom SKP · 08-10-2012, 11:30 AM

Quote:

Originally Posted by GRiker

Ah, an Escapee!

G

More than you know!!! I am a member of the Escapees RV Club and we boondock as much as we can relying on solar and lots of batteries.
Cell internet and ebooks and public libraries and a little TV keep us in touch as we wander

GRiker · 08-10-2012, 02:22 PM

A bug has been reported with 0.8.64 - if a device is connected while building the catalog, you may experience an error. The bug has been identified, and a fix will be in the next release (0.8.65). For now, disconnect your device while building the catalog until the next release.

G

MartyTX · 08-10-2012, 05:00 PM

Super job! Thank you very much for all your work.

Before I realized how quick and easy it was to create an EPUB catalog, I was using a MANY step process to get a book catalog onto my Nook ST:

1. Create a CSV catalog
2. Open with Excel
3. Tweak, format and sort by Author, Title, Series
4. Move and copy to another sheet and sort by Title, Author, Series
5. Save finished Excel workbook
6. Convert to PDF format
7. Add to Calibre as Catalog.PDF
8. Load Catalog.PDF onto Nook

Definitely was a long and rather tedious process!

deb27 · 08-12-2012, 04:29 PM

Running 0.8.64.
Tried the new catalog feature. It fails with the following error:

Generate catalog
Python function terminated unexpectedly
u'short_name' (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 192, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 68, in gui_catalog
File "site-packages\calibre\library\catalogs\epub_mobi.py", line 178, in run
KeyError: u'short_name'

I initially tried setting exclusions and prefixes.
I ended up deleting every rule in the Excluded books section and all the rules in the Prefixes section. All that's left is:
Included sections:

all options are checked

Excluded genres:

default value remains: \[.+\]

Excluded books: empty
Prefixes: empty
Other options --I did not change these, they defaulted to:

Thumb width 1.00 in,
Extra note: DRMFree?
Merge with Comments: PersonalNotes
"Before" radio button selected
Separator is checked.

Got any suggestions on how to fix this? I love my catalog!

GRiker · 08-12-2012, 04:33 PM

Quote:

Originally Posted by deb27

Running 0.8.64.
Tried the new catalog feature. It fails with the following error:

Generate catalog
Python function terminated unexpectedly
u'short_name' (Error Code: 1)

See post #9 just above. Until 0.8.65 is release, disconnect your device before building the catalog, then reconnect. The bug is fixed in the next release.

G

deb27 · 08-12-2012, 05:53 PM

Perfect! My apologies for not seeing post #9. I disconnected the kindle and the catalog generated just fine. The prefix update is a nice feature. Thanks

nesler · 09-05-2012, 03:20 PM

Could someone post a couple screenshots as to what the catalogs look like on an e-reader? Do any readers support the ability to select books within the catalog, and immediately open the relevant book? Extremely unlikely, I know...

GRiker · 09-05-2012, 03:21 PM

None of the catalog formats support the ability to open the book on the reader.

G

08-10-2012, 05:00 PM	#10
MartyTX Dedicated Posts: 441 Karma: 11279376 Join Date: Jun 2012 Location: Amarillo, TX Device: iPad Mini 1 & 4, Nook ST, Dell 11-3000, iPhone 5s	Thanks GRiker! Super job! Thank you very much for all your work. Before I realized how quick and easy it was to create an EPUB catalog, I was using a MANY step process to get a book catalog onto my Nook ST: 1. Create a CSV catalog 2. Open with Excel 3. Tweak, format and sort by Author, Title, Series 4. Move and copy to another sheet and sort by Title, Author, Series 5. Save finished Excel workbook 6. Convert to PDF format 7. Add to Calibre as Catalog.PDF 8. Load Catalog.PDF onto Nook Definitely was a long and rather tedious process!

08-12-2012, 04:29 PM	#11
deb27 Enthusiast Posts: 43 Karma: 10 Join Date: Jan 2010 Device: Kindle 2	No luck with catalog Running 0.8.64. Tried the new catalog feature. It fails with the following error: Generate catalog Python function terminated unexpectedly u'short_name' (Error Code: 1) Traceback (most recent call last): File "site.py", line 132, in main File "site.py", line 109, in run_entry_point File "site-packages\calibre\utils\ipc\worker.py", line 192, in main File "site-packages\calibre\gui2\convert\gui_conversion.py", line 68, in gui_catalog File "site-packages\calibre\library\catalogs\epub_mobi.py", line 178, in run KeyError: u'short_name' I initially tried setting exclusions and prefixes. I ended up deleting every rule in the Excluded books section and all the rules in the Prefixes section. All that's left is: Included sections: all options are checked Excluded genres: default value remains: \[.+\] Excluded books: empty Prefixes: empty Other options --I did not change these, they defaulted to: Thumb width 1.00 in, Extra note: DRMFree? Merge with Comments: PersonalNotes "Before" radio button selected Separator is checked. Got any suggestions on how to fix this? I love my catalog!

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Mobi catalog for kindle	Noughty	Library Management	4	02-10-2012 02:31 PM
Mobi Catalog Conversion for Kindle 2	nynaevelan	Library Management	2	03-17-2011 08:00 AM
mobi catalog = periodical, ePub catalog = book	trekchick	Library Management	7	03-12-2011 01:11 PM
Still can't generate a Mobi catalog	Sydney's Mom	Library Management	11	02-02-2010 08:55 AM
Another MOBI Catalog Problem	aceflor	Calibre	1	02-01-2010 04:25 AM

08-07-2012, 02:30 PM	#1
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	New Catalog features for generating EPUB/MOBI/AZW3 catalogs (0.8.64) Moderator Notice For general information about calibre's Catalog feature, refer to the Creating catalogs: Start here sticky. This post specifically discusses changes to catalog generation in 0.8.64. Catalog generation has been enhanced in 0.8.64 with the following new features: AZW3 output is now supported. Added a reset button to the 'Tags to exclude' edit box, restoring the default values (See callout #2) Excluded books with multiple exclusion rules. Matching criteria can be a tag, or a custom column value from Text, Date, Yes/No, or Composite columns. All exclusion criteria are tested for each book. (See callout #3) Prefix rules with user-selected prefix markers. Matching criteria can be a tag, or a custom column value from Text, Date, Yes/No, or Composite columns. The first matching prefix rule is used - only one prefix per book. (See callout #4) Included sections (no changes) Books by Author - all books, sorted by author, presented in a list Books by Title - all books, sorted by title, presented in a list Books by Series - all series books, sorted by series, presented in a list Books by Genre - all enabled genres, with individual genres presented in a list sorted by Author and Series Recently Added - all included books sorted in reverse chronological order by date added to your library over the last 30 days, plus a month-by-month listing of added books. Presented in a list. Descriptions - detailed descriptions of each book, including a cover thumbnail and Comments. Description pages are sorted by author, with regular titles followed by series titles. Excluded genres (added reset button) A regex pattern describing tags to exclude as genres. Tags are used to create the genre sections in your catalog. You may be using certain tags that you don't want included as genres in the catalog. For example, you might use a tag [Amazon freebie], but you don't want that tag included as a genre in your catalog. The default exclusion pattern \[.+]]\|\+ excludes any tags of the form <tag> as genres, as well as excluding +, the default tag for read books. The recycle button resets the regex pattern to the default value. Excluded books (new feature) Exclusion rules specify criteria for books that will not be included in the generated catalog. The checkbox in the first column enables the rule. Name is a rule name that you provide indicating the purpose of the rule. Field is either Tags or a custom column. Value is the content of Field to match. When an enabled rule is satisfied, the book will be excluded from the generated catalog. Two exclusion rules have been specified in the example above: Rule #1, 'Catalogs' specifies that any book with a 'Catalog' tag will be excluded from the generated catalog. Rule #2, 'Archived books', specifies that any book with a value of 'Archived' in the custom column 'Status' will be excluded from the generated catalog. All rules are evaluated when deciding to exclude books. Disabled or incomplete rules are ignored. Prefixes (new feature) Each rule specifies a matching criteria and prefix. The checkbox in the first column enables the rule. Name is a rule name that you provide indicating the purpose of the rule. Field is either Tags or a custom column. Value is the content of Field to match. When an enabled rule is satisfied, the book will be marked with the prefix from the Prefix column. Three exclusion rules have been specified in the example above: Rule #1, 'Read book', specifies that a book with any date in a custom column 'Last read' will be prefixed with a checkmark. Rule #2, 'Wishlist item', specifies that any book with a 'Wishlist' tag will be prefixed with an X. Rule #3, 'Library books', specifies that any book with a value of True (or Yes) in a custom column 'Available in Library' will be prefixed with a double arrow. If a book matches more than one prefix rule, the first matching rule supplies the prefix. Only one prefix is used. Disabled or incomplete rules are ignored. Other options (no changes) Thumbnail width specifies a width preference for cover thumbnails included in the Descriptions section. The catalog generator caches these thumbnails for performance reasons; changing this value invalidates the cache. If you want to experiment with different widths, try building a catalog with just a few books until you've found your optimal width, then build your entire catalog. The first time a catalog is generated with a new thumbnail width will be slower, but subsequent builds will take advantage of the cache. Extra note specifies a custom column's contents to be inserted into the Description header, opposite the cover. For example, you might want to display the date you last read a book using a Last Read custom column described earlier. For advanced use of the Description note feature, see this post. Merge with Comments specifies a custom column whose content will be non-destructively merged with the Comments field during catalog generation. For example, you might have a custom colum Author Bio that you'd like to append to the Comments section. You can choose to insert the custom column contents before or after the Comments section, and optionally separate the appended content with a horizontal rule. Eligible custom column types include text, comments, and composite. Questions, comments, feedback specific to the new features in 0.8.64 should be posted to this thread. Last edited by GRiker; 08-09-2012 at 03:31 PM.

08-09-2012, 12:18 PM	#3
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	I think the changes will accomplish what you want to do. I'm going to be traveling next week, so I thought I'd dive in and get it out there while the changes were fresh in my head. I'll be monitoring the forum while traveling, so I'll keep an eye on the feedback but I won't be able to make any bug fixes until I return. Please try the 0.8.64 release as soon as it's available and you have time. If you find any problems I should be able to fix them before I leave. G

08-10-2012, 10:59 AM	#4
GeekyGal Enthusiast Posts: 40 Karma: 13888 Join Date: Sep 2010 Device: Kindle FireHD, Kindle3G, iPad, MacBook, iRex Illiad (ret)	Brilliant, brilliant, brilliant! THANK YOU. 8.64's new catalog features simplified this for me so very significantly. I can tell at a glance which books I have read, own but have not read, have a sample for (so I know that they are /were e-available) and which are not yet e-available -- and, best yet, I can do it with a custom "Status" column I was already maintaining. That allowed me to delete tags for the same information, which were duplicative additional work and created Genres I didn't want anyway. Thank you.

08-10-2012, 11:14 AM	#5
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	Thanks for the feedback. Sounds like you have some excellent use cases. G

08-10-2012, 02:22 PM	#9
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	A bug has been reported with 0.8.64 - if a device is connected while building the catalog, you may experience an error. The bug has been identified, and a fix will be in the next release (0.8.65). For now, disconnect your device while building the catalog until the next release. G

08-12-2012, 05:53 PM	#13
deb27 Enthusiast Posts: 43 Karma: 10 Join Date: Jan 2010 Device: Kindle 2	Perfect! My apologies for not seeing post #9. I disconnected the kindle and the catalog generated just fine. The prefix update is a nice feature. Thanks

09-05-2012, 03:20 PM	#14
nesler Zealot Posts: 107 Karma: 548458 Join Date: Dec 2010 Location: California Device: Too many to list	Could someone post a couple screenshots as to what the catalogs look like on an e-reader? Do any readers support the ability to select books within the catalog, and immediately open the relevant book? Extremely unlikely, I know...

09-05-2012, 03:21 PM	#15
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	None of the catalog formats support the ability to open the book on the reader. G

Advert

Advert