Creating Catalogs (Archived) - Page 27

alexxx · 01-14-2011, 02:20 AM

Hi people,
I am trying to create a catalog just for the books that on my Kindle are in a given collection ("TO READ").
The problem is that I can filter them in the Device panel by this search: Collections:"TO READ", but in the Device panel I cannot create catalogs (of course, "Convert Books" is grayed out).
While instead in the Library panel I cannot filter for those books, since the Collections column doesn exist.
There is some way to filter - in the Library panel - for books that on the device are on a given collection?
... I hope I've been clear ...

thanks!

alessandro

GRiker · 01-14-2011, 09:31 AM

Quote:

Originally Posted by alexxx

Hi people,
I am trying to create a catalog just for the books that on my Kindle are in a given collection ("TO READ").
The problem is that I can filter them in the Device panel by this search: Collections:"TO READ", but in the Device panel I cannot create catalogs (of course, "Convert Books" is grayed out).
While instead in the Library panel I cannot filter for those books, since the Collections column doesn exist.
There is some way to filter - in the Library panel - for books that on the device are on a given collection?

According to @chaley, the expert on such matters:

Quote:

There isn't an easy way to do this.

The only way I can think of is to write a plugin that copied the device collections field to a cust col in the library. The plugin would traverse the device's database (the one in the device gui view) grabbing the appid (db id) and the collections for books with in-library ==true. It would next find the books in the library with the appids and insert the info into a cust col.

Not the answer you're looking for, but that's the authoritative answer.

G

alexxx · 01-14-2011, 10:54 AM

Quote:

Originally Posted by GRiker

According to @chaley, the expert on such matters:

Not the answer you're looking for, but that's the authoritative answer.

G

that's a pity, but I thank you anyway for the info!

alessandro

flo_tm · 01-14-2011, 02:10 PM

Quote:

Originally Posted by GRiker

If you're referring to the year of publication appended to the book title in the 'Authors' list, it is always included. If you want a switch for that, create an enhancement request.

...

I need an example of the incorrect date.

Actually, that was the date I meant - I'll see about opening an enhancement request.

And sorry, I wasn't clear enough - the date is only "incorrect" for me because I either don't update the metadata and it is set by default or because the downloaded metadata contains not the date of the first publication, but for example the date of a current publication.

My familiarity with html and css is definitely not good enough - but I'm getting better. So thanks again for the detailed instructions !

Best regards,
Florianm

GRiker · 01-14-2011, 02:34 PM

As of 0.7.39, it's possible to do some fancy manipulations of a book's metadata to generate text that can be added to the Descriptions pages of your catalogs.

Let's say that you use the tags 'NYT #1 Fiction' and 'NYT 10 Best of the Year'. What we'd like to do is create a custom column 'Awards' whose content will be 'NYT 10 Best of <year of publication>', or 'NYT #1 <year of publication>', which can then be inserted into the Description page opposite the cover.

The top oval contains links to all books in your catalog tagged with 'NYT #1 Fiction' or 'NYT 10 Best of the Year'. The bottom oval is the result of following the steps in this post, which generate the Awards section from the tags.

Step 1: Create a custom column 'Awards'

Open Preferences|Interface|Add your own columns
Click Add custom column
Lookup name: awards
Column heading: Awards
Column type: Column built from other columns
Template: program:'' (we will add the template in Step 3)
Click OK to close the dialog
Click Apply to close the Add your own columns dialog
Click Close to close the Preferences dialog.

Step 2: Create the template function

Open Preferences|Advanced|Template Functions

Copy the code below, paste it into the Program Code window.

Code:

def evaluate(self, formatter, kwargs, mi, locals, *args):
  from calibre.utils.date import format_date
  SEPARATOR = u' \u00B7 '   # Unicode representation of &middot;
  awards = {}
  for arg in args:
    strings = arg.split(':')
    awards[strings[0]] = strings[1]
  a_list = [awards[t] + format_date(kwargs.get('pubdate'), u' yyyy') for t in kwargs.get('tags') if t in awards]
  return SEPARATOR.join(a_list)

In the Function field, type award_date
Set Arg count to -1, representing a variable number of arguments
Click Create to add the award_date function
Click Apply, then close the Preferences dialog

Step 3: Add the template to the custom column

Double-click any book's cell in the Awards column to open the Edit template dialog window.

Paste the following code into the edit box:

Code:

program:award_date(
'NYT 10 Best of the Year:NYT 10 Best of',
'NYT #1 Fiction:NYT #1',
'NYT #1 Nonfiction:NYT #1')

Click OK to close the Edit template window

Step 4: Create the catalog, adding Awards to the Description pages

Click Convert Books↓Create catalog of books in your calibre library
Switch to the E-book options tab
In Other options|Description note, select Awards
Catalogs created with these settings will show matching books awards in the Description notes section, opposite the cover

An even more advanced approach: per-tag processing

Spoiler:

Thanks to @chaley for adding python functionality to the template feature making this possible, and his assistance in developing the sample code.

Learn more about calibre's template language feature
Discuss other uses of calibre's template language feature

G

N.B. The year added to the award name is actually the year of the book's publication, which is almost always the same year the book won the award. If you're more OCD than I am, and this bothers you, create a custom column dedicated to awards.

GRiker · 01-14-2011, 03:58 PM

I've had a number of requests to disable the year of publication that's appended to the title in various section lists. Currently, the implementation is inconsistent:

Books by Author: Yes
Books by Title: No
Books by Series: Yes
Books by Genre: No
Recently Added: No
Descriptions: Yes (template can be edited to remove/modify)

I'm seeking input on how this might be presented in the Generate catalog|E-book options dialog. I envision three possible scenarios:

Offer a global 'Append date of publication' checkbox. This would append the date after the title in all enabled sections, or none at all.
Offer per-section checkboxes.
Don't change anything, it's perfect the way it is.

Please reply in this thread if you have an opinion about this possible enhancement.

G

flo_tm · 01-15-2011, 07:48 PM

Regarding the three scenarios, I would prefer #2 over #1 over #3.

But a hard coded "ignore" for "undefined pub_date" would also be a good first step.

Flo

GRiker · 01-15-2011, 08:03 PM

Quote:

Originally Posted by flo_tm

But a hard coded "ignore" for "undefined pub_date" would also be a good first step.

That patch is already in place for the next release.

G

flo_tm · 01-15-2011, 08:05 PM

And since I just read a rant by the developer of Sigil regarding the bloating of opensource software because every user wants his little speciality ..
(can we have the button to turn off the pub_date in pink, please ? =))

Instead of creating separate checkboxes only for the publication date, would it be possible to create templates for the "assembly" of the section lines ? Please ignore me if I'm naive here, I have no idea how complex the software is and how much work would be involved.

Example: Section "By Title"

Default:
(title) (series) (series_index) - (author)

Other Possibilities:
(title) - (author) (pub_year)
(title) (series) (series_index) - (author) (rating)
(title) - (author) (user defined tag)

Of course this would also mean that a lot of bimbos (like me) start asking stupid questions why their templates don't work ...

flo_tm · 01-15-2011, 08:23 PM

Quote:

Originally Posted by GRiker

That patch is already in place for the next release.

G

This is great !

(You also wrote that in the ticket and I missed it ...)
I'm probably a nitpicker, but I anticipate the next release because of this.

Btw, using 01 0000 in the GUI column to set publication date to undefined works, but I can't get it to work in the metadata editor with Jan 0000.
(Only setting it with a lot of fiddling in the drop down menu date fields to 01.01.101 works).

But thats ok, since editing in the column is faster anyway.

theducks · 01-15-2011, 08:44 PM

Quote:

Originally Posted by flo_tm

This is great !

(You also wrote that in the ticket and I missed it ...)
I'm probably a nitpicker, but I anticipate the next release because of this.

Btw, using 01 0000 in the GUI column to set publication date to undefined works, but I can't get it to work in the metadata editor with Jan 0000.
(Only setting it with a lot of fiddling in the drop down menu date fields to 01.01.101 works).

But thats ok, since editing in the column is faster anyway.

in the meta data editor, there are 2 0r 3 (non-obvious) fields in the date box.
Jan 2011 are 2 separate sections. But setting the 2011 to 0000 or 101 does not un-assign Publisher. Changing (tweak) the date format to dd MMM yyyy does not help in the Editor.

Fashtas · 01-16-2011, 05:20 PM

I'm having some issues generating a catalog. I've generated one before with (almost) the same set of books, but added some more and changed a few tags and now I am getting the following error:

ValueError: Cannot open image from empty data string

Unlike the issues with Inconsistent Author Sort values (which I had before) there is no sign that I can see of what book is causing the issue

The only thing I haven't tried was the recommended check for Tag problems (The quick way to confirm this is to change the regex expression in the EPUB/MOBI options tab) because I can't find any setting tab or screen called "EPUB/MOBI Options"

This is using 0.7.38. I had tried 39, with the obvious problems, then went to 40, which had errors uploading to the Kindle, so rolled back to 38 because I needed to fill a Kindle before an overseas trip

Spoiler:

GRiker · 01-16-2011, 05:59 PM

That's a bug that was fixed in 0.7.40, so you'll have to wait until your kindle issue is resolved, or identify the book with the bad cover. You have a cover that's just an empty string. You could try typing 'cover:true' in the search box, then paging through the titles until you find the title(s) that don't actually have a cover.

G

GRiker · 01-17-2011, 06:06 PM

@Fashtas, 0.7.41 will display a dialog when it encounters a bad cover, telling you which book has the problem.

G

Fashtas · 01-17-2011, 07:07 PM

Thanks for the response!

I can confirm that 0.7.40 worked in creating the catalog. The error I saw uploading to the kindle previously also didn't reappear, I may have been confused with .39 given I was installing/uninstalling several times

The catalogs created weren't quite what I was expecting, not tying into the Kindle collection system, but rather being a book in it's own right with no way (it seems) to "click" one and go to the book itself.

After some research I've gone with using an app called the Kindle Collection Manager to edit collections on the device, and making one Catalog with Calibre per collection (or for the larger collections anyway) which I can browse in each collection folder for more information on the books in it

Works nicely so far!

01-14-2011, 02:34 PM	#395
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	New OCD features (0.7.39) As of 0.7.39, it's possible to do some fancy manipulations of a book's metadata to generate text that can be added to the Descriptions pages of your catalogs. Let's say that you use the tags 'NYT #1 Fiction' and 'NYT 10 Best of the Year'. What we'd like to do is create a custom column 'Awards' whose content will be 'NYT 10 Best of <year of publication>', or 'NYT #1 <year of publication>', which can then be inserted into the Description page opposite the cover. The top oval contains links to all books in your catalog tagged with 'NYT #1 Fiction' or 'NYT 10 Best of the Year'. The bottom oval is the result of following the steps in this post, which generate the Awards section from the tags. Step 1: Create a custom column 'Awards' Open Preferences\|Interface\|Add your own columns Click Add custom column Lookup name: awards Column heading: Awards Column type: Column built from other columns Template: program:'' (we will add the template in Step 3) Click OK to close the dialog Click Apply to close the Add your own columns dialog Click Close to close the Preferences dialog. Step 2: Create the template function Open Preferences\|Advanced\|Template Functions Copy the code below, paste it into the Program Code window. Code: def evaluate(self, formatter, kwargs, mi, locals, args): from calibre.utils.date import format_date SEPARATOR = u' \u00B7 ' # Unicode representation of · awards = {} for arg in args: strings = arg.split(':') awards[strings[0]] = strings[1] a_list = [awards[t] + format_date(kwargs.get('pubdate'), u' yyyy') for t in kwargs.get('tags') if t in awards] return SEPARATOR.join(a_list) In the Function* field, type award_date Set Arg count to -1, representing a variable number of arguments Click Create to add the award_date function Click Apply, then close the Preferences dialog Step 3: Add the template to the custom column Double-click any book's cell in the Awards column to open the Edit template dialog window. Paste the following code into the edit box: Code: program:award_date( 'NYT 10 Best of the Year:NYT 10 Best of', 'NYT #1 Fiction:NYT #1', 'NYT #1 Nonfiction:NYT #1') Click OK to close the Edit template window Step 4: Create the catalog, adding Awards to the Description pages Click Convert Books↓Create catalog of books in your calibre library Switch to the E-book options tab In Other options\|Description note, select Awards Catalogs created with these settings will show matching books awards in the Description notes section, opposite the cover An even more advanced approach: per-tag processing Spoiler: It is also possible to process matching tags individually. In this approach, the mappings are assigned in the function. Step 1: Create a custom column 'Awards' Open Preferences\|Interface\|Add your own columns Click Add custom column Lookup name: awards Column heading: Awards Column type: Column built from other columns Template: program:'' (we will add the template in Step 3) Click OK to close the dialog Click Apply to close the Add your own columns dialog Click Close to close the Preferences dialog. Step 2: Create the template function Open Preferences\|Advanced\|Template Functions Copy the code below, paste it into the Program Code window. Code: def evaluate(self, formatter, kwargs, mi, locals, val): from calibre.utils.date import format_date SEPARATOR = u' \u00B7 ' # Unicode representation of · list_1 = {'NYT 10 Best of the Year':'NYT Top 10 Best of'} list_2 = {'NYT #1 Fiction':'NYT Best Seller', 'NYT #1 Nonfiction':'NYT Best Seller'} ans = [] for tag in kwargs.get('tags'): if tag in list_1: ans.append(list_1[tag] + format_date(kwargs.get('pubdate'),u' yyyy')) elif tag in list_2: ans.append(list_2[tag]) ans = sorted(ans) return SEPARATOR.join(ans) In the Function field, type award_date Set Arg count to 1, representing a single argument Click Create to add the award_date function Click Apply, then close the Preferences dialog Step 3: Add the template to the custom column Double-click any book's cell in the Awards column to open the Edit template dialog window. Paste the following code into the edit box: Code: program: award_date(None) Click OK to close the Edit template window Step 4: Create the catalog, adding Awards to the Description pages Click Convert Books↓Create catalog of books in your calibre library Switch to the E-book options tab In Other options\|Description note, select Awards Catalogs created with these settings will show matching books awards in the Description notes section, opposite the cover Thanks to @chaley for adding python functionality to the template feature making this possible, and his assistance in developing the sample code. Learn more about calibre's template language feature Discuss other uses of calibre's template language feature G N.B. The year added to the award name is actually the year of the book's publication, which is almost always the same year the book won the award. If you're more OCD than I am, and this bothers you, create a custom column dedicated to awards. Last edited by GRiker; 01-18-2011 at 08:50 AM.

01-14-2011, 03:58 PM	#396
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	Catalog Users: Input needed I've had a number of requests to disable the year of publication that's appended to the title in various section lists. Currently, the implementation is inconsistent: Books by Author: Yes Books by Title: No Books by Series: Yes Books by Genre: No Recently Added: No Descriptions: Yes (template can be edited to remove/modify) I'm seeking input on how this might be presented in the Generate catalog\|E-book options dialog. I envision three possible scenarios: Offer a global 'Append date of publication' checkbox. This would append the date after the title in all enabled sections, or none at all. Offer per-section checkboxes. Don't change anything, it's perfect the way it is. Please reply in this thread if you have an opinion about this possible enhancement. G Last edited by GRiker; 01-14-2011 at 07:17 PM.

01-16-2011, 05:20 PM	#402
Fashtas Junior Member Posts: 2 Karma: 10 Join Date: Jan 2011 Device: Kindle 2	I'm having some issues generating a catalog. I've generated one before with (almost) the same set of books, but added some more and changed a few tags and now I am getting the following error: ValueError: Cannot open image from empty data string Unlike the issues with Inconsistent Author Sort values (which I had before) there is no sign that I can see of what book is causing the issue The only thing I haven't tried was the recommended check for Tag problems (The quick way to confirm this is to change the regex expression in the EPUB/MOBI options tab) because I can't find any setting tab or screen called "EPUB/MOBI Options" This is using 0.7.38. I had tried 39, with the obvious problems, then went to 40, which had errors uploading to the Kindle, so rolled back to 38 because I needed to fill a Kindle before an overseas trip Spoiler: ERROR: Conversion Error: <b>Failed</b>: Generate catalog Generate catalog Catalog_EPUB_MOBI(): Generating mobi for kindle in GUI environment connected_device: 'Amazon Kindle' mount point: G:\documents book count: 732 Sections: Authors, Titles, Recently Added, Genres, Descriptions opts: authorClip: 60 catalog_title: u"Walter's Books" connected_kindle: True descriptionClip: 100 exclude_book_marker: u':' exclude_genre: u'\\[.+\\]' exclude_tags: u'~,Catalog' header_note_source_field: '' merge_comments: '::True' output_profile: 'kindle' read_book_marker: u'tag:+' search_text: None sort_by: None sort_descriptions_by_author: True sync: True thumb_width: '1.00' wishlist_tag: u'Wishlist' existing thumb cache at C:\Users\Chris\AppData\Roaming\calibre\caches\cata log\thumbs.zip, cached_thumb_width: 1.00" Begin catalog source generation DPI = 168; thumbnail dimensions: 84 x 112 Collecting Kindle bookmarks matching catalog entries 24 genre tags in database (excluding genres matching \[.+\]): Adventure, Catalog, Childrens, Classic, Diskworld, Fiction, Hard Science, Historical, Law and Order, Literature, Magic, Nonfiction, Sci-Fi Short, Spy, Strange, Thriller, detective, fantasy, humor, mystery, science fiction, vampire, war, zombie Genre summary: 16 active genre tags used in generating catalog with 732 titles Childrens: 13 titles Classic: 2 titles Diskworld: 40 titles Fiction: 711 titles Hard Science: 3 titles Historical: 11 titles Law and Order: 11 titles Magic: 19 titles Nonfiction: 12 titles Spy: 64 titles Strange: 10 titles Thriller: 12 titles fantasy: 134 titles humor: 41 titles mystery: 48 titles science fiction: 284 titles Python function terminated unexpectedly Cannot open image from empty data string (Error Code: 1) Traceback (most recent call last): File "site.py", line 103, in main File "site.py", line 85, in run_entry_point File "site-packages\calibre\utils\ipc\worker.py", line 107, in main File "site-packages\calibre\gui2\convert\gui_conversion.py", line 61, in gui_catalog File "site-packages\calibre\library\catalog.py", line 5020, in run File "site-packages\calibre\library\catalog.py", line 1357, in buildSources File "site-packages\calibre\library\catalog.py", line 2881, in generateThumbnails File "site-packages\calibre\library\catalog.py", line 4672, in generateThumbnail File "site-packages\calibre\utils\magick\draw.py", line 77, in thumbnail File "site-packages\calibre\utils\magick\__init__.py", line 111, in load ValueError: Cannot open image from empty data string

01-16-2011, 05:59 PM	#403
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	That's a bug that was fixed in 0.7.40, so you'll have to wait until your kindle issue is resolved, or identify the book with the bad cover. You have a cover that's just an empty string. You could try typing 'cover:true' in the search box, then paging through the titles until you find the title(s) that don't actually have a cover. G Last edited by GRiker; 01-16-2011 at 06:04 PM.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search

01-14-2011, 02:20 AM	#391
alexxx Connoisseur Posts: 68 Karma: 479602 Join Date: Aug 2006 Device: Kindle DX	Hi people, I am trying to create a catalog just for the books that on my Kindle are in a given collection ("TO READ"). The problem is that I can filter them in the Device panel by this search: Collections:"TO READ", but in the Device panel I cannot create catalogs (of course, "Convert Books" is grayed out). While instead in the Library panel I cannot filter for those books, since the Collections column doesn exist. There is some way to filter - in the Library panel - for books that on the device are on a given collection? ... I hope I've been clear ... thanks! alessandro

01-15-2011, 07:48 PM	#397
flo_tm Member Posts: 12 Karma: 10 Join Date: Oct 2010 Device: Kindle 3 WiFi, Kindle Touch WiFi	Regarding the three scenarios, I would prefer #2 over #1 over #3. But a hard coded "ignore" for "undefined pub_date" would also be a good first step. Flo

01-15-2011, 08:05 PM	#399
flo_tm Member Posts: 12 Karma: 10 Join Date: Oct 2010 Device: Kindle 3 WiFi, Kindle Touch WiFi	And since I just read a rant by the developer of Sigil regarding the bloating of opensource software because every user wants his little speciality .. (can we have the button to turn off the pub_date in pink, please ? =)) Instead of creating separate checkboxes only for the publication date, would it be possible to create templates for the "assembly" of the section lines ? Please ignore me if I'm naive here, I have no idea how complex the software is and how much work would be involved. Example: Section "By Title" Default: (title) (series) (series_index) - (author) Other Possibilities: (title) - (author) (pub_year) (title) (series) (series_index) - (author) (rating) (title) - (author) (user defined tag) Of course this would also mean that a lot of bimbos (like me) start asking stupid questions why their templates don't work ...

01-17-2011, 06:06 PM	#404
GRiker Comparer of the Ephemeris Posts: 1,496 Karma: 424697 Join Date: Mar 2009 Device: iPad	@Fashtas, 0.7.41 will display a dialog when it encounters a bad cover, telling you which book has the problem. G

01-17-2011, 07:07 PM	#405
Fashtas Junior Member Posts: 2 Karma: 10 Join Date: Jan 2011 Device: Kindle 2	Thanks for the response! I can confirm that 0.7.40 worked in creating the catalog. The error I saw uploading to the kindle previously also didn't reappear, I may have been confused with .39 given I was installing/uninstalling several times The catalogs created weren't quite what I was expecting, not tying into the Kindle collection system, but rather being a book in it's own right with no way (it seems) to "click" one and go to the book itself. After some research I've gone with using an app called the Kindle Collection Manager to edit collections on the device, and making one Catalog with Calibre per collection (or for the larger collections anyway) which I can browse in each collection folder for more information on the books in it Works nicely so far!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Content Newspapers piling up in my Archived Items	Millbrook	Amazon Kindle	9	07-05-2012 03:09 PM
Creating Catalogs: Start here	GRiker	Library Management	0	01-26-2011 03:58 PM
Removing 'Archived Items'	seagull	Kindle Developer's Corner	7	09-29-2010 01:38 PM
Archived items question	Corran006	Amazon Kindle	1	08-28-2010 02:52 AM
T?ol to create Structured catalogs?	itimpi	Calibre	5	11-19-2008 11:30 AM