Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 02-27-2011, 06:50 AM   #1
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,601
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
Can custom book data be displayed in a custom column?

I store a key called 'goodreads_id' in the plugin data store for books using db.add_custom_book_data() as recommended here and it is working great.

I am wondering if it is possible (for if a user wanted it) to display that value in a custom column. Alternatively for example have a column that indicates whether the value is populated. So users could see which books they have linked to Goodreads (and search based on this).

I had a brief look at program template functions, but I couldn't obviously see how you would have access to the custom book data.

Is it possible? I know chaley loves a challenge...
kiwidude is offline   Reply With Quote
Old 02-27-2011, 07:02 AM   #2
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
No. The template processor can see information only if it is in a Metadata instance. Every attribute in a Metadata instance must be serializable (able to be persistent), so it cannot contain a db handle.

If you want to provide this feature, then you will need to ask the user for a CC to use and populate it yourself.
chaley is offline   Reply With Quote
Advert
Old 02-27-2011, 07:10 AM   #3
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,601
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
That's a shame, thanks.
kiwidude is offline   Reply With Quote
Old 02-27-2011, 08:11 AM   #4
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Thinking further about this, a few changes would make it possible. I am willing to make them if Kovid says 'go ahead' (something I am not convinced he will do for various reasons).

The idea is to include all the custom book data in the meta2 view. It would be done similarly to how the author sort map is done today, but returning a JSON-encoded string such as
{ "custom_data_name" : JSON_data_from_DB , custom_name_2, JSON_data_2, ...}. This string would be added to the values returned by get_metadata (mi.custom_data = row[nnn]). A template function would be provided that decodes that data, selects out a custom_data_name, then selects an attribute from that name.

An alternative to adding the information to the view is doing the necessary select in get_metadata. My problem with this approach is that we might end up doing the same select multiple times, because there is no guarantee that get_metadata is not called more than once for the same book.

This scheme has the advantage that almost no processing is required unless a) custom data exists, and b) someone uses the information.

The disadvantages that I can see are:
1) Even if there is no custom data, there would be a small performance penalty to put the None into the view. I think this penalty is very close to zero.
2) If there is custom data, then the JSON strings must be fetched and concatenated. This operation might add a few percent to the time to build the view, depending on the amount of data.
3) The template function would be rather slow, because it would decode the JSON data. It would also require that the custom data be a dict of (dicts/lists). The function would look something like "custom_data(data_name,item_name)". If the data under data_name is a list, then item_name must be an integer. If the data is a dict, then item_name must be a string.

With the above, one can create a composite custom column to show the custom data. For example, and making a large number of assumptions, the goodreads_id could be displayed using the template
Code:
program: custom_data('goodreads_plugin', 'goodreads_id')
It could become Yes/No using
Code:
program: test(custom_data('goodreads_plugin', 'goodreads_id'), 'Yes', 'No')
chaley is offline   Reply With Quote
Old 02-27-2011, 09:33 AM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,773
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
There's going to be an identifiers table added to the database to support non ISBN identifiers, lije DOI, lccn, google, adobe, asin, etc. so that will be the correct solution for this.
kovidgoyal is online now   Reply With Quote
Advert
Old 03-02-2011, 05:00 AM   #6
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,601
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
So now you have committed the new identifier stuff, would I be correct in saying that the correct way to display one of the new identifiers in a custom column is to use a program template and do something like:
Code:
name:GoodreadsId
def evaluate(self, formatter, kwargs, mi, locals, val):
    return mi.get_identifiers().get('goodreads_id','')
and then in my custom column I do this:
{'':GoodreadsId()}

Or is there a better way?

EDIT: I suppose this would improve things:
Code:
name:Identifier
def evaluate(self, formatter, kwargs, mi, locals, val):
    return mi.get_identifiers().get(val,'')
and then in my custom column I do this:
program: Identifier('isbn')

Last edited by kiwidude; 03-02-2011 at 05:08 AM.
kiwidude is offline   Reply With Quote
Old 03-02-2011, 05:05 AM   #7
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by kiwidude View Post
So now you have committed the new identifier stuff, would I be correct in saying that the correct way to display one of the new identifiers in a custom column is to use a program template and do something like:
Code:
name:GoodreadsId
def evaluate(self, formatter, kwargs, mi, locals, val):
    return mi.get_identifiers().get('goodreads_id','')
and then in my custom column I do this:
{'':GoodreadsId()}

Or is there a better way?
The first thing you must do is add the identifier to the book using db2.set_identifier. Once that is done, then to see the identifier in a custom column, use the new 'select' function, which is a generalized version of what you proposed. Something like {identifiers:select(goodreads_id)}.

To search, use 'identifiers:goodreads_id:what-have-you.' This is a keypair search (something new), searching for key:value pairs. Matching the key and the value are independent, and can use contains, exact, and regexp matching. For example, identifiers:good:true will find all books with a key containing 'good'.
chaley is offline   Reply With Quote
Old 03-02-2011, 05:22 AM   #8
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,601
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
Quote:
Originally Posted by chaley View Post
Once that is done, then to see the identifier in a custom column, use the new 'select' function, which is a generalized version of what you proposed. Something like {identifiers:select(goodreads_id)}.
Ahhh a new function, no wonder I couldn't spot that on the doc page. I figured you would put something more convenient than a write your own one in...

I have been trying to test this just with the isbn value column since I haven't set my goodreads id yet. I can do queries in the search bar like this:
identifiers:isbn:9781842320136

But when I put this as a custom column it displays as blank:
{identifiers:select(isbn)}
kiwidude is offline   Reply With Quote
Old 03-02-2011, 05:26 AM   #9
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by kiwidude View Post
But when I put this as a custom column it displays as blank: {identifiers:select(isbn)}


I should have mentioned -- this morning during testing I found that Metadata.format_field did the wrong thing with the identifiers dict. I fixed it, but it isn't in trunk yet.

If you want to test, then add
Code:
=== modified file src/calibre/ebooks/metadata/book/base.py
--- src/calibre/ebooks/metadata/book/base.py	2011-03-01 15:05:18 +0000
+++ src/calibre/ebooks/metadata/book/base.py	2011-03-02 08:50:11 +0000
@@ -595,6 +595,8 @@
             elif key == 'series_index':
                 res = self.format_series_index(res)
             elif datatype == 'text' and fmeta['is_multiple']:
+                if isinstance(res, dict):
+                    res = [k + ':' + v for k,v in res.items()]
                 res = u', '.join(sorted(res, key=sort_key))
             elif datatype == 'series' and series_with_index:
                 res = res + ' [%s]'%self.format_series_index()
The change should appear in trunk tonight.
chaley is offline   Reply With Quote
Old 03-02-2011, 05:35 AM   #10
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,601
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
Cool. I was trying all sorts of variants wondering if I had screwed up, knowing chaley never steers me wrong, heh. I've got a workaround of my own custom function as per my edited post above I did while you were posting a response in the meantime anyways.

One thing I noticed when I was trying the identifiers:isbn:xxx search. It seems to be doing a starts with match (if I drop a few digits off the ISBN I get multiple results). Is that intentional, I wouldn't have thought that would be particular useful for an "identifier" field? If it is intentional, how do I force an exact match?

EDIT: Never mind, realised it was just a normal search and just use :=

Last edited by kiwidude; 03-02-2011 at 05:39 AM.
kiwidude is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Need help with this custom Column. Rie142 Library Management 2 02-20-2011 09:51 AM
Custom Column - Book Format (ie mobi, epub etc) jphphotography Calibre 3 01-17-2011 12:34 AM
custom column i need a little help shinken Calibre 3 09-15-2010 03:41 AM
custom column not searchable SkyDream Calibre 5 07-26-2010 05:38 AM
HTML column type in custom meta-data. mukoan Calibre 1 07-13-2010 08:59 AM


All times are GMT -4. The time now is 09:55 AM.


MobileRead.com is a privately owned, operated and funded community.