03-11-2011, 02:31 PM | #1 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
New Metadata class (empty books) and custom columns
In my plugin I have a class called "SeriesBook", which wraps access to an instance of a Metadata object, like below. The SeriesBook class has various functions I require which include getting and setting values in particular custom columns that Metadata object has.
Code:
class SeriesBook(object): def __init__(self, mi): self.mi = mi Code:
mi = Metadata(title, authors) book = SeriesBook(mi) Any suggestions on how to solve this? I don't want to persist the mi to the database at this point and re-read it. What I want to do is somehow create the custom column infrastructure on the mi instance when I create the empty book, so my SeriesBook class will not care. This has to be done in such a way that when I call db.set_metadata() way later in the code (the user clicking ok on the dialog), it will be quite happy with the way that custom column data has been set on the mi instance. I hope that makes sense. |
03-11-2011, 02:44 PM | #2 |
Sigil & calibre developer
Posts: 2,488
Karma: 1063785
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
|
This has nothing to do with your question, but why are you tracing a MetaInformation inside of your SeriesBook class instead of making it a subclass? This would allow you to pass the SeriesBook in places a MetaInformation can be used. This construct should provide you with cleaner code that is easier to work with.
|
03-11-2011, 02:50 PM | #3 | |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Quote:
Besides which, the call to db.get_metadata is going to give me mi instances which is my normal starting point for the data. So I would have the problem of coercing one of those into a SeriesBook class? |
|
03-11-2011, 03:51 PM | #4 |
Grand Sorcerer
Posts: 11,742
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
I agree with user_none. You should be using a subclass. You wouldn't be the first. There are subclasses of Metadata in the device drivers.
You can avoid attribute/method collision by prefixing your methods and attributes with something, such as sb_. Regarding getting instances from get_metadata, do something like: Code:
class SeriesBook(Metadata): def __init__(self, title, author): Metadata.__init__(self, title, author) .... |
03-11-2011, 04:13 PM | #5 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Ok, we are getting OT with my original problem. I still don't get how you can downcast a class though? If I was doing nothing but creating books, I would have no problem with what you suggest, but I just don't get how when my starting point is a list of Metadata objects retrieved like this:
mi = db.get_metadata(row.row()) that I can somehow downcast that Metadata object returned from that function into a SeriesBook class? I apologise if this is Python ignorance here, its just something you can't do in C# when an object has been created as the base class? |
03-11-2011, 04:35 PM | #6 |
Grand Sorcerer
Posts: 11,742
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Try something like this test program (which works):
Code:
from calibre.ebooks.metadata.book.base import Metadata from calibre.library.database2 import LibraryDatabase2 from calibre.utils.config import prefs class MyMetadata(Metadata): def __init__(self, mi): Metadata.__init__(self, None) self.smart_update(mi, replace_metadata=True) def mm_print(self, attr): print attr, '=', getattr(self, attr, 'not found') src = prefs['library_path'] db = LibraryDatabase2(src) my_mi = MyMetadata(db.get_metadata(1283, index_is_id=True)) print my_mi my_mi.mm_print('application_id') |
03-11-2011, 04:50 PM | #7 |
Grand Sorcerer
Posts: 11,742
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
I wonder if it makes sense to give get_metadata a new parameter which is the class to instantiate? That would eliminate the smart_update in cases where a Metadata subclass is being used.
It would look something like: Code:
def get_metadata(self, idx, index_is_id=False, get_cover=False, metadata_class=Metadata): ''' Convenience method to return metadata as a :class:`Metadata` object. Note that the list of formats is not verified. ''' row = self.data._data[idx] if index_is_id else self.data[idx] fm = self.FIELD_MAP mi = metadata_class(None) |
03-11-2011, 04:59 PM | #8 |
Grand Sorcerer
Posts: 11,742
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
BTW: the only ways to get custom metadata into an instance of Metadata are to put it there yourself or to call get_metadata. It will never be there if you create new instances of Metadata(), with or without a subclass.
The following should work to add custom metadata to a Metadata instance: Code:
mi.set_all_user_metadata(db.custom_field_metadata()) Last edited by chaley; 03-11-2011 at 05:07 PM. |
03-11-2011, 05:24 PM | #9 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Thanks for the replies Charles.
I see that smart_update() was the secret ingredient to make the subclassing work, thanks for all the effort of the example etc. I've changed my code, but I wouldn't say I was overly delighted with putting prefixes in front of all the functions and attributes as future protection. As I only wanted a handful of attributes off the mi for use in my plugin I'm not sure that any perceived gain is worth the negatives of readability, but at least I now know "how" to do it, thanks. That set_all_custom_metadata call looks like the special call I was after, thanks. That and confirming I wasn't doing something stupid with the behaviour I was seeing. |
03-11-2011, 06:24 PM | #10 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Charles, I am getting a KeyError: '#extra#' error when I try to call set_metadata when I use that set_all_user_metadata call above. I take it there is another special trick required to set that 'extra' data?
Code:
from calibre.ebooks.metadata.book.base import Metadata from calibre.library.database2 import LibraryDatabase2 from calibre.utils.config import prefs src = prefs['library_path'] db = LibraryDatabase2(src) mi = Metadata('Test Title', ['Test Author']) mi.set_all_user_metadata(db.custom_field_metadata()) db.import_book(mi, []) print 'imported book:', mi.id |
03-12-2011, 03:32 AM | #11 | ||
Grand Sorcerer
Posts: 11,742
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Quote:
The only reason to put user (custom) metadata into a Metadata instance is if you are going to put a value in there. If you have a value for a particular column, then set the metadata for that column, then set the value for that column. Something like Code:
meta = db.metadata_for_field(key) mi.set_user_metadata(key, meta) mi.set(key, val=whatever_it_is, extra=whatever_it_is) The underlying issue here is that Metadata instances can come from a variety of places, most of which are not the db. OPF files are a common example, It is normal that there isn't any strict correspondence between the columns in the library and the information in the Metadata instance. Get_metadata gives you what is in the current library, which may have nothing to do with what will be in some other library -- think copying between libraries. The consequence is that the code must ensure that the metadata is appropriate for its purpose. If you need to set a particular custom column, then you must add it to the record and set its value. Unless you are sure of the source you shouldn't trust the metadata for a given column, because #read might be bool in library 1 but enumeration in library 2. This is why one sees column and type guards in set_metadata: Code:
for key in user_mi.iterkeys(): if key in self.field_metadata and \ user_mi[key]['datatype'] == self.field_metadata[key]['datatype']: doit(self.set_custom, id, val=mi.get(key), extra=mi.get_extra(key), label=user_mi[key]['label'], commit=False) |
||
03-12-2011, 05:16 AM | #12 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Hi Charles,
That was a simplified example to replicate the error. Perhaps I should explain exactly what I am trying to do. This is for my manage series plugin, and a tester suggested they wanted the ability to manage custom series columns as well as the built in series one. So in my dialog I have a dropdown allowing the user to select which column to manipulate which changes the set of values displayed and updated. In this dialog the user can also add empty books to fill in gaps in a series. That means they may or may not decide to populate the custom series column, depending on which column they had chosen to manipulate in my dialog. After all, not all series columns will have a value. If I was to use this approach instead of set_all_metadata: Code:
meta = db.metadata_for_field(key) mi.set_user_metadata(key, meta) mi.set(key, val=whatever_it_is, extra=whatever_it_is) Your point about not trusting the source of the custom column is a very interesting one, I appreciate the warning. However what is the appropriate approach to overwrite the column in that instance, because the code you showed me just ignores it. Certainly in my case now that would be undesirable - the user would be left with a book that they could not set a value for in that column because in it's original source in some other database the column had a different datatype? |
03-12-2011, 05:37 AM | #13 | ||
Grand Sorcerer
Posts: 11,742
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Quote:
Final case: you construct a Metadata instance for a book that is already in the DB without using get_metadata. In this case, standard metadata will be replaced with 'non-False' data from the mi object. Custom columns will be replaced with values from the mi object if a) they exist in the mi object, and b) they have the same type. FYI: I intend to add a parameter to set_metadata to override the 'non-False' check, because as is, set_metadata cannot set a standard metadata column to nothing. For example, you cannot set the tags to empty or empty the ISBN. |
||
03-12-2011, 06:22 AM | #14 |
Calibre Plugins Developer
Posts: 4,637
Karma: 2162064
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
|
Thanks Charles.
Just one last point on that invalid column thing as you kind of freaked me a bit with it. Say you have two Calibre databases. In database A you defined a #myseries column of type Series. And in Database B you defined a #myseries column of type Yes/No. Then I copy a book out of Database B into Database A. Funky I know, but if people download/import books off the internet which have opf files it could happen. So in Database A, for that particular book that has just been imported, when I read that book using get_metadata and look at the value in the #myseries column, what am I likely to see? If I try to set a value into that #myseries column for that book, will it be using my database A definition of the column being a series field, or a book source based definition from Database B of it being a Yes/No field? |
03-12-2011, 06:47 AM | #15 | ||
Grand Sorcerer
Posts: 11,742
Karma: 6997045
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
|
Quote:
Quote:
The definition of a column is always dependent on the database the column lives in. There is no circumstance where the definition will change because of an import. If there is a key/type match, set_metadata will change the value and extra in db A to what came from db B, otherwise it will set nothing. It will never change anything else, including the optional description information in the 'display' field. Because of this rule, there are circumstances where you might be holding an mi structure that does not correspond to what is in the DB. Your example is one; the mi structure from db B (presumably made from an OPF) will have different information in it from what would be returned by get_metadata in db A after an import. It is therefore not safe to assume that after a dbA.set_metadata(dbB.some_mi), dbA contains a copy of the information in dbB.some_mi. The information can vary because of custom column type/key mismatches, 'false' values in some_mi (the tags issue), or even subtle things like tag id order. If if is important that you are using correct information after a call to set_metadata, you should do a get_metadata immediately after. |
||
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Curious - what custom columns are you creating/using? | texasnightowl | Calibre | 16 | 05-28-2015 10:26 AM |
0.7.46 and custom columns | meme | Library Management | 4 | 02-21-2011 04:21 AM |
Managing Custom Columns | ddjohn | Library Management | 3 | 02-19-2011 10:42 AM |
Custom Columns - the Future? | Starson17 | Calibre | 2 | 07-13-2010 09:56 AM |
Creating custom columns is broken in 0.7.3 | chaley | Calibre | 6 | 06-20-2010 03:40 AM |