06-30-2015, 06:51 AM | #16 |
Grand Sorcerer
Posts: 24,906
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
The code in library/server is the current content server code. The code in srv is the rewrite that Kovid is doing. Unfortunately, he seems to have dropped the changes I made for kepubs. I'll have a look at putting it back in and see if he is OK with it.
The list of formats to update the metadata is a bit harder. The problem is these need the metadata writer plugin for the format. But, as this is not included in calibre for kepubs, there would need to be extra checking to make an update wasn't attempted when the writer wasn't there. From a Kobo users point of view, adding the metadata writer and reader plugins to calibre would be best. But, I understand Kovid's desire not to include single site formats in the base calibre. |
06-30-2015, 07:51 AM | #17 |
Generally Awesome Person
Posts: 1,061
Karma: 2178845
Join Date: Jan 2013
Location: /dev/kmem
Device: Kobo Clara HD, Kindle Oasis
|
If there were some way for plugins to hook into the content server and say "support my format too" it would be enough. I think right now I just need to add 'kepub' to a couple lists at initialization time?
|
06-30-2015, 08:35 AM | #18 |
Grand Sorcerer
Posts: 24,906
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
I wouldn't have thought that was needed. Calibre knows what metadata writer plugins it has and what formats they support. If that is built for the server at startup, it should be able to handle formats correctly. But, maybe the server is not loading all the plugins.
In any case, I'm not sure what the state of the new server is. I can start it and reach it from a browser, but I get an empty file as a response. It's probably a configuration problem on my part, but I can't see it right now. |
06-30-2015, 08:55 AM | #19 |
Wizard
Posts: 3,108
Karma: 60231510
Join Date: Nov 2011
Location: Australia
Device: Kobo Aura H2O, Kindle Oasis, Huwei Ascend Mate 7
|
Here are the very simple changes I made. I do not plan to submit them myself. Obviously the changes to the file in srv are not necessary given this is Kovid's re-write. With the changes to the file in library/server there is no problem now with creating a content server metadata plugboard for the kepub format. When downloading kepub books from the content server they do download as kepub epub and the metadata plugboard is applied, at least so far as my setup to modify the title is concerned. It does not, however, actually create a collection or add the book to the collection. Am I correct in assuming that this is a function purely of the driver? It does work over USB using the Kobo Touch driver, but not with the content server.
Given that adding support for new formats seems to be relatively trivial, is it actually worthwhile to support all formats with metadata writers? Support can easily be added on request of users or developers who use the particular format. Support for epub, mobi, azw3 and now kepub seems to cover all of the most common formats, with the exception, I suppose, of pdf, which is not really a great format for ereaders anyway. Spoiler:
|
06-30-2015, 09:08 AM | #20 | |||
Grand Sorcerer
Posts: 24,906
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
Quote:
Quote:
Quote:
|
|||
06-30-2015, 09:29 AM | #21 |
Wizard
Posts: 3,108
Karma: 60231510
Join Date: Nov 2011
Location: Australia
Device: Kobo Aura H2O, Kindle Oasis, Huwei Ascend Mate 7
|
Thanks for your comments @davidfor. I was fairly sure I was correct about the driver handling the collections, for pretty much the reasons you confirmed. However, I am neither competent or confident in python. I can usually roughly follow the logic of the program but that is about it. After your post mentioning Kovid's re-write I did suspect that it may be in the code under srv, but without a better understanding than I have could not be sure.
And the changes to the file under srv? It will need to be changed eventually to reflect not only the changes I suggested but to add back your own code, at least if kepub is to be supported in the new server. My thinking was that it does not need to be changed now given that the new server still seems to be in the early stages of a rewrite. Given this, I was wondering whether Kovid would welcome changes to it at this stage. Also, it does not seem to incorporate your kepub.epub code at the moment so adding kepub to the formats in one line would probably not be all that useful. In principle, it would be nice to support all formats with a metadata writer. |
06-30-2015, 09:39 AM | #22 |
Generally Awesome Person
Posts: 1,061
Karma: 2178845
Join Date: Jan 2013
Location: /dev/kmem
Device: Kobo Clara HD, Kindle Oasis
|
I would prefer to keep KePub out of the main calibre code, only because it's a single-site format that not even all users of the single site even like. It's also subject to change at Kobo's whims. Rather, I would prefer to ask Kovid to update (or accept a patch for) https://github.com/kovidgoyal/calibr...ontent.py#L219 to allow a plugin to register itself as providing the necessary format support.
|
06-30-2015, 11:20 AM | #23 |
creator of calibre
Posts: 43,926
Karma: 22669820
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
|
The new server is nowhere near ready just now. As for allowing plugins to register themselves, I dont want to do that, because not all plugins are suitable for use in a server context. Some of them can be quite slow/leak memory/crash(pdf) which is why the list of formats is artificially restricted, as opposed to being the list of formats supported by all metadata writer plugins.
There is also the question of plugboards, IIRC the list of formats that can be used with plugboards in the server is also hardcoded. I'm not sure what the best way to proceed here is. I suppose I could add kepub to the hardcoded list and have the code disard it if not kepub writer plugin is available. |
06-30-2015, 05:58 PM | #24 |
Grand Sorcerer
Posts: 6,212
Karma: 16534894
Join Date: Sep 2009
Location: UK
Device: Kobo: KA1, ClaraHD, Forma, Libra2, Clara2E. PocketBook: TouchHD3
|
Just for the sake of philosophical discussion that description could equally well describe Amazon's azw3/kf8, couldn't it?
|
06-30-2015, 07:07 PM | #25 |
Generally Awesome Person
Posts: 1,061
Karma: 2178845
Join Date: Jan 2013
Location: /dev/kmem
Device: Kobo Clara HD, Kindle Oasis
|
Well, yes Amazon has changed the extension when they've changed the format (.azw, .awz3). Kobo hasn't changed their format yet, so we don't know what they would do. But it would be trivial to say "open the ZIP file, look for this file with these specific contents; if present use the new format parser" and keep the same .kepub.epub double extension. Either way, we could play that cat-and-mouse game. Now that you make me think about it a bit, it feels like a code maturity thing. Kovid made the decision to incorporate AZW and AZW3, but it's my impression that the code around that is much more mature and stable than the KePub code is.
Who knows, perhaps someday if I can produce KePub files nearly as well as Kobo can perhaps Kovid will want to incorporate it all into calibre core despite it being site-specific and not near-universally preferred. If he does I'll happily make the patches to bring everything in, but I'd think that would also need a bunch of work with davidfor to merge the code in cleanly; if KePub is coming into calibre core it shouldn't be as yet another driver. A huge chunk of my problem is that I can't justify purchasing a book from Kobo (they're so expensive, and nothing I actually want is eligible for coupons) but I need a recently released book from them to make sure I'm generating KePub files that work like their current releases. How about this: if someone has a recently released (say, released during the month of June) well-formatted English-language KePub file they've stripped the DRM from and they're willing to email me privately, I'll put in another effort to get the plugin code generating better KePub files. A lot of the recent updates have been made based on books that are a few years old by now, and I think Kobo has made some internal improvements to how they generate KePub files since then. If Kovid wants to bring KePub into core calibre and David doesn't object to me merging the drivers, I'll also dedicate a good chunk of my copious spare time to the patch to integrate it all to calibre. |
06-30-2015, 07:49 PM | #26 |
Grand Sorcerer
Posts: 12,212
Karma: 73758904
Join Date: Nov 2007
Location: Toronto
Device: Nexus 7, Clara, Touch, Tolino EPOS
|
I'll examine my wife's collection of romance books and see if any of them are recent publications...
|
06-30-2015, 08:05 PM | #27 |
Generally Awesome Person
Posts: 1,061
Karma: 2178845
Join Date: Jan 2013
Location: /dev/kmem
Device: Kobo Clara HD, Kindle Oasis
|
Trying to motivate me to finish quickly I see
|
06-30-2015, 10:23 PM | #28 |
Grand Sorcerer
Posts: 12,212
Karma: 73758904
Join Date: Nov 2007
Location: Toronto
Device: Nexus 7, Clara, Touch, Tolino EPOS
|
Not really; just I know she buys a LOT from Kobo, and she is away right now, so access to her PC is very easy! I'll go spelunking tomorrow when I'm more with it.
|
07-01-2015, 12:39 AM | #29 |
Grand Sorcerer
Posts: 24,906
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
There should be some free books that fit the bill. Finding them might be the problem. Looking through my books, I can find a few that might have been added to Kobo this year. The most recent I can find quickly is https://store.kobobooks.com/en-US/ebook/backtrek. The files in the kepub are dated 30/3/2015. I will have another look when I have more time.
I did notice one little problem with the conversion recently. The added spans surround the sentence, but the space between the sentences is between the spans. It looks like: Code:
end of sentence.</span> <span ..>Start of next sentence. |
07-01-2015, 01:37 AM | #30 | |
Grand Sorcerer
Posts: 24,906
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
|
OK, I've been wondering what the status was. But, unless it is going to be a few months, I don't want to do anything in the current server.
Quote:
Hang on, I hadn't looked at the code in the version in this thread. It doesn't do the cover settings, so it is just adding the extension. jgoguen, is there are reason you dropped setting the cover property? |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[Metadata Reader Plugin] KePub Metadata Reader | jgoguen | Plugins | 40 | 05-17-2020 10:24 AM |
Regarding using metadata objects in identify method of metadata download plugin api | aprekates | Development | 1 | 07-06-2014 03:35 AM |
Correcting metadata for kepub books | GeoffR | Kobo Reader | 9 | 03-25-2013 06:06 AM |
Bug in epub metadata writer? Invalid attribute role="aut" | Man Eating Duck | Calibre | 3 | 09-26-2012 09:55 AM |
[Metadata Download Plugin] Goodreads Metadata **Deprecated** | kiwidude | Plugins | 30 | 04-23-2011 02:10 PM |