Forma how can I force Calibre to use geek1011's kepubify to convert books to kepubs?

[droopy]

I was checking out geek1011's homepage and came across his kepubify work (https://pgaskin.net/kepubify/.

Is there a way to force Calibre to convert books into kepub using geek1011's program, and not calibre's built-in kpub converter?

[JSWolf]

Originally Posted by droopy

I was checking out geek1011's homepage and came across his kepubify work (https://pgaskin.net/kepubify/.

Is there a way to force Calibre to convert books into kepub using geek1011's program, and not calibre's built-in kpub converter?

You can if you can write a plugin to use kepbify.

[jackie_w]

@droopy,

I thought geek1011 originally created Kepubify by looking at what the KoboTouchExtended calibre driver was already doing to create kepubs. I think he may have added a couple of extras.

If you think there is some desirable feature missing from KoboTouchExtended you should ask about it in its own dedicated thread. Warning, it's huge because it's been around a long time.

[geek1011]

I made kepubify based on the KoboTouchExtended driver, my findings in the firmware, and a few existing kepub books. After I implemented the basics, I added a few unobtrusive features (like CSS addition and text replacement), and made seriesmeta.

About a Calibre plugin, I kind of got started on making one, but I never finished it, as I didn't really see the point (I don't use Calibre, and the Calibre conversion process mostly negates some of the benefits of using kepubify).

I suggest you decide whether or not you want to use Calibre, and go from there. Note that you can import books converted by kepubify into Calibre if you have many books, and you want the speed benefits of kepubify, but still use Calibre for management. If you are just transferring books, you might as well just use kepubify and seriesmeta by itself, as that is how it was meant to be used.

I use the following script (on Linux) to sync my books (I've simplified it a bit): git -C "$HOME/Documents/Books/Books/" pull && kepubify --update --output "$(kobo-find -wf)/kepubify" "$HOME/Documents/Books/Books/" && umount "$(kobo-find -wf)" ; seriesmeta "$(kobo-find -wf)"

P.S. My homepage is quite outdated, and I'm working on a new one when I have time. You'll find my latest projects on GitHub.

[droopy]

Originally Posted by jackie_w

@droopy,

I thought geek1011 originally created Kepubify by looking at what the KoboTouchExtended calibre driver was already doing to create kepubs. I think he may have added a couple of extras.

If you think there is some desirable feature missing from KoboTouchExtended you should ask about it in its own dedicated thread. Warning, it's huge because it's been around a long time.

I have KoboTouchExtended. Does it do everything that is on geek1011's list of kepubify's benefits? For example, conversion speed; removing extraneous data?

In this thread, geek1011 mentioned having added ”a few unobtrusive features (like CSS addition and text replacement), and made seriesmeta.” I don't know what ask of these featuress do, but they surely are good stuff to have, right? If so, does KoboTouchExtended do these things too?

[geek1011]

Originally Posted by droopy

I have KoboTouchExtended. Does it do everything that is on geek1011's list of kepubify's benefits? For example, conversion speed; removing extraneous data?

In this thread, geek1011 mentioned having added ”a few unobtrusive features (like CSS addition and text replacement), and made seriesmeta.” I don't know what ask of these featuress do, but they surely are good stuff to have, right? If so, does KoboTouchExtended do these things too?

KTE doesn't do everything. It is around 12x slower (partly due to Calibre, and partly due to KTE). Inherently, KTE also adds the extra calibre classes and metadata, but that's a matter of personal preference. Calibre can do some of the extra data cleanup with the Modify EPUB plugin. Last time I checked, KTE doesn't work from the command line. Note that I made kepubify mainly to address the inherent limitations of KTE and Calibre.

Seriesmeta adds series info to the Kobo DB. KTE does this automatically for books managed by Calibre, but not ones loaded separately. Both can be used together without issues.

[droopy]

Originally Posted by geek1011

Note that I made kepubify mainly to address the inherent limitations of KTE .

great! This is reason enough to use your kepubify.i hope someone can come in with a way to use Calibre as a book manager while harnessing the power of kepubify as a plug-in.


Maybe the creator of KTE can splice kepubify's code into it

[jackie_w]

Originally Posted by geek1011

Inherently, KTE also adds the extra calibre classes.

What extra calibre classes are these?

@droopy,
If you want to ask questions about KTE you should ask them in the KTE thread I previously linked. Although be aware that a KTE expert is not necessarily a Kepubify expert.

[geek1011]

Originally Posted by jackie_w

What extra calibre classes are these?

I'm talking about the calibre# classes, which is technically Calibre, but KTE only works on Calibre so...

@droopy,
If you want to ask questions about KTE you should ask them in the KTE thread I previously linked. Although be aware that a KTE expert is not necessarily a Kepubify expert.

Yes, and a Kepubify expert isn't necessarily a KTE expert, especially when said expert isn't much of a KTE user.

[jackie_w]

Originally Posted by geek1011

I'm talking about the calibre# classes, which is technically Calibre, but KTE only works on Calibre so...

Are you confusing the KTE driver with a full calibre conversion to KEPUB format using the Kepub Output plugin?

KTE does not rewrite the source epub's CSS - except when requested by the user via its 'Modify CSS' option. This option does not create any calibre# classes either.

If your speed comparisons between calibre and Kepubify are based on a full calibre conversion to KEPUB it's no wonder you're quoting a x12 factor! KTE is much faster than a calibre conversion.

[geek1011]

Originally Posted by jackie_w

Are you confusing the KTE driver with a full calibre conversion to KEPUB format using the Kepub Output plugin?

Probably. I didn't know they were different things until you just told me .

KTE does not rewrite the source epub's CSS - except when requested by the user via its 'Modify CSS' option. This option does not create any calibre# classes either.

I wasn't intending to say that was due to KTE, but a consequence of using Calibre in general.

If your speed comparisons between calibre and Kepubify are based on a full calibre conversion to KEPUB it's no wonder you're quoting a x12 factor! KTE is much faster than a calibre conversion.

I wasn't actually aware there was a difference, as I always pre-converted my books when I was testing Calibre. I'm going to try directly sending a book later this week, but I suspect it will still be around half the speed due to the overhead from Calibre and lack of parallel processing.

[Lucas Malor]

Excuse my ignorance, what's the difference between Kepubify and the Calibre plugin?

[geek1011]

Originally Posted by Lucas Malor

Excuse my ignorance, what's the difference between Kepubify and the Calibre plugin?

Oh, seriously. I've answered this question way too many times. I've answered most of this in this thread, the kepubify thread, and the kepubify webpage. If I was to put it an oversimplified way, I'd say to use kepubify if you aren't already a heavy Calibre user, if Calibre is too slow for your liking, or to use it for a single book if Calibre stalls (or just takes over 10 minutes) to convert one of your books. Also, sometimes, kepubify works better for books with complex formatting or bad structure (it's a bit more minimal and less intrusive, and it is slightly more tolerant of errors). As for seriesmeta, I would say to use it if you don't use Calibre exclusively to manage books with series metadata (note that it won't conflict with KTE).

[davidfor]

Originally Posted by geek1011

Oh, seriously. I've answered this question way too many times. I've answered most of this in this thread, the kepubify thread, and the kepubify webpage.

But, in this very thread, you admitted you didn't know what the KoboTouchExtended driver did. You made a statement about what it did which was very wrong and you were corrected.

And, the fact you don't know what it does is a bit strange. To paraphrase what you state here, I've answered the question of what the KoboTouchExtended driver does way too many times. And in at least one discussion with you (the thread you opened about how to create the kepubify plugin).

If I was to put it an oversimplified way, I'd say to use kepubify if you aren't already a heavy Calibre user, if Calibre is too slow for your liking, or to use it for a single book if Calibre stalls (or just takes over 10 minutes) to convert one of your books.

I can't think of a way for the KoboTouchExtended driver to stall. The conversion could, but you would need a really bad book.

Also, sometimes, kepubify works better for books with complex formatting or bad structure (it's a bit more minimal and less intrusive, and it is slightly more tolerant of errors).

The latter is an explicit decision to support books that are reasonably error free when being send to the device. If there are errors with books that the device can handle fine, if they are reported, they will probably be fixed. And if you have an example of the former, I would be interested in seeing it.

But, I will vehemently disagree with the "less intrusive". We have discussed the fact that kepubify does things that are not documented. And how is "removes leftover Adobe DRM tags, cleans up MS Word tags, and cleans up the html" less intrusive? Especially when there is no option to turn this off. The KoboTouchExtended driver explicitly only does what is needed to have the book as a kepub on the device. Or are you thinking of a conversion with this statement?

As for seriesmeta, I would say to use it if you don't use Calibre exclusively to manage books with series metadata (note that it won't conflict with KTE).

Yes, if you are using calibre, then seriesmeta is unneeded.

But, maybe all my comments are pointless as maybe you are referring to the Kepub Output plugin and hence a conversion in calibre. If you are, then a lot of your points are valid and all I can say is that this is the way it works. But, most people talking about using calibre to get books onto the device a kepubs are talking about the KoboTouchExtended driver.

Lets put this simply, the following are the ways I know of to get a book on a Kobo device as a kepub:

Buy the book from Kobo and sync either of WiFi or via the desktop application. The advantage of this is that Kobo is doing the work to get the book into the correct format. And you can sync the reading status between devices and their apps. The disadvantages are that your book choice is "limit" to what Kobo have in their store and you have to buy the book.

Convert the book in calibre. This uses the Kepub Output plugin and puts the book through the full calibre conversion pipeline. The advantage of this is that it can be used to convert from any format that calibre supports and if you have multiple devices the conversion is only needed once. The disadvantage is that it will add the kepub to the calibre library and you can't do much with it there except send it to the device.

Use the KoboTouchExtended driver to transform the book to a kepub when sending it to the device. The advantages are that you don't need to store the kepub, that calibre will convert to epub if you only have some other format in the library and the driver (via the built-in KoboTouch driver) supports other functions such as updating metadata and collections on the device. And, while there are options to do some extra things, it makes the minimum changes needed for a kepub to work. The disadvantage is that it does the work each time the epub is sent to a device (more a concern if you have multiple devices).

Use kepubify to convert the book. (Note: I have never actually used kepubify and my knowledge comes from @geek1011's comments, the support page and looking at the code.) The big advantage is that you don't need calibre and it is reported to be faster. The disadvantage is that for a calibre user, you need to step outside calibre to do something, and you will probably need an extra step or two to achieve the desired result. And you are probably doing the work each time you send the epub is sent to a device (more a concern if you have multiple devices).


To me, the choice is simple: If you use calibre, use the KoboTouchExtended driver. If you don't use calibre, use kepubify. The main time kepubify would be an advantage to a calibre user is if they are setting up a new or factory reset device and are sending a lot of books to the device.

[geek1011]

Originally Posted by davidfor

But, in this very thread, you admitted you didn't know what the KoboTouchExtended driver did. You made a statement about what it did which was very wrong and you were corrected.

And, the fact you don't know what it does is a bit strange. To paraphrase what you state here, I've answered the question of what the KoboTouchExtended driver does way too many times. And in at least one discussion with you (the thread you opened about how to create the kepubify plugin).

I can't think of a way for the KoboTouchExtended driver to stall. The conversion could, but you would need a really bad book.

The latter is an explicit decision to support books that are reasonably error free when being send to the device. If there are errors with books that the device can handle fine, if they are reported, they will probably be fixed. And if you have an example of the former, I would be interested in seeing it.

But, I will vehemently disagree with the "less intrusive". We have discussed the fact that kepubify does things that are not documented. And how is "removes leftover Adobe DRM tags, cleans up MS Word tags, and cleans up the html" less intrusive? Especially when there is no option to turn this off. The KoboTouchExtended driver explicitly only does what is needed to have the book as a kepub on the device. Or are you thinking of a conversion with this statement?


Yes, if you are using calibre, then seriesmeta is unneeded.

But, maybe all my comments are pointless as maybe you are referring to the Kepub Output plugin and hence a conversion in calibre. If you are, then a lot of your points are valid and all I can say is that this is the way it works. But, most people talking about using calibre to get books onto the device a kepubs are talking about the KoboTouchExtended driver.

Lets put this simply, the following are the ways I know of to get a book on a Kobo device as a kepub:

Buy the book from Kobo and sync either of WiFi or via the desktop application. The advantage of this is that Kobo is doing the work to get the book into the correct format. And you can sync the reading status between devices and their apps. The disadvantages are that your book choice is "limit" to what Kobo have in their store and you have to buy the book.

Convert the book in calibre. This uses the Kepub Output plugin and puts the book through the full calibre conversion pipeline. The advantage of this is that it can be used to convert from any format that calibre supports and if you have multiple devices the conversion is only needed once. The disadvantage is that it will add the kepub to the calibre library and you can't do much with it there except send it to the device.

Use the KoboTouchExtended driver to transform the book to a kepub when sending it to the device. The advantages are that you don't need to store the kepub, that calibre will convert to epub if you only have some other format in the library and the driver (via the built-in KoboTouch driver) supports other functions such as updating metadata and collections on the device. And, while there are options to do some extra things, it makes the minimum changes needed for a kepub to work. The disadvantage is that it does the work each time the epub is sent to a device (more a concern if you have multiple devices).

Use kepubify to convert the book. (Note: I have never actually used kepubify and my knowledge comes from @geek1011's comments, the support page and looking at the code.) The big advantage is that you don't need calibre and it is reported to be faster. The disadvantage is that for a calibre user, you need to step outside calibre to do something, and you will probably need an extra step or two to achieve the desired result. And you are probably doing the work each time you send the epub is sent to a device (more a concern if you have multiple devices).


To me, the choice is simple: If you use calibre, use the KoboTouchExtended driver. If you don't use calibre, use kepubify. The main time kepubify would be an advantage to a calibre user is if they are setting up a new or factory reset device and are sending a lot of books to the device.

In general, I have been referring to my experience with the full conversion, as that the main thing I looked at when first trying Calibre. And the fact I don't totally know what it does refers mainly to the difference between the two processes (I wasn't even totally aware that there was one until it was mentioned here).

About the less intrusive part, I am not just referring to KTE, but the fact it requires Calibre and everything it comes with. And I admit kepubify does slightly more then it strictly needs to (but that's about as much extra as it does), but I disagree the HTML cleanup is unnecessary. For cleaning up the HTML, I those fixed were for actual issues I encountered in books, for example self closing title tags, which cause the document to appear empty in stricter parsers such as the one Kobo uses. Note that cleanup doesn't mean beautifying or validating or anything of that sort, which is intrusive. Invalid HTML will remain as invalid as it used to be, but as little as possible more. Badly formatted HTML will mostly remain that way. I have also had issues with the MS word tags, although I forgot the exact case. But, you have a point about the Adobe stuff being a bit of an extra.

I have had some conversion issues with The C++ Programming Language by Bjarne Stroustrop. I don't recall if it was due to KTE or the conversion, but I remember trying multiple times and having it hang or stall.

That two paragraphs sums it up pretty well. It's also why I mostly abandoned making a kepubify plugin for Calibre, but may still implement an option to convert an entire library. It is possible to convert an entire Calibre library using kepubify and still use it to manage the books (and multiple people have used this before). Basically, you drag the library on to kepubify, you batch rename the converted ones to .kepub, then you merge it back. You either need to already have kepub entries, update the metadata in the books and reimport as another format, or do a database trick (I think I have the SQL somewhere in one of my emails). I think I detailed most of the process somewhere else before.