One book, multiple versions of one format

[Katsunami]

Hi

Is it somehow possible to have one book, containing multiple versions of the same format?

I was thinking: The Kindle does not (officially) support adding user fonts. My main library is entirely EPUB-based, while the Kindle uses AZW3. That's nice, because now I can convert all books to AZW3 and embed the font I'd like, if I so choose.

Now, let's say someone has a Kobo. If this person converts the EPUB to one with built-in font, the original EPUB will be called ORIGINAL_EPUB. While it works, I personally don't really like it, as there are other functions that use or create that format.

What happens if someone has 3 EPUB readers, and wants different conversions for all of them, for some reason? As far as I can see, you can't have three EPUB's in the library and then choose which to upload to the connected reader. You could of course create a seperate library for each e-reader, copy a book there, reconvert the file, and throw ORIGINAL_EPUB away. When connecting that specific reader, switch to that library.

However, I'd much prefer to give a format a name, when converting. Then it would be possible to take the original EPUB, and convert it for any reader, like so:

Dickens, Charles - A Christmas Carol.epub (original file, input file for later operations)
Dickens, Charles - A Christmas Carol.kobo_glo.epub
Dickens, Charles - A Christmas Carol.kobo_aura.epub
Dickens, Charles - A Christmas Carol.sony_t1.epub
Dickens, Charles - A Christmas Carol.bokeen_gen3.epub

When sending a specific format to a device, one would be able to choose.

Is this something that can be done already, somehow, or if not, would it be possible to implement, without re-writing half of the program?

Is it possible to add a name to a format

[Adoby]

You can have as many versions of the same book as you want, if you treat them as separate books. And you can use tags or a custom column to keep track of what device the book is formatted for.

[Katsunami]

Yes, obviously. You can be sure I thought about that already. I don't want to treat them as seperate books, because Calibre doesn't do that either. It clutters the library. If I have to use a copy of the book, I'd rather use a main library, and a seperate library for each ereader.

[Sabardeyn]

I cannot imagine a way to implement something like this under calibre's current file structure without entering them as multiple books. You are looking for a one-to-many database function and it doesn't exist in calibre at this time.

Kovid did mention altering the database structure to allow one-to-many situations, but I cannot recall if this was for a specific aspect only (multiple covers, editions, formats, reprints, etc), or in general throughout the database. I do recall Kovid mentioning it was a low priority project as other aspects of calibre would have to be substantially altered before this could be made to work.

At least, that is the overall impression I got, I might have misunderstood.

Despite the above, I'm not sure that what you want fits within the overall workflow method that calibre uses. My limited impression is that calibre is meant to take whatever original source ebook format you have and, on demand, to convert the book to a format that your software or hardware ereader can view at time of transfer. This means that ebooks are always as capable and functional as Kovid & Programmers, et al, can manage at the time of conversion. You always have and can re-convert as needed because you always have the original ebook file as the publisher released it.

What you are asking for, in effect, is that a book be converted either at the time it's added or on demand. The resulting ebook format would then be retained indefinitely - losing the advantage of any newly programmed conversion engine upgrades. Additionally, while hard drive space is inexpensive, their is little need to have multiple variant copies of the same ebook, differing primarily on screen resolution and CSS. (Alright, that might be oversimplifying things, but...)

Lastly, and I'm not sure I read this part right, but if you're getting rid of the original file, you can never reconvert. You're going to have the artistic problem of a "copy of a copy of a copy" with it's degrading quality. Particularly if you discover much, much later that their was a problem in that first conversion. Poorly converted ebooks tend to waste CPU cycles, causing slow book loading and page turns as the ebook's vague commands and structure are interpreted by the viewer's engine.

I think something that might make more sense with calibre's current functionality is a Load/Save Output Settings function. This would allow the various settings from the Preferences>Conversion section to be imported and exported, swapped as needed, when a user has multiple devices. In this manner all of the negative issues I mention above are resolved with minimal work by the user (after initial setup).

I'm not sure how many users would find this feature useful however.

[Katsunami]

I have a feeling that you are making this much more complicated as I intend.

Lets say, you have a book in AZW3 format, because you bought it from Amazon. However, you have a Kobo reader, which reads EPUB. So, you convert from AZW3 to EPUB, and upload the book.

If you have two ereaders, one Kobo and one Sony, but want different conversions for the same book, you would need two EPUB formats. Therefore, you would convert from AZW3 to Kobo_EPUB, and from AZW3 to Sony_EPUB.

This has to be possible; probably even by changing the output plugins so they save the format as <name>_EPUB (or <name>_whatever) instead of just EPUB. In that case, there would probably not be a difference in saving everything as it is now; it would just be another format; just like AZW3, MOBI, EPUB... you'd have <name>_<format>.

edit:

Originally Posted by Sabardeyn

Lastly, and I'm not sure I read this part right, but if you're getting rid of the original file, you can never reconvert.

I do not rely on Calibre to store my original files. I store them seperately. If I ever need to reconvert, I'll remove all formats from a book, and drop in the original files. Worst case, I re-import the file as a new book.

At this time, my base format is EPUB, and my reader's format is AZW3, so there is no conflict. I can embed the font I want, and convert. If I get another reader, I would just delete all AZW3 files.

I don't like "ORIGINAL_EPUB" (or any other such files) as the solution. IMHO it should be possible to designate a format that should NEVER be touched. (Possibly, use "Default input format" for this). If you then convert an EPUB to EPUB, the original (very first) EPUB should stay as it is, and the conversion would be called <name>_EPUB.

[BetterRed]

I've 'struggled' with this issue since I started using calibre a couple of years ago, and there have been many threads created seeking solutions. Two methods have or are working for me.

• Put them in an archive that Calibre wont process. Add the archive to the book and put a note in Comments - eg "The xxx archive contains the following additional formats...". Calibre will keep track of the archive through renames, library rebuilds etc. All you'll be able to to with is to open it with your unarchiver. From there you could drag a format into a 'scratch library' (via the file system) for additional calibre processing.
  
• Or keep them in a folder outside the library and insert a link to the folder into Comments labelled "Other formats are Here". With a list of what's there

I now use the second method in preference to the archive method - I keep the additional formats in my CalibreLibraries directory within a directory structure that mirrors the library structure :-

Spoiler:

CalibreLibraries 
    Main Library
        Author A
            Book 1
            Book 2
        Author B
            Book 3
            Book 4
                cover.jpg
                metadata.opf
                Book 4 - Author B.epub
        Author C
            Book 5
            Book 6
         metadata.db
     MainSupplementary
        Author B
            Book 4
                Book 4 - Author B.for jill.epub
                Book 4 - Author B.for jack.epub
        Author C
            Book 6
     Media Library
         ...
         metadata.db

Most of my books don't have 'supplementary files' which results in a sparse mirror, I can't say that this approach would work (procedurally) if that was reversed, I'd have to suck it and see. Off the top of my head if most of my books had 'supplementary files' then I'd probably use the archive 'solution'.

The most common reasons for me having two files of the same format for the same book are different editions (e.g. draft and final) and different languages. Also multiple film and TV mini-series productions, IIRC the BBC alone has done at least 3 Madam Bovary's and there are maybe a dozen films...

BR

[Katsunami]

Hmm, ok. That's too complicated for me. This is just a brainstorming session on my part, because I was experimenting with several different fonts to embed.

If I really get into the situation of having 2 ereaders that use the same output format, and my default EPUBs don't work (for example, because I want to embed fonts), I'll just use different libraries, like this:

Import book -> Calibre Library Workplace
*** do stuff ***
Copy book -> Calibre Library Main
Copy book -> Calibre Library Sony
Copy book -> Calibre Library Kobo

(Up until copying a book to a main library my current workflow already. I don't "do stuff" in my main library, except creating AZW3's, side-loading, and browsing.)

Then, in the seperate libraries, I'll just do a re-convert, embedding the fonts I want, and then destroy "ORIGINAL_EPUB". These libraries will then only be used to upload books to the readers.

If something significant would change in the main library, then deleting the changed books from the specific libraries, creating a new copy and a reconvert will take care of that.

[Sabardeyn]

Originally Posted by Katsunami

I have a feeling that you are making this much more complicated as I intend.

Not really. Most users don't need to retain multiple variants indefinitely on their HDDs. Users only need variants when transferring ebooks to their reading device. Users already complain that calibre uses too much hard drive space with it's mandatory file structure. If output profiles could be loaded/saved it would be a reasonable compromise between user's needs: little HDD space would be used, and complex variant formats could be (re-)created almost instantly (after creating the initial output settings profile).

Or, let me try getting my point across another way: Why must you have a *.kobo_glo.epub, *.kobo_aura.epub, *.sony_t1.epub, and *.bokeen_gen3.epub always sitting around on your HDD?

It would make far more sense to load your Kobo Aura output settings, select the books you want, and then send to device. Switch to your Sony T1 and then your Bokeen Gen3 and do the same. What would it take to load these settings... a minute, worse case?

Originally Posted by Katsunami

I do not rely on Calibre to store my original files. I store them seperately.

Good! I was seriously worried for your ebook collection.

Originally Posted by Katsunami

IMHO it should be possible to designate a format that should NEVER be touched.

I don't have a problem with being able to write-lock a specific format to avoid accidentally tampering with it. The naming convention used isn't something I'm particularly worried about. I'm unaware of issues with regards to the naming.

Originally Posted by BetterRed

The most common reasons for me having two files of the same format for the same book are different editions (e.g. draft and final) and different languages.

Do I understand you to be saying that you've only got one record for every title in your database? In effect there is one and only one Madam Bovary record to cover all print, movies, tv series, audio, images, photos, etc?

I completely understand the related supplementary files issue. I haven't really needed to deal with that in detail yet. But then I'm using multiple custom series columns which allow me to fork my numbering structure to allow for material that is related, but not specifically a part of any book title/series. So I can have say... Frank Herbert's Dune series, allow for the books that his son (and company) wrote, and still account for review, analysis, cliff notes, atlases, and anything else, regardless of author.

[BetterRed]

Originally Posted by Sabardeyn

Do I understand you [BetterRed] to be saying that you've only got one record for every title in your database? In effect there is one and only one Madam Bovary record to cover all print, movies, tv series, audio, images, photos, etc?

You got it - that's the objective and so far its working for me.

ETA : I also have author related supplementary stuff in the 'mirror' Author folders, I put the uri into calibre's Author Link property. Then when I click on the author name in Book details I get that folder rather than whatever I have in preferences. In there I can put multiple urls - to WikiP, the authors sites, publisher sites, obits etc etc. And things like radio interviews that are not book specific.

It would be good if all 'entities' had that link feature - eg Publisher and any custom column the user nominates being as an 'entity' - maybe it would useful for the Series property, as a place to put things about an entire series

BR

[Dopedangel]

IRC the only difference between an epub file for sony or other readers would be the cover size. And from what I can tell calibre updates that according to the device being send to. So keeping different versions for different readers is not needed.

[Katsunami]

Originally Posted by Sabardeyn

<snip calibre library size>

Or, let me try getting my point across another way: Why must you have a *.kobo_glo.epub, *.kobo_aura.epub, *.sony_t1.epub, and *.bokeen_gen3.epub always sitting around on your HDD?

If I have two or three e-readers and I want different conversions (or maybe different embedded font for each) without having to do a conversion before uploading the books, then all versions need to be on the HDD.

At this time, my calibre library contains 310 books. It's 410MB. Adding the AZW3 versions would take it up to around 850MB or so. If the library contained 3000 books, it would be 8.5GB. Add another two formats for a library of 17GB.

In my opinion, 17GB of data is insignificant on today's terabyte-sized hard drives. Many current games are larger than that.

Originally Posted by Dopedangel

IRC the only difference between an epub file for sony or other readers would be the cover size. And from what I can tell calibre updates that according to the device being send to. So keeping different versions for different readers is not needed.

That is your opinion; there isn't an option to embed fonts and add extra CSS for nothing.

[jackie_w]

@Katsunami,
I am having trouble understanding why you need to embed fonts in your master calibre epub at all. I also have several Kobo and Sony devices. I can do all the font customisation I need to do (small-caps, dropcaps, monospace etc etc) without ever needing to embed any fonts at all. There are better methods which can be set up either on the device itself (Sony devices) or, with Kobos, part device/part calibre Kobo device driver. Do you know about, or have you tried, any of these other techniques? If not I can try and help you if you like.

[Katsunami]

Originally Posted by jackie_w

@Katsunami,
I am having trouble understanding why you need to embed fonts in your master calibre epub at all.

I *don't* want to embed fonts in the main file in Calibre. That is why I would want to save different versions of the same format.

I also have several Kobo and Sony devices. I can do all the font customisation I need to do (small-caps, dropcaps, monospace etc etc) without ever needing to embed any fonts at all. There are better methods which can be set up either on the device itself (Sony devices) or, with Kobos, part device/part calibre Kobo device driver. Do you know about, or have you tried, any of these other techniques? If not I can try and help you if you like.

I use the font embedding as an example.

Embedding extra CSS is another one.

Not all readers render files the same way especially not when converted from other formats. Sometimes, extra CSS may be needed. For example, FeedBooks uses a specific chapter header markup, that does not work well on the Kindle Paperwhite 1. It can be fixed with adding extra CSS during conversion from EPUB to AZW3. Maybe the Kindle Paperwhite 2 renders the header as it should, and if so, I'd like to have another conversion to AZW3, but without the extra CSS.

[jackie_w]

What I was trying to say is that Sonys and Kobos have other ways of applying Extra CSS than by doing a calibre conversion. Both Sonys and Kobos can store an Extra CSS file on the device itself. Although the method of getting it applied to each epub (or kepub) does vary by brand/device. The Extra CSS may be font-related but other types of CSS can also work effectively. Given the nature of CSS specificity I wouldn't claim that all CSS problems can be fixed this way, but a little creativity can go a long way. If you've tried them all and it doesn't work for you, fair enough. I just wanted to make sure you knew that other potential options exist.

[Thasaidon]

Originally Posted by Katsunami

Hi

Is it somehow possible to have one book, containing multiple versions of the same format?


Is it possible to add a name to a format

This is only an idea but occurs to me it should be possible to add suffixes to each of the the file formats e.g xxxx.epub.aura, or xxxx.epub.t1. This should allow them to be imported into a single book title. It should then be possible to set up templates which strip off the unwanted suffix when you use Caliber to put them on the reader concerned.

Hope the above is clear.

I may be overlooking an obvious reason why this will not work but cannot think of one.