Question about keeping both epub AND mobi formats

[unboggling]

To expand on what I said earlier: excepting some PDFs, I keep only EPUBS even though I read mostly MOBI sent to Kindle "on the fly". I consider these EPUBs, once cleaned up, the master copies. I've rarely had problems with original MOBI conversion to EPUB "master copy" then a later conversion "on the fly" back to MOBI again. In the few cases when I tried to read the result on Kindle and found problems not previously evident or fixed after my initial assessment, it wasn't difficult or time-consuming to fix those specific problems to make a better EPUB "master copy" then successfully convert that "on the fly" to MOBI.

There are other reasons besides those I mentioned in the prior post (KISS, limit confusion, EPUB fixability). One is to speed up performance. I use a custom #formats column because it is useful, particularly during initial assessment after adding, to show the formats in booklist rather than looking for them sequentially by title on the Book Details panel or Edit Metadata form. I suspect that if I had 3 or 4 formats per book title in a large library, calibre would need more time to startup initially or accomplish any other operation where formats needed to be iterated through. As a side benefit, minimizing the number of formats per title also reduces time and space needed for backups and library copies.

Yes disk space is cheap, but I also have backups. The calibre library multiplies itself. One automatic hourly backup to one disk. One automatic daily backup of a separate disk that holds a (weekly) backup copy of the calibre library folders. Original library, backupOfDisk1. copy of original, backupOfDisk2. So for 3 formats per title it's not just 3 times the space to accommodate those extra formats. it's 3 formats x 4 disk instances, complicated further by factoring in calibre and processor time per frequency of relevant operation.

I understand many experienced calibre users feel strongly about keeping the "Original Incoming Format." After a brief assessment at arrival into calibre and occasional cleanup where needed, I've found that keeping "Original Incoming Format" isn't necessary, and dealing with the few exceptions per 1000 EPUB "master copies" is worth the trade-off. The time and hassle saved outweigh perhaps having to buy a few extra books per year to replace any that I've previously made unfixably serious mistakes with -- a last resort I haven't needed once during 12+ months doing this basic workflow and reading this workflow's products.

That it works well for me doesn't mean it will work well for other people. There are many factors: Choices of original sources and material, which vary in format quality and format complexity. Skill at assessment and cleanup when necessary after adding books. Time available for doing initial assessment and cleanup. Type of reading material (in my case mostly fiction with few graphics and minimal complex formatting). And personal and subjective things such as tolerance-level and annoyance-level for relatively minor imperfections in the formats.

[speakingtohe]

Not exactly what you are asking, but I have a custom column where I rate each book on quality and or problems to clean up later when time permits.

Many pdf`s as you have noted do not convert well on the first effort. Some never do.

Mostly I delete .lit .lrf etc. when the epub looks good. But I am not fanatical about it. Duplicate titles and extra tags annoy me more than extra formats.

Helen

[unboggling]

Originally Posted by speakingtohe

Not exactly what you are asking, but I have a custom column where I rate each book on quality and or problems to clean up later when time permits.
Helen

I do that too. Since a title can have multiple formats, and each format of that title can be of varying format quality, then some confusion arises whenever there is more than one format for any one title. Also any format previously unconverted by me with my particular settings may produce unknown problems in future conversions. Which particular format is referred to by the format-quality-rating or any format problem tags? To avoid that confusion I assess all formats per title, choose the best format, convert it to EPUB, and fix any problems right away. Assess the result to be sure. Enter format-quality-rating. Delete all other formats, except problem PDFs whose resultant EPUBs are too unfixably annoying. Now there is no confusion of what format-quality related tags apply to what format.

If I decided I needed to keep multiple formats per title, I would want to break those formats out into one format per record and thus have duplicate records except for the varying formats. This would at least avoid the confusion of which format-level tag applies to which format. But to me it would be annoying, unwieldy, and unnecessary.

With duplicate records each containing only one format, I compare, keep best, delete worst.

[kiwidude]

It doesn't have to be as complicated as you describe. I just have a #done custom column keeping whether I have completely finished with processing that book. So to begin with there could be multiple formats which I keep only the best one of, convert to ePub if needed, delete the others, convert to mobi for my kindle and tick the done column. Any issues I later notice I can fix in the ePub and reconvert. Only books with done ticked are ones eligible to send to my kindle, easily searched or restricted.

I refuse to keep formats I will never use like lit, ePub is as good a choice of master as any with editability via tweak and Sigil so that is a no brainer to have. But since I read on my kindle I want my mobi file ready to go so keep that too, I want to plug and go when I top up, not wait for conversions.

[unboggling]

Originally Posted by kiwidude

But since I read on my kindle I want my mobi file ready to go so keep that too, I want to plug and go when I top up, not wait for conversions.

Good point. The lagtime of conversion on the fly is mildly annoying, as is deleting the MOBIs later to end up with just the EPUB master. I've been thinking about it, and I'm not sure if there really is a performance penalty with multiple formats in large library, which may have been something of a common-sense assumption.

[theducks]

Originally Posted by unboggling

Good point. The lagtime of conversion on the fly is mildly annoying, as is deleting the MOBIs later to end up with just the EPUB master. I've been thinking about it, and I'm not sure if there really is a performance penalty with multiple formats in large library, which may have been something of a common-sense assumption.

the 'Data' table grows a record for each format, the 'Formats' Index grows.

These are both small (record wise), so you would need a pretty huge library to seriously impact day-to-day performance (there are some tasks that will see a jump in time. These are not done regularly.)

[unboggling]

Originally Posted by theducks

the 'Data' table grows a record for each format, the 'Formats' Index grows.

These are both small (record wise), so you would need a pretty huge library to seriously impact day-to-day performance (there are some tasks that will see a jump in time. These are not done regularly.)

Hmmm. Depends on which tasks. I'd assume initial calibre startup when loading the DB. And any display tasks for the booklist that involve my custom column #formats which is based on format.

Here are some I know from experience: externally copying entire library for backup purposes at system level. Saving out formats for large number of records. Searching format:PDF is significantly faster than searching #formats:PDF, on the order of 50x to 100x. Any searches that iterate through the default field format have to take longer with multiple formats, but even at 2x or 4x the number of formats per record it would probably require less than a second per each subset of 10K books, on my MacBookPro 2.2 GHz Intel i7 quadcore.

There are probably other affected tasks I've missed remembering or knowing about.

[Vidfamne]

Originally Posted by dwig

If you were to import a library with multiple formats with calibre's "Automerge" turned off you will get separate entries for each format. If one title existed in three formats they would import as three separate books. If, on the other had, Automerge was enabled they would import as a single book entry containing three formats.

Is there a way to merge these after import is done? I've seen the join books option, but as I use non-english version am a little unsure if the 'join' just appendes the text after oneanother (an example would be to merge/join; chapter1 + chapter 2 + chapter 3) into one book or if it merges/joins the different formats (as in book.epub + book.mobi would be book with formats epub, mobi if you understand my babbling )

[speakingtohe]

It is the M function for merge book records in the english version. Forth option under edit metadata menu

[louwin]

My library is in four actual libraries - Adult Fiction, Adult Non Fiction, SciFi and Children and is about 35K books. It was given to me and previously maintained under Linux or Unix hence the huge number of extra long filenames (which Windows does NOT like). Like over a thousand books are "lost" accordingly. I can't see anyway to rescue these books except manually

There is SOME duplication in Adult Fiction and SciFi and the formats are generally in pdf, txt, mobi and epub formats.

The Adult Non Fiction library is about 10K books and 70gigs. I am cleaning them up but haven't tackled this section yet.

I intend to merge them all into one library in the not too distant future (unless someone convinces me otherwise ). I have already merged Adult Fiction and SciFi.

[Vidfamne]

Originally Posted by speakingtohe

It is the M function for merge book records in the english version. Forth option under edit metadata menu

Thanks for the quick response!