Table of contents does not have all entries

[Carlos_ER]

Yes, that's right, I was converting from Libre Office ODT to EPUB every time.

I thought that the "original_epub" file was just a backup copy from the previous conversion attempt, I never imagined it could be taken as source for another conversion.

If it is taken as source, I'd like to disable its creation so as not to get unexpected results, or I'll have to look for them manually and delete them in all directories. And may restart calibre in the process.

[BetterRed]

Carlos - enter formats:"=ORIGINAL_EPUB" into the search bar

select all books, click the down arrow on the Remove button, select Remove files of a specific format, select ORIGINAL_EPUB

Job Done.

If you deleted the files manually, then the database will be out of synch with the book folders. My advise is to right click the Library button, select Library Maintenance->Check Library - it will first check the database integrity, it will then offer to "check the files in the library..." click Yes - it will rattle around and present a list of anomalies and options to resolve them - I can't recall the details, because I don't have an anomalies.

There is no need to add, delete or change data or names in the Calibre Library folders manually. Everything can, and should be done from within Calibre. At its heart Calibre has a relational database which is logically associated with the library folders and their contents, if one alters things in the author/book folders then things can go wrong. My advice is 'look but don't touch'.

BR

[Carlos_ER]

Interesting. I did that search, and it found one - however, the operating system file browser could not find any. So I went to library maintenance, restore data base, and repeated the search: none. A ghost.

So the database was also corrupted.

[BetterRed]

Originally Posted by Carlos_ER

Interesting. I did that search, and it found one - however, the operating system file browser could not find any. So I went to library maintenance, restore data base, and repeated the search: none. A ghost.

So the database was also corrupted.

No it wasn't corrupt, its database said it had one - a search interrogates the database not the file system - see my updated post above

If you use Calibre to maintain the library things wont go wrong, if you try to use the OS to maintain the library then things will go wrong.

It's a hard lesson to learn as we're all accustomed to manipulating files, but Calibre isn't a file based system - its a database system. It just happens to use the file system as the place to store large lumps of data - like epubs, jpgs, odt files etc. Desktop database systems are not suited for storing large lumps of data, at least not the free ones or the widely known ones that run on the raft of operating systems that Calibre supports.

BR

[DoctorOhh]

Originally Posted by BetterRed

But Doc, he was converting from Libre Office ODT to EPUB

To begin with, yes.

Originally Posted by BetterRed

The only action I know that creates an ORIGINAL_EPUB is to convert EPUB to EPUB, which I sometimes do inadvertently. I keep a beady eye on Formats in Book details and delete them when I do that.

At some point he did a ePub to ePub for multiple books. The TOC magically had the correct amount of entries after he removed the original_epub.

I could be wrong but all indicators are that it was operator error at every step. At some point he lost track of the automatically filled in source format during conversion.

Originally Posted by BetterRed

And if Carlos_ER was converting ODT to EPUB, and he was resetting defaults then why would the existence of an ORIGINAL_EPUB have any effect.

Exactly, calibre can get confusing. I believe if, Under Preferences - Behavior, he has EPUB at the top of his Preferred input format order then (if it exists) EPUB is automatically the source format listed in conversions.

Originally Posted by BetterRed

I just looked at my preferences->behaviour. EPUB was at the top of the Preferred Input Formats, again

I see you have experience with mistaken epub conversions too.

Originally Posted by Carlos_ER

I thought that the "original_epub" file was just a backup copy from the previous conversion attempt, I never imagined it could be taken as source for another conversion.

If it is taken as source, I'd like to disable its creation so as not to get unexpected results, or I'll have to look for them manually and delete them in all directories. And may restart calibre in the process.

It's creation isn't the problem it is only created for Epub to ePub conversions and is only used as a source for ePub to anything conversions. Don't have ePub as your source and you'll be fine.

[BetterRed]

Originally Posted by DoctorOhh

I see you have experience with mistaken epub conversions too.

Who hasn't, I ask

Originally Posted by DoctorOhh

It's creation isn't the problem it is only created for Epub to ePub conversions and is only used as a source for ePub to anything conversions. Don't have ePub as your source and you'll be fine.

Now that Polish Books exist, why would one want to do an EPUB to EPUB conversion, other than habit?

I've sometimes inadvertently created ORIGINAL_RTF, ORIGINAL_MOBI...

That's why it would be nice to have a tweak that forces an Are you sure? if I attempt to convert like to like - not saying I'd never want to do it, but its a rare exception when I do.

BR

[Carlos_ER]

BetterRed:

Thanks for the extended explanation. I'll try remember do not use the OS to delete files. If not, I'll have to tell Calibre to check the database consistency when I suspect problems.

However, I do edit the Llibre Office files in the Calibre directories, then ask for a conversion - else, I have to delete the book, and add it again from the original directory. I hope that is safe.

[kovidgoyal]

Lots of people do epub to epub conversion because calibre generated epubs often work with devices where the original epub fails, or because the conversion automatically fixes unreadable font sizes in the input document, or it allows you to easily tweak aspects of the books look and feel via the various conversion options to suit your tastes.

*You* might not want to do epub to epub conversions, dont assume everyone else is like you.

The OPs problem has nothing to do with original_epub, it was caused by switching the input format for the conversion from odt to epub. And note that this does not happen automatically. If you run a conversion on a previously converted book, the same input format as was used the last time is chosen, unless that input format is no longer available. While its conceivable that there could be a bug in that code, I doubt it, and I'd need some reproducible steps to convince me otherwise.

[Carlos_ER]

Originally Posted by DoctorOhh

To begin with, yes.

At some point he did a ePub to ePub for multiple books. The TOC magically had the correct amount of entries after he removed the original_epub.

I was never aware that I was doing an epub to epub conversion. I did try a multiple conversion of 3 books, but I intended it to work on the .odt files, not on epubs.

I could be wrong but all indicators are that it was operator error at every step. At some point he lost track of the automatically filled in source format during conversion.

Exactly, calibre can get confusing. I believe if, Under Preferences - Behavior, he has EPUB at the top of his Preferred input format order then (if it exists) EPUB is automatically the source format listed in conversions.

Let me see... yes, I have, that's the default.

I don't know if I should change that.
I have some bought or downloaded epubs that I do not want changed, I don't know what other effects changing that default would have.

I see you have experience with mistaken epub conversions too.

It's creation isn't the problem it is only created for Epub to ePub conversions and is only used as a source for ePub to anything conversions. Don't have ePub as your source and you'll be fine.

I never noticed it was using epub as a source, I never intended that.


Ok, lesson learned... thanks.

[DoctorOhh]

Originally Posted by kovidgoyal

The OPs problem has nothing to do with original_epub, it was caused by switching the input format for the conversion from odt to epub. And note that this does not happen automatically.

Thanks for the clarification, I couldn't recall the exact details.

Originally Posted by Carlos_ER

However, I do edit the Llibre Office files in the Calibre directories, then ask for a conversion - else, I have to delete the book, and add it again from the original directory. I hope that is safe.

Many of us do the same thing to epubs using Sigil. It has proven to be safe in the past.

Originally Posted by Carlos_ER

Ok, lesson learned... thanks.

Every week there are new features added to calibre, so calibre has taught us all lessons in the past and most likely will again in the future as we stumble over new features.

Welcome to the club.

[kovidgoyal]

Originally Posted by DoctorOhh

Thanks for the clarification, I couldn't recall the exact details.

I'd guess that the OP was doing a bulk conversion, for which the input format is chosen based on the preferred order, since there is no single previous setting to be used anymore.

[Carlos_ER]

Originally Posted by kovidgoyal

*You* might not want to do epub to epub conversions, dont assume everyone else is like you.

No, of course. I can see its uses :-)

The OPs problem has nothing to do with original_epub, it was caused by switching the input format for the conversion from odt to epub. And note that this does not happen automatically.

I assure you, I never noticed this was happening. It is of course possible that I clicked on something I should not, but I never noticed.

If you run a conversion on a previously converted book, the same input format as was used the last time is chosen, unless that input format is no longer available. While its conceivable that there could be a bug in that code, I doubt it, and I'd need some reproducible steps to convince me otherwise.

I'm trying again. When I click on a previously converted file from odt, the dialog shows at the top-left that the source is EPUB. I change that to ODT and convert.

Then I convert a second time, and the dialog has now ODT.

I restart calibre, and the selector stays at ODT - for that book.

But at sometime, somehow, the selector was EPUB. I'll watch that selector in the future. In fact, all 4 odt books had the selector at EPUB, I have converted them all again. Another book I converted a year ago from odt had also the selector at EPUB.

I look at to other books I'm testing (lyx --> xhtml --epub) and the selector there was epub, not zip. Changed that too.

So initially the selector is always EPUB - I have to watch that and change that thing when starting a conversion in the future.

Of course, I did a database rebuild a while ago... maybe they were reset.

Ok, test again. Delete two books. Add them again from the original directory. Convert - good, the selector displays "ZIP". Restart calibre - still ZIP. So it is working as it should, and I do not know when those books conversions changed to epub as source (before the database restore). Something I did that I did not notice.

Thanks -- I have to get used to this :-)

[Carlos_ER]

Originally Posted by kovidgoyal

I'd guess that the OP was doing a bulk conversion, for which the input format is chosen based on the preferred order, since there is no single previous setting to be used anymore.

Yes, indeed, I tried that once, as I was converting 4 books.

I was not convinced that was what I wanted, and on the next attempts I selected the 4 books and clicked individual conversion instead.

So that was it.

[DoctorOhh]

Originally Posted by kovidgoyal

I'd guess that the OP was doing a bulk conversion, for which the input format is chosen based on the preferred order, since there is no single previous setting to be used anymore.

I think you hit the nail on the head. My brain sometimes naps while I walk through life trying to continue as if it was wide awake. .

[BetterRed]

Originally Posted by Carlos_ER

BetterRed:

Thanks for the extended explanation. I'll try remember do not use the OS to delete files. If not, I'll have to tell Calibre to check the database consistency when I suspect problems.

However, I do edit the Llibre Office files in the Calibre directories, then ask for a conversion - else, I have to delete the book, and add it again from the original directory. I hope that is safe.

No that's fine

The main thing is don't change file or folder names, or delete files. But you don't have to go to the file system to edit a file either, IMO best practice is to start Writer from with Calibre.

Assuming that you do NOT have ODT set to use the Calibre Viewer (Preferences->Behaviour) then you can use Alt/V on a book and select ODT, that will start Writer providing its associated with ODT in the OS.

Or you can click on the ODT format in the Book Details panel, btw if you right click a format in Book Detail panel you have the option to save the format to disk or delete it. Or you can click on the ODT in the Edit Metadata panel. Or you can use the Open With plugin.

So, why is it best to start Writer from within Calibre?

When Writer opens a file and you make a change to it then Writer 'locks' the file until you save and close the file (at least my Windows and Mint versions do that - they are old Open Office versions from Sun).

When you change say the Title on a book in Calibre, then it will check to see if there are any locked files for the book - if there are then the change will be rejected. This is one of the mechanisms that Calibre uses to keep its database in synch with folders.

When a book's Title is changed in Calibre a new folder is created, the files are moved to it and the old folder is deleted. If Calibre didn't do the locked file check and allowed the Title change to proceed, then when Writer saved the file then the folder the file came from wouldn't be there so... what happens next depends on the OS and file system in use. From memory Windows will (re)create the folder, and Mint will throw an error...

BR