How to force auto-convert to EPUB on emailing to use MOBI as input?

[lisainchicago]

@davidfor & @kovidgoyal

Thank you so much for trying to figure this out with me.

Obviously, I can re-create this consistently, so... to do a clean test I created a new Calibre Library. Then restarted Calibre.
Checked the settings (images attached).

Then I went to Amazon, "Manage your content and devices", and chose "download & transfer via USB" on a book I have purchased.
The book was downloaded to my Downloads folder, so it was auto-added to Calibre & converted to MOBI instantly. (image is attached)

Then I chose Connect/Share, Email to..., and chose the Paperwhite alias. Received the "autoconvert to EPUB question" (image attached). Answered yes and email was sent. (job image is attached).

Next images show that the EPUB was created and that they do exist in my file system.

Also attached is an image of the email from Amazon saying it failed.


I'll save that EPUB and now manually convert the MOBI to EPUB and email. (I assume it will work.) And also save that EPUB. Not sure what I'd be looking for precisely though. Do you want me to post them here? Not sure if that is allowed.

I'm running:
Mac Mini M1 Monterey v12.3.1
Calibre 6.1
deDRM 10.0.3
(however this was all happening with calibre 5.14 and deDRM 7.1.0 - I upgraded yesterday in case that was the problem)

Here is an additional link to the images in case its easier to view them on Imgur: https://imgur.com/a/W2bYUFY

[lisainchicago]

OK, here's something else I noticed...
When I went to manually convert (Convert Books, Convert Individually), the INPUT FORMAT was defaulted to AZW3 (seems like it should have been MOBI since MOBI is ahead of AZW3 in my preferences).
I changed it to MOBI, changed the Output Format to EPUB and converted, emailed that EPUB to my Kindle and, of course, Amazon accepted it.

I then deleted that EPUB and tried to just email myself the book (auto convert question was asked and I selected yes) and NOW that EPUB was received at Amazon with no problems.
I quit and re-opened Calibre and now the Input Format on Convert Individually is saved as MOBI.

So I tried a different book, downloaded, auto added, auto converted (images attached).
Then took a look at the Convert Individually settings. Once again, Input Format is defaulted to AZW3. Even though the first book is still defaulting to (correctly) to MOBI. (images attached).

This seems to indicate that there is definitely some bug with the Input Format default preference. Once you individually convert a book it seems to save that preference, but it's only for that book. Each new book added reverts back to using AZW3 as the input.

https://imgur.com/a/kGj5MWl


I will now run Calibre in debug mode whilst adding a new book and report back.

FYI: I posted a very in-depth post previous to this one that is still awaiting moderation. It includes a lot of images, so that might be why it's not here at the time I'm writing this. If it shows up, it should be read BEFORE this one.

[lisainchicago]

I started Calibre in Debug mode and did all the same steps...

Images here: https://imgur.com/a/2idmfkK

Grabbed a purchased book from Amazon, saved it to downloads folder, Calibre auto added/converted. (images 1 & 2)
Then I emailed it to my Kindle. (image 3)
Proof it was converted and emailed. (images 4, 5, 6)
Amazon's failure email. (image 7)

Debug .txt file is attached (I redacted only my last name with xxxxxx for privacy).

After re-opening Calibre, I did check to see what Input Format defaulted and it was incorrectly AZW3 (as expected) for this newly added book. (image8). But West with Giraffes (the one I individually converted) still defaults to MOBI, and Golden Poppies (which I did not individually convert) defaults to AZW3.

Let me know if there's any other info you need to get this bug fixed! Thanks!

[BetterRed]

Originally Posted by lisainchicago

. . .

This seems to indicate that there is definitely some bug with the Input Format default preference. Once you individually convert a book it seems to save that preference, but it's only for that book. Each new book added reverts back to using AZW3 as the input.

This may be relevant ==>> How options are set/saved for conversion

There are two places where conversion options can be set in calibre. The first is in Preferences->Conversion. These settings are the defaults for the conversion options. Whenever you try to convert a new book, the settings set here will be used by default.

You can also change settings in the conversion dialog for each book conversion. When you convert a book, calibre remembers the settings you used for that book, so that if you convert it again, the saved settings for the individual book will take precedence over the defaults set in Preferences.

There's a Restore defaults button, bottom left of the conversion dialogue.

BR

[lisainchicago]

Originally Posted by BetterRed

This may be relevant ==>> How options are set/saved for conversion



There's a Restore defaults button, bottom left of the conversion dialogue.

BR

I'm not sure what specifically you thought might be relevant to the issue I'm having. I'm willing to try anything, so maybe you can help me understand what you were thinking...

That being said, in looking at the settings in Preferences>Conversion>Common Options>Page Setup, I had my "input profile" set to "kindle". I tried changing that to "mobipocket books", restarted Calibre, and then tried the whole process on a newly added book again, but no luck. EPUB was still rejected by Amazon. And when I go into the Convert Individually dialogue, the input format still defaults to AZW3.

I'm also not sure why you suggested "Restore Defaults", but that doesn't change anything. I think AZW3 is the "default" input format, even though I have MOBI above AZW3 in Preferences>Behavior>Preferred input format order.

[lisainchicago]

@davidfor & @kovidgoyal

Just wanted to add, because I'm nothing if not thorough, I did try disabling deDRM, restarting Calibre and ran through the whole process again. Same issue. So I don't think deDRM is causing any problems.

[BetterRed]

The word 'may' implies uncertainty, and users often fail to notice the Restore defaults button.

BR

[lisainchicago]

@kovidgoyal

Here's something interesting based on what @BetterRed was alluding to...

1. Download AZW3 book file from Amazon.
2. Calibre auto adds/converts to MOBI.
3. I right-click on newly added book, Convert Books>Convert Individually. Dialogue pops up with AZW3 as the Input Format. I click Restore Defaults, nothing appears to happen, so I cancel out of dialogue.
4. AGAIN, I right-click on newly added book, Convert Books>Convert Individually. Dialogue pops up with MOBI now as the Input Format. (Interesting, huh?) I cancel out of dialogue.
5. I right-click Connect/Share>Email to my kindle. Calibre asks to convert to EPUB, I click YES.
6. EPUB is created and emailed.

And it WORKS!

I believe what this is telling us is that Calibre is recognizing that MOBI is set as the default input (thus the Restore Defaults setting it right on exiting/re-entering that dialogue box).
However, it's not recognizing it after the auto add/convert. Seems like because it already did a conversion once from AZW3 to MOBI on the auto-add, AZW3 is now saved as my Preferred Input for this book. So when I email at this point, the auto-conversion to EPUB uses the AZW3 as the input.

I feel like we're now in "it's not a bug it's a feature" territory, but that's not how I really want it to work. I'd like there to be some setting where the Preferred Input Format Order is always respected. At least for converting on emailing.

[kovidgoyal]

Like I said in my previous post, as long as the book *has not been converted before*. If it has been converted calibre will use the previous conversion, as a template.

[lisainchicago]

Originally Posted by kovidgoyal

Like I said in my previous post, as long as the book *has not been converted before*. If it has been converted calibre will use the previous conversion, as a template.

Yeah, I understand that now. As I said, this was all part of the automatic add/convert function so that wasn't obvious to me.


It would be nice if you could add this adjustment so that we could more automatically deal with this change Amazon is implementing. Without you making this adjustment, the workflow has an extra step:
1. Download the AZW3 from Amazon.
2. Let Calibre auto add/convert to MOBI.
3. Manually convert MOBI to EPUB.
4. Email to Kindle.

If you had a setting (either on emailing or just in general) that would force conversions to always use the preferred input format, that would eliminate step 3 above.

[kovidgoyal]

Not something I am interested in. The only reason this becomes neccessary at all is because amazon's epub intake is totally broken. they really need to fix that.

[chaley]

Originally Posted by lisainchicago

It would be nice if you could add this adjustment so that we could more automatically deal with this change Amazon is implementing. Without you making this adjustment, the workflow has an extra step:
1. Download the AZW3 from Amazon.
2. Let Calibre auto add/convert to MOBI.
3. Manually convert MOBI to EPUB.
4. Email to Kindle.

If you had a setting (either on emailing or just in general) that would force conversions to always use the preferred input format, that would eliminate step 3 above.

An alternate workflow that might be easier is to use Bulk Metadata Edit to remove saved conversion settings from books. The "Manually convert MOBI to EPUB" becomes "Remove saved conversion settings", which can be done in one go for multiple books. After that, emailing any book will convert from MOBI.

[JSWolf]

Sending an ePub to Amazon via email to then download to your Kindle should not be nearly this difficult. A have to agree that the ePub conversion is very much broken.

[CathyA]

1. Download the AZW3 from Amazon.
...
4. Email to Kindle.

If you're starting with an AZW3 file that's already in your Amazon library, why do you need to email it back to Amazon?

[JSWolf]

Originally Posted by CathyA

If you're starting with an AZW3 file that's already in your Amazon library, why do you need to email it back to Amazon?

To be able to sync on multiple Kindles or Kindle apps. But IMHO, I don't see the need. Just read different books on your different devices.