Email to Kindle - plugboard behavior inconsistent

[lairdb]

If I use Calibre's email feature to email to my Kindle address, sometimes the title plugboard is effective, sometimes it's not. It is consistent within a book (book X works, book Y never works) but I've tried tweaking various things and can't get it to ever take effect on book Y.

If I email book Y to my own email address, the plugboard is effective.

My only working theory is that there are multiple possible "title" metadata fields, and some other one than the expected is preferred by the Kindle PDOC importer.

Have tried both old and both for MOBI conversion.

Plugboard is
any format:email = {series:|| }{series_index:0>2s|[|]: }{title} -> title

Does anybody already know the solution, or how can I better troubleshoot?

[kovidgoyal]

both will not work Always use old.

[eschwartz]

To elaborate: calibre does not have the ability to update the metadata within the AZW3 component of an existing dual-MOBI (type "both"). This is due to the difficulty of robustly modifying the metadata headers in a dual-MOBI.

Kovid hasn't devoted the time to fix this, since it basically only affects people who email to their Kindle and want to receive AZW3 via the PDOC service. Which is a very small percentage of users, and it is simply not worth investing the time it would take to create and test a fix.

So you will probably just have to live with setting the desired title metadata during the initial conversion to dual-MOBI. Or use old MOBI.

[lairdb]

Originally Posted by eschwartz

To elaborate: calibre does not have the ability to update the metadata within the AZW3 component of an existing dual-MOBI (type "both"). This is due to the difficulty of robustly modifying the metadata headers in a dual-MOBI.

Kovid hasn't devoted the time to fix this, since it basically only affects people who email to their Kindle and want to receive AZW3 via the PDOC service. Which is a very small percentage of users, and it is simply not worth investing the time it would take to create and test a fix.

That makes sense -- but I'm still not seeing the expected behavior. I've set "old" in preferences, and deleted the MOBI copy locally, and deleted the Amazon copy, and re-sent via email, which prompts conversion... and still no plugboard result.

Looking more closely, I'm wondering if this is a clue. In the conversion log, I see:

Creating MOBI Output...
Serializing resources...
Splitting markup on page breaks and flow limits, if any...
		Splitting on page-break at id=calibre_pb_0
		Splitting on page-break at id=calibre_pb_1
	Split into 3 parts
Creating KF8 output
	Generating KF8 markup...
	Compressing markup...
	Creating indices...
Creating MOBI 6 output
Applying case-transforming CSS...
Parsing manglecase.css ...
Rasterizing SVG images...
Converting XHTML to Mobipocket markup...
Serializing markup content...
  Compressing markup content...
Generating MOBI index for a book
MOBI output written to C:\Users\Laird\AppData\Local\Temp\calibre_185vi_\tsnb7r.mobi

...despite:



Should it be creating KF8 and MOBI 6 if the option is set to "old"? Might that be related to what I'm seeing?

[lairdb]

Originally Posted by lairdb

Should it be creating KF8 and MOBI 6 if the option is set to "old"? Might that be related to what I'm seeing?

Further supporting this theory, here is a log snippet from converting an EPUB where I did get the plugboard result:

Creating MOBI Output...
Serializing resources...
Creating MOBI 6 output
Applying case-transforming CSS...
Parsing manglecase.css ...
Rasterizing SVG images...
Converting XHTML to Mobipocket markup...
Serializing markup content...
  Compressing markup content...
Generating MOBI index for a book
MOBI output written to C:\Users\Laird\AppData\Local\Temp\calibre_185vi_\zkxi9t.mobi

Note no "Creating KF8 output" appears.


ETA:

Increasingly believing this is a straight-up bug (or invisible behavior) in conversion preference handling. If I interactively request conversion using the right-click menu and select "Bulk convert" to MOBI, it picks up the configured preference, regardless of book.

On selected books, if I select "Bulk convert", it picks up the preference, but if I select "Convert Individually" it sets "both" and "[PDOC]" in the MOBI options, ignoring the configured preference. To be clear, this behavior differs on some books vs. others.

I'm suspecting the same behavior is happenign on the auto conversion invoked by Email.

[eschwartz]

This behavior (not respecting the Preferences to use old MOBI) is documented in the manual, see: https://manual.calibre-ebook.com/con...for-conversion

tl;dr
Once a book is first converted, it remembers the settings you used, to pre-load the Conversion dialog options.
You can reset that via the relevant option in Bulk Edit Metadata.

[lairdb]

Originally Posted by eschwartz

This behavior (not respecting the Preferences to use old MOBI) is documented in the manual, see: https://manual.calibre-ebook.com/con...for-conversion

tl;dr
Once a book is first converted, it remembers the settings you used, to pre-load the Conversion dialog options.
You can reset that via the relevant option in Bulk Edit Metadata.

Hmm. Granted that it's documented, but IMO, misfeature. At least there is an explanation.

“If the designers of X Windows built cars, there would be no fewer than five steering wheels hidden about the cockpit, none of which followed the same principles—but you’d be able to shift gears with your car stereo. Useful feature, that.”
- Marcus J. Ranum


(Also, the word "email" appears nowhere in that section, leaving it as a moderately obscure inference that the behavior carries across to email autoconversion. Which, in turn, reminds me of:

“Ken Thompson has an automobile which he helped design. Unlike most automobiles, it has neither speedometer, nor gas gauge, nor any of the other numerous idiot lights which plague the modern driver. Rather, if the driver makes a mistake, a giant “?” lights up in the center of the dashboard. “The experienced driver,” says Thompson,“will usually know what’s wrong.””)

[eschwartz]

Feel free to your opinion.
The calibre rationale is that people who have manually configured a conversion for a specific book, won't want to go back to the generic defaults.

I shan't argue for one side or the other.

...

Email autoconversion is actually not.

It is an email. If the format desired does not exist yet, it will first trigger a conversion.
Same thing applies for Send-to-Device.

Conversions follow the normal rules of conversions. I cannot think why anyone would think otherwise.

Your Thompson quote doesn't work...

[lairdb]

Originally Posted by eschwartz

Feel free to your opinion.

Agreed -- Golden rule: Kovid has the gold, Kovid makes the rules.

The calibre rationale is that people who have manually configured a conversion for a specific book, won't want to go back to the generic defaults.
[...]
Conversions follow the normal rules of conversions. I cannot think why anyone would think otherwise.

Your Thompson quote doesn't work...

Where I would disagree, or at least quibble, is that there's no UI indication that conversion options are sticky on a per-book basis. Yes, RTFM - but absent reading that particular paragraph, I would expect that the "normal rule of conversions" would be that my chosen preferences are followed.

[eschwartz]

And as I said, I shan't argue for one side or the other.
On the topic of "should settings be sticky".

So if you are disagreeing with me, it must be regarding my statement "Conversions follow the normal rules of conversions. I cannot think why anyone would think otherwise" which I said in order to justify email autoconversions following the rules of manual conversions (whatever those rules turn out to be).

...

Since you seem to have quoted me in a rather broken and confusing manner, I am not 100% sure what parts of my post you are commenting/disagreeing with/quibbling over/other.
So take that for whatever it is worth.

[lairdb]

I'm not disagreeing with your statements, as much as with whether the behaviour is evident or intuitive (which you seem to be saying). (Thus the Thompson joke.)

Given that conversion options are sticky per-book, yes, it's behaving as I would expect.

Nowhere in the application behaviour or UI is it suggested that conversion options are sticky per-book.

IMNSHO, that's not particularly user-friendly -- but my opinion is just that; Kovid makes the rules.

[BetterRed]

Originally Posted by lairdb

Nowhere in the application behaviour or UI is it suggested that conversion options are sticky per-book.

@lairdb - There's a Restore Defaults button on the Conversion dialogue. That suggested to me book specific conversion settings were used, and I was convinced by a peek into the database, where I saw the conversions options table with a unique key of book id and format.

Eventually I spotted this How options are set/saved for Conversion Ψ²

BR