Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Conversion

Notices

Reply
 
Thread Tools Search this Thread
Old 08-27-2011, 01:51 AM   #1
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,447
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
New MOBI output plugin, testing needed

Hi all,

calibre version 0.8.16, released today, has a new MOBI output plugin. This is based on a month long effort to further reverse engineer the parts of the MOBI format that were still opaque.

The major bug that the new plugin fixes is that periodicals (news/magazines) created with calibre now have fully working navigation on the Kindle 3. When you click on View articles and sections while reading an article, the Kindle now returns to the current article in the list of articles. See https://bugs.launchpad.net/calibre/+bug/748741 for details

In addition various small things were figured out that should make the MOBI files produced by calibre have a binary structure closer to those produced by Amazon's tools. Note that the "binary structure" refers to internals of the MOBI file, not the text formatting, which remains the same as before.

Since the new plugin is a complete rewrite, it may well have bugs, so I would appreciate it if a few of you turned it on and tested it over the next week or two, before I release it widely.

To enable the new plugin, go to Preferences->Tweaks, click the "Plugin Tweaks" button and type

Code:
new_mobi_writer = True
into the box and then click OK and Apply. This will enable the new MOBI output plugin.

Report any problems you encounter here.

Thanks.
kovidgoyal is offline   Reply With Quote
Old 08-27-2011, 02:39 AM   #2
Joseph R
Zealot
Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.Joseph R ought to be getting tired of karma fortunes by now.
 
Joseph R's Avatar
 
Posts: 136
Karma: 2120000
Join Date: Aug 2011
Location: Edmonton, Alberta
Device: Kindle Paperwhite, Nexus 7, Kobo Glo
I volunteer..

Thank you for your hard work!
Joseph R is offline   Reply With Quote
 
Enthusiast
Old 08-27-2011, 11:28 AM   #3
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,447
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You're welcome
kovidgoyal is offline   Reply With Quote
Old 08-27-2011, 11:51 AM   #4
user_none
Sigil & calibre developer
user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.
 
user_none's Avatar
 
Posts: 2,433
Karma: 950001
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
@Kovid,

Are you going to update the MOBI format documentation file in the format_docs folder with all you've learned? Having a better understanding of the structure of the MOBI format would benefit a lot of people. Not everyone is comfortable reading Python code to understand the format.
user_none is offline   Reply With Quote
Old 08-27-2011, 12:47 PM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,447
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Nope, I much prefer my documentation to be at source level. The new MOBI writer is heavily commented, but that's about what I'm willing to do documentation wise.

EDIT: calibre also includes a tool to decompile MOBI files into a nice ASCII representation, which should really help with understanding the MOBI format as well, calibre-debug --inspect-mobi filename.mobi (corresponds to the --extract-to argument to ebook-convert)

Last edited by kovidgoyal; 08-27-2011 at 01:09 PM.
kovidgoyal is offline   Reply With Quote
Old 08-27-2011, 02:33 PM   #6
Bratzzo
Addict
Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.
 
Bratzzo's Avatar
 
Posts: 342
Karma: 533558
Join Date: Feb 2010
Location: WOB / Germany
Device: Kindle
Hi,

Quote:
Originally Posted by kovidgoyal View Post
The major bug that the new plugin fixes is that periodicals (news/magazines) created with calibre now have fully working navigation on the Kindle 3. When you click on View articles and sections while reading an article, the Kindle now returns to the current article in the list of articles. See https://bugs.launchpad.net/calibre/+bug/748741 for details

this is great!
Until now, I not even knew that I would miss it!

Thank you!


But one thing, I don't like.
And I don't know if this is a bug or a feature.

At the beginning of every new section, it starts with a small section-TOC just above the first article of the section. But only if I use the pageturn buttons.
If I use the Kindle 5-way, it only shows the article.

Maybe the images explain what I mean.
The first image shows the first article of a new section when I use the 5-way to change between articles. It looks OK.
The second image shows the first article of a new section when I use the pageturn button. The red box shows the small TOC.

I don't like the small TOC above the new article. I would either prefer a seperate page at the beginning of a new section, or no seperate TOC at the beginning of a new section.



Bratzzo
Attached Thumbnails
Click image for larger version

Name:	without_TOC.gif
Views:	144
Size:	19.6 KB
ID:	75809   Click image for larger version

Name:	with_TOC.gif
Views:	150
Size:	27.4 KB
ID:	75810  
Bratzzo is offline   Reply With Quote
Old 08-27-2011, 02:46 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,447
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
That small TOC is necessary to make the Kindle's View Articles and Sections feature work, I agree it's annoying, but if you dont have it, the feature doesn;t work...don't ask me to explain why.
kovidgoyal is offline   Reply With Quote
Old 08-27-2011, 03:14 PM   #8
Bratzzo
Addict
Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.Bratzzo ought to be getting tired of karma fortunes by now.
 
Bratzzo's Avatar
 
Posts: 342
Karma: 533558
Join Date: Feb 2010
Location: WOB / Germany
Device: Kindle
Thanks for your answer.

But why it shows the TOC only if I turn the page, but doesn't shows the TOC if I change the article?

And now, I discovered this:

last page last article section 1 --> 5-way to the next article (first article section 2) --> article without TOC --> pageturn backwards --> same article with TOC --> pageturn backwards --> last page last article section 1.

This is a litte annoying. But thanks god, I seldom turn backwards.

While the benefit is much bigger than the "pain", I think, I have to accept this.



Bratzzo
Bratzzo is offline   Reply With Quote
Old 08-27-2011, 08:10 PM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,447
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
changing the article goes from article to article and skips content in between, its like the chapter to chapter jumps in a book.
kovidgoyal is offline   Reply With Quote
Old 09-02-2011, 01:41 PM   #10
bobb40
Member
bobb40 has a complete set of Star Wars action figures.bobb40 has a complete set of Star Wars action figures.bobb40 has a complete set of Star Wars action figures.
 
Posts: 24
Karma: 274
Join Date: Jul 2011
Device: Kindle3, Kindle4, KoboGlo
Quote:
Originally Posted by kovidgoyal View Post
To enable the new plugin, go to Preferences->Tweaks, click the "Plugin Tweaks" button and type

Code:
new_mobi_writer = True
into the box and then click OK and Apply. This will enable the new MOBI output plugin.
Does that enable it for CLI?
bobb40 is offline   Reply With Quote
Old 09-02-2011, 02:07 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,447
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
yes, it does.
kovidgoyal is offline   Reply With Quote
Old 09-04-2011, 02:06 PM   #12
bobb40
Member
bobb40 has a complete set of Star Wars action figures.bobb40 has a complete set of Star Wars action figures.bobb40 has a complete set of Star Wars action figures.
 
Posts: 24
Karma: 274
Join Date: Jul 2011
Device: Kindle3, Kindle4, KoboGlo
Quote:
Originally Posted by kovidgoyal View Post
Report any problems you encounter here.
Using ebook-convert from CLI, after enabling new o/p module, converting text to mobi, source is part html, part markdown.
Paragraphs beginning with:
<p style="text-indent: 0; margin-left: 3em; text-align: left;">
were placed oddly; ran with --debug-pipeline, and found the style in the
"processed" directory stylesheet.css but with text-align set to "justify"; edited the stylesheet, zipped directory, ran ebook-convert on zipfile, and paragraph placement is as expected. So the inline style text-align got lost somewhere along the way.
bobb40 is offline   Reply With Quote
Old 09-04-2011, 02:20 PM   #13
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,447
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
That has nothing to do with MOBI output, its a txt input issue. You should open a bug report and attach a txt file demonstrating the problem.
kovidgoyal is offline   Reply With Quote
Old 09-07-2011, 10:53 AM   #14
siebert
Developer
siebert has a complete set of Star Wars action figures.siebert has a complete set of Star Wars action figures.siebert has a complete set of Star Wars action figures.
 
Posts: 136
Karma: 280
Join Date: Nov 2010
Device: Kindle 3 (Keyboard) 3G / iPad 3 WiFi / Nexus 4 (Android)
Hi,

I've noticed that with the new output plugin calibre pretends to be kindlegen by setting the EXTH records 204-207.

Unless this is necessary for some features to work (which would surprise me), I'd lobby to either use a unique value for record 204 which would identify calibre and set records 205-207 according to the current calibre version or, as we can't predict which values for record 204 amazon will use in the future and the calibre version is already available in the contributor record, don't set the records 204-207 at all.

As the calibre output can't be 100% identical to the kindlegen, no matter how much effort we put into it, pretending that kindlegen created the mobi file might break tools which depend on the kindlegen version to be correct.

I've also seen that calibre has a kindlegen option which when enabled uses the kindlegen binary to create the mobi file instead of the calibre python code.

It looks like a debugging option and I couldn't find a way to enable it in the calibre gui, but as this option is the best of both worlds for some users, having the epub conversion done by calibre, which is far superior than the epub conversion from kindlegen, together with a mobi binary created by the official amazon tool kindlegen, it would be great if this would be a user selectable option on the mobi output option screen of the gui.

With that option the AZW Conversion Output Plugin (http://www.mobileread.com/forums/sho...80&postcount=1) should be obsolete, too.

Ciao,
Steffen
siebert is offline   Reply With Quote
Old 09-07-2011, 12:15 PM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,447
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
As you say, the fact that calibre created the MOBI file is documented in the creator record. So if some tool needed to differentiate between calibre generated and kindlegen generated files (highly unlikely since there are multiple versions of kindlegen as well as non kindlegen tools amazon uses to produce MOBI files) that tool can check for the creator EXTH record containing calibre. I have just about zero motivation to generate the half a dozen different types of MOBI files and check that everything still works on a Kindle without the 204-207 records. Incidentally, does such a tool actually exist?

--kindlegen is a debug option and as such is not suitable for inclusion in the GUI. I do not intend to maintain it/make it robust. Maintaining one set of mappings from epub to mobiml is quite enough work, thank you. Remember that running kindlegen does not just create the binary wrapper it also re-processes the markup. In most cases this is harmless, but I highly doubt it will be harmless in all cases. And then kindlegen is a moving target, one over which I have no control.

Last edited by kovidgoyal; 09-07-2011 at 12:33 PM.
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
AZW Conversion Output Plugin tylau0 Plugins 59 08-14-2011 10:36 PM
OS X testing needed kovidgoyal Calibre 55 02-06-2010 04:51 AM
Windows testing needed kovidgoyal Calibre 19 01-17-2010 02:35 PM
eReader file testing needed user_none Calibre 8 10-12-2009 02:28 AM


All times are GMT -4. The time now is 05:59 PM.


MobileRead.com is a privately owned, operated and funded community.