HELP! Mobi conv. prob. in Kindle for PC--big issue

Hitch · 07-13-2010, 04:29 PM

Hi, Gang:

I'm having a moderately serious problem which I believe is actually being caused by Amazon's additional layer of DRM for books being sent to the Kindle for PC (KFPC), but is screwing me up nonetheless, because my clients are unhappy, not without reason.

The files that I've made--using Sigil to generate an epub, then using Calibre to convert to mobi--are not working on the Kindle for PC after they've passed through Amazon's DTP. Some Amazon Kindle "tech" told my client that it's my fault, that "They say the problem with [bookname] is that it wasn't created using their (Amazon's) creation and coding tools and that it's protected by DRM-1, which Kindle for PC and most new Kindles won't recognize, as opposed to DRM-2."

I naturally had a fit when I read this, as obviously, I don't add DRM. But nonetheless, it appears that files made with mobipocket aren't having this problem. Or freaking files made with Smashwords, god help me. I do not understand enough, I admit it, about the conversion via Calibre to grasp what the hell might be going wrong.

@Kovid--can you please help?

Hitch

AnemicOak · 07-13-2010, 04:50 PM

Not that it solves what you're asking, but what about using KindleGen to convert the ePub instead? I generally use Calibre when converting for my own use, but I've tried KindleGen on some epubs and the conversion's have looked just as good. I just drag the epub onto the KindleGen exe icon and in a few seconds have a mobi file (or you can of course run it from the command line).

Hitch · 07-13-2010, 05:58 PM

Quote:

Originally Posted by AnemicOak

Not that it solves what you're asking, but what about using KindleGen to convert the ePub instead? I generally use Calibre when converting for my own use, but I've tried KindleGen on some epubs and the conversion's have looked just as good. I just drag the epub onto the KindleGen exe icon and in a few seconds have a mobi file (or you can of course run it from the command line).

I can give it a try; I've never used KindleGen. I don't mean to be thick, but the destructions say you have to use an html file, not an epub? NB: the conversion "worked," sort of...

EDIT and ADD: Also got a runtime error in Kindle for PC when I opened the resulting mobi, which didn't have a table of contents and had two covers. :-( Crashed Kindle for PC.

Seriously, am I at the point where I have to open Sigil, copy everything from xhtml into an html editor, save it as html (can't wait to see how THAT goes), then run it through mobipocket? Or KindleGen? FFS!!!
Hitch

AnemicOak · 07-13-2010, 06:36 PM

Hmm, I don't know. I've only ever used it to convert ePubs and haven't ever had a problem. I know the download says it's for HTML, but the publishing guide mentions using it for ePubs.

Hitch · 07-13-2010, 06:41 PM

Quote:

Originally Posted by AnemicOak

Hmm, I don't know. I've only ever used it to convert ePubs and haven't ever had a problem. I know the download says it's for HTML, but the publishing guide mentions using it for ePubs.

@Anemic Oak: how did you create the TOC's for your books in epub? Did you create it in Sigil, or create via Calibre, or...? It mostly works, but I can't send out a book w/o a TOC, particularly for the authors who have "about the author" and "other books by this author" sections, their heads would explode. Maybe if I edit the doodah manually...????

Kovid, are you out there somewhere???

Hitch

Hitch · 07-13-2010, 07:50 PM

Further to this topic, as an experiment, I tried to convert an epub to rtf via Calibre and then open it with Word, to save as html...well, you guys get the drift. The rtf wouldn't open in Word, I got a message that the file was corrupted. {shrug}. So my options right now appear to be to literally cut-and-paste out of sigil into an html editor, strip anything sigil-related (oh, joy), then save as html and drop it onto Kindlegen, with which I hope I'll get a TOC and a file that can be read on Kindle for PC, as Calibre seems to not work for that application with their DRM.

Does ANYONE at all have any other thoughts on this?

Hitch

AnemicOak · 07-13-2010, 10:59 PM

Quote:

Originally Posted by Hitch

@Anemic Oak: how did you create the TOC's for your books in epub? Did you create it in Sigil, or create via Calibre, or...? It mostly works, but I can't send out a book w/o a TOC, particularly for the authors who have "about the author" and "other books by this author" sections, their heads would explode. Maybe if I edit the doodah manually...????

I've only used KG on maybe a dozen or so ePub's. Some were commercial ones I bought, some were created from HTML witch was obtained from PDF's and put together with Sigil (and sometimes run through Calibre). I don't recall problems with any of them. I wasn't however running them through DTP.

I just quick ran a book that I needed to convert for my Mom and it worked great. Would sending you the toc.ncx from that file so you could compare it to what you've got possibly help?

I know you're using Sigil to create the epub, but what's the initial source? Is it Word saved as HTML? saved as HTML filtered? Do you run the HTML through any tidying software? Just wondering if the problem is Sigil, Calibre or possibly the initial source.

Quote:

Originally Posted by Hitch

Further to this topic, as an experiment, I tried to convert an epub to rtf via Calibre and then open it with Word, to save as html...well, you guys get the drift. The rtf wouldn't open in Word, I got a message that the file was corrupted. {shrug}. So my options right now appear to be to literally cut-and-paste out of sigil into an html editor, strip anything sigil-related (oh, joy), then save as html and drop it onto Kindlegen, with which I hope I'll get a TOC and a file that can be read on Kindle for PC, as Calibre seems to not work for that application with their DRM.

What about unzipping the epub and merging that HTML which you can then manipulate? I really don't know, just throwing thoughts out and hoping something will work. You probably know more about the topic than I do overall.

Have you gone to the DTP forums and seen if anyone there has any thoughts?

I don't know if more folks here would see this if it was in the Mobi format forum or not.

Hitch · 07-14-2010, 02:24 AM

Quote:

Originally Posted by AnemicOak

I've only used KG on maybe a dozen or so ePub's. Some were commercial ones I bought, some were created from HTML witch was obtained from PDF's and put together with Sigil (and sometimes run through Calibre). I don't recall problems with any of them. I wasn't however running them through DTP.

Well, right this second, it isn't yet DTP; it's that the TOC doesn't show up in the KindleGen output. Even Amazon isn't quite sure what's going on, despite what their techs say.

Quote:

I just quick ran a book that I needed to convert for my Mom and it worked great. Would sending you the toc.ncx from that file so you could compare it to what you've got possibly help?

With a TOC? Which you created how? Sure, sending me the toc.ncx couldn't hurt. PM me.

Quote:

I know you're using Sigil to create the epub, but what's the initial source? Is it Word saved as HTML? saved as HTML filtered? Do you run the HTML through any tidying software? Just wondering if the problem is Sigil, Calibre or possibly the initial source.

Scanned physical book to Word; word in BD to strip the formatting (and preserve the italics and any bold, although the latter is rare); saved as .doc; re-opened in Word (to get out of BD and its horrible CSS) and set base font; saved as filtered html-->Sigil-->epub-->Calibre-->mobi. Long ass way to go to get to Amazon and suddenly have my files stop working, allegedly it being my problem.

Quote:

What about unzipping the epub and merging that HTML which you can then manipulate? I really don't know, just throwing thoughts out and hoping something will work. You probably know more about the topic than I do overall.

Not dramatically faster than just copying the xhtml files into NoteTab and stripping out the xhmtl tags there. <shrug>. Still means I end up running two processes in parallel; make one book (epub) in Sigil; take the codeview text, COPY it for crap's sake, PASTE it into a text editor, save as html, then into KindleGen or Mobipocket. Not to mention, I lose all the nice css formatting I use in the epubs which I currently sneak into the Kindle versions (and, yes, for the most part, it works. The dropcaps leave a lot to be desired, but you still get a text effect--not quite a dropcap, but an enlarged font).

Quote:

Have you gone to the DTP forums and seen if anyone there has any thoughts?

Oh, yeah, that's been swell. One guy there wants to convert me (pun intended) to his method of making mobis from Word; the other wants to help me because "[he] uses a program called Calibre." Nobody reads anymore, ironically enough. Of course, none of them use epubs, so they could not care less about the fact that I need to make an epub, AND the fact that epubs from Sigil give me options that I cannot get via html-->Mobipocket (like using the title attribute for a heading tag to create a TOC item).

Quote:

I don't know if more folks here would see this if it was in the Mobi format forum or not.

Some folks are readers, some are publishers, some are both. I'm both. I came to this forum because the step from html is from sigil to epub to mobi, so I thought that Kovid might have some ideas, but, here we are.

Hitch

AnemicOak · 07-14-2010, 09:06 AM

Quote:

Originally Posted by Hitch

With a TOC? Which you created how? Sure, sending me the toc.ncx couldn't hurt. PM me.

This isn't a book I created. It's an ePub my Mother bought from Penguin, and yes it converts perfectly and the TOC works. Maybe something in it's toc.ncx will jump out when compared to the one giving you trouble.

I did notice that this particular book also has an inline TOC so perhaps KindleGen is using that instead of the ncx. I don't know if your book has an inline TOC or not.

I've attached the ncx below...

kovidgoyal · 07-14-2010, 09:46 AM

Sorry, I have never dealt with Amazon's DTP so I have no idea what could be causing it.

Just to be clear, you're saying that the files you get back from the DTP service work on the kindle but not on K4PC? I can have a look at it if you create a simple file that when converted with calibre results in a not working file but when converted with kindlegen results in a working file.

Hitch · 07-14-2010, 05:06 PM

Quote:

Originally Posted by kovidgoyal

Sorry, I have never dealt with Amazon's DTP so I have no idea what could be causing it.

Just to be clear, you're saying that the files you get back from the DTP service work on the kindle but not on K4PC? I can have a look at it if you create a simple file that when converted with calibre results in a not working file but when converted with kindlegen results in a working file.

Hi, Kovid:

Yes. When I create a .mobi from a Sigil-created epub via Calibre, the books dl'ed from Amazon (after being uploaded through the DTP, during which they are DRM'ed), work on the physical Kindle but NOT F4PC. FYI, it's important here to note, if you do not already know this, that Amazon attaches a second layer of DRM--a session ID, I am given to understand--to the K4PC books that is NOT present in the Kindle books. I don't know that this is the problem, but naturally, I'm suspicious. The mobi file pre-DTP works fine in both KNDL and K4PC.

I don't know how I would--I don't know how I could know if a file I made with KindleGen would work whereas a file I made with Calibre wouldn't, without putting up a new book as an experiment (not sure how excited my author would be about that) that I created with KindleGen (which for some reason ate my TOC {sigh}), running it through the DTP, and then "buying" it and dl'ing it to K4PC. I could send you the dl'ed post-DTP file that was sent for K4PC, but that will have those two pesky layers of encryption.

I'm genuinely happy to cooperate in any way, but I truly have no idea how to test a post-DTP KindleGen-created file for K4PC considering Amazon's meager resources. I can't put up a book for $0.00 as a crash-test dummy. I'm not sure I can put one up at $0.01...but tell you what, maybe I'll take some Public Domain works, create an epub, make a Kindle .mobi from Calibre, then one from KindleGen, run 'em through the DTP (man, what a PITA), and see what I get. if you, Kovid, or anyone else, has any alternative suggestions, man, I'd be happy to hear them.

Actually, Kovid, I could send you the two different .mobi files, though, "pre" DTP--the one I created with KG and the one I created with Calibre; the "content" is identical, only the coding would be different. Does that sound viable? I know it's "pre-DTP," but maybe you would spot something?

Hitch

kovidgoyal · 07-14-2010, 06:06 PM

Quote:

Originally Posted by Hitch

Actually, Kovid, I could send you the two different .mobi files, though, "pre" DTP--the one I created with KG and the one I created with Calibre; the "content" is identical, only the coding would be different. Does that sound viable? I know it's "pre-DTP," but maybe you would spot something?

Hitch

That would only be useful if I knew that one of the mobi files is going to survive DTP and the other not. That way I can compare and contrast, without wasting too much of my time. As it is MOBI is a completely opaque binary format that relies on lots and lots of random bits in the header of the file to control how it works. Given that ideally, I'd need a handful of such pairs, but I'm willing to try with just one.

Hitch · 07-18-2010, 06:18 PM

Quote:

Originally Posted by kovidgoyal

That would only be useful if I knew that one of the mobi files is going to survive DTP and the other not. That way I can compare and contrast, without wasting too much of my time. As it is MOBI is a completely opaque binary format that relies on lots and lots of random bits in the header of the file to control how it works. Given that ideally, I'd need a handful of such pairs, but I'm willing to try with just one.

Hi, Kovid:

Well, here's the latest from Amazon, which is UTTERLY useless:

Quote:

I'm following up with you regarding your inquiry on Kindle content.

I've checked with our technical team and confirmed that the calibre conversion has caused this problem. Please re-create your content using our conversion tools (MobiPocket or KindleGen) and upload it to DTP. Let us know if you have any difficulties, so I can look into it right away.

So, here's my next question: as I am not an expert in html, is there any QUICK way for me to convert the 20 xhtml files I have in an epub into html, so I can use KindleGen or MobiPocket? I tried drag-n-dropping a Sigil-generated epub onto KindleGen, as per another poster's suggestion, but I got ZIP for a TOC, not a thing, so unfortunately, THAT seems to be out. My client's ready to have my hide.

Help, thanks in advance,

Hitch

kovidgoyal · 07-18-2010, 06:22 PM

convert from the calibre produced mobi to something else and use the debug option. in the debug directory you'll find a single html file in the input subdirectory. And the people at Amazon are morons.

wallcraft · 07-18-2010, 06:37 PM

kindlegen does support epub input. It may not like all legal ePubs, but it should produce a 100% compatible MOBI for Amazon (essentially by definition).

If the Sigil ePub does not convert cleanly to MOBI using mobigen, what I would try next is:

a) Convert from ePub to MOBI using Calibre, then
b) convert from MOBI to ePub using Calibre, then
c) convert from ePub to MOBI using kindlegen.

This is a very round about way to get a Kindle MOBI, but Amazon can hardly blame Calibre if kindlegen is what creates the MOBI.

07-13-2010, 04:29 PM	#1
Hitch Bookmaker & Cat Slave Posts: 11,503 Karma: 158448243 Join Date: Apr 2010 Location: Phoenix, AZ Device: K2, iPad, KFire, PPW, Voyage, NookColor. 2 Droid, Oasis, Boox Note2	HELP! Mobi conv. prob. in Kindle for PC--big issue Hi, Gang: I'm having a moderately serious problem which I believe is actually being caused by Amazon's additional layer of DRM for books being sent to the Kindle for PC (KFPC), but is screwing me up nonetheless, because my clients are unhappy, not without reason. The files that I've made--using Sigil to generate an epub, then using Calibre to convert to mobi--are not working on the Kindle for PC after they've passed through Amazon's DTP. Some Amazon Kindle "tech" told my client that it's my fault, that "They say the problem with [bookname] is that it wasn't created using their (Amazon's) creation and coding tools and that it's protected by DRM-1, which Kindle for PC and most new Kindles won't recognize, as opposed to DRM-2." I naturally had a fit when I read this, as obviously, I don't add DRM. But nonetheless, it appears that files made with mobipocket aren't having this problem. Or freaking files made with Smashwords, god help me. I do not understand enough, I admit it, about the conversion via Calibre to grasp what the hell might be going wrong. @Kovid--can you please help? Hitch

07-13-2010, 04:50 PM	#2
AnemicOak Bookaholic Posts: 14,391 Karma: 54969924 Join Date: Oct 2007 Location: Minnesota Device: iPad Mini 4, AuraHD, iPhone XR +	Not that it solves what you're asking, but what about using KindleGen to convert the ePub instead? I generally use Calibre when converting for my own use, but I've tried KindleGen on some epubs and the conversion's have looked just as good. I just drag the epub onto the KindleGen exe icon and in a few seconds have a mobi file (or you can of course run it from the command line). Last edited by AnemicOak; 07-13-2010 at 04:59 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
pdf to mobi conversion issue	dkritso109	Kindle Formats	9	10-06-2010 05:03 PM
Big RSS feeds and mobi formating	Zorz	Calibre	14	01-27-2010 02:35 AM
Another reason 'sharing' may not be as big an issue...	ficbot	News	19	12-20-2009 07:21 PM
Strange issue with News and Kindle/mobi	kennyc	Calibre	10	12-14-2009 01:58 PM
Is DRM in ebooks a big issue?	tech_au	News	85	01-23-2009 11:28 AM

07-13-2010, 06:36 PM	#4
AnemicOak Bookaholic Posts: 14,391 Karma: 54969924 Join Date: Oct 2007 Location: Minnesota Device: iPad Mini 4, AuraHD, iPhone XR +	Hmm, I don't know. I've only ever used it to convert ePubs and haven't ever had a problem. I know the download says it's for HTML, but the publishing guide mentions using it for ePubs.

07-13-2010, 07:50 PM	#6
Hitch Bookmaker & Cat Slave Posts: 11,503 Karma: 158448243 Join Date: Apr 2010 Location: Phoenix, AZ Device: K2, iPad, KFire, PPW, Voyage, NookColor. 2 Droid, Oasis, Boox Note2	Further to this topic, as an experiment, I tried to convert an epub to rtf via Calibre and then open it with Word, to save as html...well, you guys get the drift. The rtf wouldn't open in Word, I got a message that the file was corrupted. {shrug}. So my options right now appear to be to literally cut-and-paste out of sigil into an html editor, strip anything sigil-related (oh, joy), then save as html and drop it onto Kindlegen, with which I hope I'll get a TOC and a file that can be read on Kindle for PC, as Calibre seems to not work for that application with their DRM. Does ANYONE at all have any other thoughts on this? Hitch

07-14-2010, 09:46 AM	#10
kovidgoyal creator of calibre Posts: 45,144 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Sorry, I have never dealt with Amazon's DTP so I have no idea what could be causing it. Just to be clear, you're saying that the files you get back from the DTP service work on the kindle but not on K4PC? I can have a look at it if you create a simple file that when converted with calibre results in a not working file but when converted with kindlegen results in a working file.

07-18-2010, 06:22 PM	#14
kovidgoyal creator of calibre Posts: 45,144 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	convert from the calibre produced mobi to something else and use the debug option. in the debug directory you'll find a single html file in the input subdirectory. And the people at Amazon are morons.

07-18-2010, 06:37 PM	#15
wallcraft reader Posts: 6,977 Karma: 5183568 Join Date: Mar 2006 Location: Mississippi, USA Device: Kindle 3, Kobo Glo HD	kindlegen does support epub input. It may not like all legal ePubs, but it should produce a 100% compatible MOBI for Amazon (essentially by definition). If the Sigil ePub does not convert cleanly to MOBI using mobigen, what I would try next is: a) Convert from ePub to MOBI using Calibre, then b) convert from MOBI to ePub using Calibre, then c) convert from ePub to MOBI using kindlegen. This is a very round about way to get a Kindle MOBI, but Amazon can hardly blame Calibre if kindlegen is what creates the MOBI.