converting epub to mobi for kindle 3 weird hyphens

[monkeygirl351]

Well I am getting a kindle 3 in a few days. I have tons of amazon ebooks, but also several epubs of my own (without drm). Anyways, I wanted to convert these epubs to mobi using calibre. I have the latest version and all my books are managed in there. Well I have noticed that when converting, I am getting these weird square symbols next to hyphens in the mobi document. These aren't there in the origional epub. I have researched and found out that these are called soft hyphens and kindle can't read these and therefore puts the square in there. How can I get rid of these? The mobi file looks great except for these annoying squares. Some books have more than others. I guess I can leave them if i have to, but i am a perfectionist and like my books to be perfect. I suppose I can read the epubs on my ipad but I really want to try them out on my kindle. Any suggestions?

[wallcraft]

This might be due to an encoding problem. For example the ePub might be declared as utf-8 when it is actually cp1252 (or the other way round). See FAQ: How do I convert my file containing non-English characters, or smart quotes.

[monkeygirl351]

Thanks for the idea! I was playing around with the different settings in calibre and found that if I check the box for the transliterate unicode in Look and Feel, the boxes go away. It makes the book much more readable so I am excited. I am just finishing tagging and organizing my books in calibre because my kindle is scheduled to deliver tomorrow. YAY! Also, does calibre directly deliver books to the kindle? That would be cool, I might get a book for tips and tricks to use it to the full potential!

[NVash]

Do those hyphens look like '�' this by any chance? I just converted a few books myself and I keep getting these. I looked over the User Guide that came with Calibre but Im not even going to pretend I know what Im doing here. I dont even know how to get to the 'Look And Feel' menu theyre talking about to see if the OP had the same problem Im having now. It talks about encodings my HTML fonts are in, I have no idea what it means. It isnt an HTML at all, its just text. Im all confused.

[wallcraft]

Originally Posted by NVash

Do those hyphens look like '�' this by any chance? I just converted a few books myself and I keep getting these. I looked over the User Guide that came with Calibre but Im not even going to pretend I know what Im doing here. I dont even know how to get to the 'Look And Feel' menu theyre talking about to see if the OP had the same problem Im having now. It talks about encodings my HTML fonts are in, I have no idea what it means. It isnt an HTML at all, its just text. Im all confused.

When you click on "convert individually" there is a "look and feel" icon on the left. It has two options that might help. One is "input character encoding" and the other is "Transliterate unicode characters to ASCII". I suggested trying cp1252 in the encoding box, and what worked for monkeygirl351 was checking the transliterate box.

It can be hard to convert TXT to something else because plain text has no metadata (describing the properties of the document). The encoding is the convention used to map from the file's binary bits to characters. For English characters this does not matter because most encodings use the same convention for them, but they differ in how they map other binary patterns to characters. What you are seeing is binary patterns that have been mapped to the wrong character.

[NVash]

Thanks for explaining that, I just tried it out and it worked for me too!

[NVash]

Everything was working great but now I just started getting this error whenever I try to open some new books.

'Could Not Load Book'
'java.lang.IllegalStateException: Error in accessing state 1872 at index 468; the file may be locked'

Some books convert, some give me this error. And if I convert it without the 'cp1252' I get the weird letters again. Anyone know whats going on?

[abatsukh]

Originally Posted by wallcraft

When you click on "convert individually" there is a "look and feel" icon on the left. It has two options that might help. One is "input character encoding" and the other is "Transliterate unicode characters to ASCII". I suggested trying cp1252 in the encoding box, and what worked for monkeygirl351 was checking the transliterate box.

It can be hard to convert TXT to something else because plain text has no metadata (describing the properties of the document). The encoding is the convention used to map from the file's binary bits to characters. For English characters this does not matter because most encodings use the same convention for them, but they differ in how they map other binary patterns to characters. What you are seeing is binary patterns that have been mapped to the wrong character.

I tried both ways and still alot of question marks appear on the book. is it because the book is in spanish? please help, i need to read this book to improve my spanish asap. i was so excited to find the book for free on epub and now can't read it.

[wallcraft]

Originally Posted by abatsukh

I tried both ways and still alot of question marks appear on the book. is it because the book is in spanish?

Spanish is included in cp1252, so if you have tried cp1252 and utf-8 I'm not sure what else to suggest. Other possibilities include macintosh for Mac OS Roman and IBM's cp850. I don't know if Calibre will recognize these code sets though. See Western Latin character sets (computing). It probably is worth trying macintosh at least.

[abatsukh]

Originally Posted by wallcraft

Spanish is included in cp1252, so if you have tried cp1252 and utf-8 I'm not sure what else to suggest. Other possibilities include macintosh for Mac OS Roman and IBM's cp850. I don't know if Calibre will recognize these code sets though. See Western Latin character sets (computing). It probably is worth trying macintosh at least.

it doesn't seem to work. i also put the epub file on my ipod touch and those question marks still appear instead of certain letters. if it was limited to only one letter or caracter i could read but there are too many and it is hard to read.

[NVash]

Im getting messages that the book may be locked and sometimes it even seems to skip over whole paragraphs if not the whole chapter. I dont know whats going on with it. That 'cp1252' trick worked originally but now when I use it I have problems and when I dont use it I have those odd question marks.

[user_none]

Originally Posted by NVash

Everything was working great but now I just started getting this error whenever I try to open some new books.

'Could Not Load Book'
'java.lang.IllegalStateException: Error in accessing state 1872 at index 468; the file may be locked'

Calibre is not a java application. I have no idea where or how you're getting this error.

[NVash]

I get that error when I try to open it in Stanza. Usually I convert the book then look it over in Stanza to make sure everything is okay before I do anything else with it.

[jeep123]

Originally Posted by wallcraft

When you click on "convert individually" there is a "look and feel" icon on the left. It has two options that might help. One is "input character encoding" and the other is "Transliterate unicode characters to ASCII". I suggested trying cp1252 in the encoding box, and what worked for monkeygirl351 was checking the transliterate box.

It can be hard to convert TXT to something else because plain text has no metadata (describing the properties of the document). The encoding is the convention used to map from the file's binary bits to characters. For English characters this does not matter because most encodings use the same convention for them, but they differ in how they map other binary patterns to characters. What you are seeing is binary patterns that have been mapped to the wrong character.

I know this is very old but I wanted to add that I used this technique to fix a couple fussy ebooks to mobi. They would read fine using view on calibre but in the kindle I got little question marks in boxes for many of the dash and quotation characters.

Thanks!

[cybmole]

Originally Posted by abatsukh

I tried both ways and still alot of question marks appear on the book. is it because the book is in spanish? please help, i need to read this book to improve my spanish asap. i was so excited to find the book for free on epub and now can't read it.

try convert to .rtf & view it in word. that may clean things up somewhat. you may also be able to repair using word's find / replace.

if the rtf looks good, then convert that to epub

but you may have a poor quality source file - already messed up- in which case there's not a lot you can do, except look for a better version