MobileRead Forums - View Single Post

slantybard · 08-20-2009, 01:24 AM

OK. I wasn't trying to make you more frustrated. I am not sure where the problem is since I don't have physical access to your machine/setup/etc. But my main guess is still on something wrong with the way the encoding is being processed.

Basically, when an electronic file of letters is made, there is usually a tag that goes with the file so that other programs can open up the file and display the characters correctly. For some reason, there are different ways of mapping the characters in those files. For most characters, this mapping doesn't matter, ie, an "a" or a "t" can me mapped in many encodings, but still will show up properly if opened by a program using a different decoding method.

The problem usually comes in punctuation. You know those "smart" quotes that curl nicely before and after a quotation? Originally, computers where never designed to show those properly, so all quotes ended up straight like this: ". Then, later, word processors mapped out a smart quote to display nicely, Unfortunately, when some other programs open up a file with smart quotes, they don't or can't understand the mapping/encoding for the smart quote and guess as to how to display that punctuation. The result is often garbled and looks horrible.

Calibre has an algorithm that tries to guess and fit the best encoding for each file converted, but the problem is that many files are mislabeled by the program that made the file, or actually use a mix of different font character encodings within the same file due to very poor publishing practises. Still, in the end, this simply leads to reader frustration.

So what to do to fix this. Well, you can try different encoding guesses, but if you don't know the file or it is mixed this can be frustrating. However, I don't mind looking at your file and trying to convert it using my calibre setup. This could rule out a calibre specific issue or a file issue. To do this though, you need to get me a copy. You could send me a copy of the original file to mcdiarmid at gmail.com (replace at and the spaces with @) and I can take a look at the file and see if I can help that way.

08-20-2009, 01:24 AM	#4
slantybard my parent's oops... Posts: 493 Karma: 1477572 Join Date: Feb 2009 Device: Vx->Handera->Clie-> Axim->505->650->KPW/Aura ->L2->iOS/CBW	OK. I wasn't trying to make you more frustrated. I am not sure where the problem is since I don't have physical access to your machine/setup/etc. But my main guess is still on something wrong with the way the encoding is being processed. Basically, when an electronic file of letters is made, there is usually a tag that goes with the file so that other programs can open up the file and display the characters correctly. For some reason, there are different ways of mapping the characters in those files. For most characters, this mapping doesn't matter, ie, an "a" or a "t" can me mapped in many encodings, but still will show up properly if opened by a program using a different decoding method. The problem usually comes in punctuation. You know those "smart" quotes that curl nicely before and after a quotation? Originally, computers where never designed to show those properly, so all quotes ended up straight like this: ". Then, later, word processors mapped out a smart quote to display nicely, Unfortunately, when some other programs open up a file with smart quotes, they don't or can't understand the mapping/encoding for the smart quote and guess as to how to display that punctuation. The result is often garbled and looks horrible. Calibre has an algorithm that tries to guess and fit the best encoding for each file converted, but the problem is that many files are mislabeled by the program that made the file, or actually use a mix of different font character encodings within the same file due to very poor publishing practises. Still, in the end, this simply leads to reader frustration. So what to do to fix this. Well, you can try different encoding guesses, but if you don't know the file or it is mixed this can be frustrating. However, I don't mind looking at your file and trying to convert it using my calibre setup. This could rule out a calibre specific issue or a file issue. To do this though, you need to get me a copy. You could send me a copy of the original file to mcdiarmid at gmail.com (replace at and the spaces with @) and I can take a look at the file and see if I can help that way.