Language Encoding issue in FBReader?

llhots · 02-11-2009, 07:37 PM

Hi everyone, I'm a bit of noob, so perhaps this question has already been answered... but I looked everywhere and didn't find the answer.
I am using the new openinkpot snapshot (downloaded last night) on my Hanlin Bebook. So far I think it is fantastic, and would like to congratulate all of you for the great work.
I was so pleased with it, that I flashed it on, so am not running the live version from an SD card.
I mostly read in English. I've downloaded some RSS feeds, converted to MOBI format with Calibre, and when I open them there are some random characters scattered throughout the text, such as boxes, the "TM" sign, a character box etc...
I suspect that the encoding is wrong, and I need to force it manually to English. Is this possible? I saw that someone had suggested modifying the /home/.FBReader/books.xml file, but I unfortunately don't know how to do this.
I tried reading the files as an EPUB file, but then the index doesn't work (ie. once I select where I want to navigate to, the page flashes but it remains stuck at the index.
Regardless, I would prefer using the MOBI format, so if anyone has any suggestions about how to solve the character issue I would be very appreciative.

THanks,

wallcraft · 02-11-2009, 08:44 PM

One thing you could try is opening the file in the Desktop FBReader. This will at least tell you what format and encoding FBReader thinks the file is using. You can also try other encodings (and fonts) to confirm that the problem can be fixed.

Note that there are two potential reasons for "boxes" in FBReader, the encoding could be wrongly selected or the font could be missing the needed symbols. If you are getting random bad characters (vs boxes), then it is almost certainly the encoding. Similarly, if the ePub version is working then it can't be the font and must be the encoding. Note the encoding isn't "English", it is probably windows-1252 or UTF-8.

If you can get SSH over USB working (possible from a Linux Desktop) then access to the books.xml is relatively easy, otherwise it isn't.

llhots · 02-11-2009, 09:42 PM

Thanks very much Wallcraft!
I appreciate your answer. I suspect you're right, because when I open the same file on in the EPUB format, it turns out fine (I would use EPUB, it's just that my index doesn't work).
I think I was able to confirm this because I went and downloaded FBReader. It turns out that the file is encoded in UTF-8, but when I open it the language is set to "Other" as opposed to English. I have a desktop running Ubuntu, so I will see if I can get access to the reader through USB... (I don't know what SSH is, but will look into it).
Will keep everyone posted should I have any luck. Thanks again.

wallcraft · 02-11-2009, 10:08 PM

Quote:

Originally Posted by llhots

I think I was able to confirm this because I went and downloaded FBReader. It turns out that the file is encoded in UTF-8, but when I open it the language is set to "Other" as opposed to English. I have a desktop running Ubuntu

The FBReader you downloaded is likely more up to date than the one in OI, and it probably correctly detected the UTF-8. Try switching it to windows-1252 explicitly. You will probably get the same strange characters as on your Hanlin. Since you are running Linux, you can look at ~/.FBReader/books.xml to see how to change the same file on the Hanlin.

SSH is the preferred method of accessing a shell (command line) remotely on most Linux systems, including the Hanlin under OI. See SSH access and What is SSH?.

llhots · 02-11-2009, 11:22 PM

Thanks again Wallcraft! You've really been helpful and I appreciate it. I'm going to read over the SSH access material over the weekend and will post my results in case it might be helpful for anyone else (us noobs).
I really hope it will work, because I feel like I'm so close.
Worst case, I'll simply use the EPUB files and scroll through the whole newspaper (given that the index doesn't work using my EPUB files in OpenInk with FBRearder-- always tough to have it all).
Thanks again for your help

llhots · 02-13-2009, 06:22 PM

I just can't quite figure it out...
Although I haven't yet tried the SSH access (I don't know much about this, so will try to learn how to do it in Ubuntu over the weekend), I figured it might be possible to change encoding manually in the files using Calibre.
I therefore changed the encoding on one file, to see if I could get FBReader to recognize it properly. I encoded the same file in a number of different formats by changing the source encoding.
I tried "US-ASCII", "windows-1252", "ISO-8859" and "UTF-8" as well as a few variants on these in case I was getting the syntax wrong (forgive me if my vocabulary is wrong... I'm not that familiar with this).
When I opened the files in FBReader on my Ubuntu desktop, the default conversion worked fine so long as I changed it to ASCII, so I know that it isn't fundementally a font issue.
The problem is that when I bring them to my ebook reader and openinkpot, I still get the similar strange symbols, despite trying all sorts of encoding in different files. In fact, I could tolerate it, but one of the symbols not represented is the apostrophe, which is variously replaced as the Euro symbol and a trademark symbol.
I downloaded some of the openinkpot fonts, just to confirm, but from the font description, I can see that the apostrophe is there, so clearly that isn't the issue.
So I guess I know it is an encoding issue.
My question is whether or not there is a default encoding in Openinkpot's FBReader. I suspect not, because otherwise one of the multiple encodings I tried should have worked no?
Which then leads me to believe that it is set to automatically detect the encoding, and keeps getting it wrong, no matter what the encoding. Is this possible?
I guess I really need to go learn how to modify the books.xls file...
If someone can confirm that there is no way to solve the problem through encoding source files, I'd appreciate it.

Thanks,

lunohod · 02-13-2009, 07:05 PM

send me a sample file, please.

llhots · 02-13-2009, 08:17 PM

Hi Lunohod, I sent it to an email I found on Openinkpot. I hope you get it. Thanks for your help!

lunohod · 02-16-2009, 05:54 AM

Those issues will be solved in the upcoming snapshot. I've packaged the new fbreader version and the autodetection seems to work better. If it fails you can change the language and encoding settings via gui.

llhots · 03-01-2009, 12:02 PM

I just wanted to let everyone know that I downloaded the new snapshot and everything is working perfectly!
Thanks Lunohod, and all developers of Openinkpot. The new release is excellent. I haven't found a single issue so far. The encoding works, the languages work, the fonts work.
Really, I can't thank you enough. Congratulations on some excellent work and best of luck.

02-11-2009, 07:37 PM	#1
llhots Junior Member Posts: 7 Karma: 10 Join Date: Feb 2009 Device: Hanlin (Bebook)	Language Encoding issue in FBReader? Hi everyone, I'm a bit of noob, so perhaps this question has already been answered... but I looked everywhere and didn't find the answer. I am using the new openinkpot snapshot (downloaded last night) on my Hanlin Bebook. So far I think it is fantastic, and would like to congratulate all of you for the great work. I was so pleased with it, that I flashed it on, so am not running the live version from an SD card. I mostly read in English. I've downloaded some RSS feeds, converted to MOBI format with Calibre, and when I open them there are some random characters scattered throughout the text, such as boxes, the "TM" sign, a character box etc... I suspect that the encoding is wrong, and I need to force it manually to English. Is this possible? I saw that someone had suggested modifying the /home/.FBReader/books.xml file, but I unfortunately don't know how to do this. I tried reading the files as an EPUB file, but then the index doesn't work (ie. once I select where I want to navigate to, the page flashes but it remains stuck at the index. Regardless, I would prefer using the MOBI format, so if anyone has any suggestions about how to solve the character issue I would be very appreciative. THanks,

02-13-2009, 06:22 PM	#6
llhots Junior Member Posts: 7 Karma: 10 Join Date: Feb 2009 Device: Hanlin (Bebook)	I'm stumped... I just can't quite figure it out... Although I haven't yet tried the SSH access (I don't know much about this, so will try to learn how to do it in Ubuntu over the weekend), I figured it might be possible to change encoding manually in the files using Calibre. I therefore changed the encoding on one file, to see if I could get FBReader to recognize it properly. I encoded the same file in a number of different formats by changing the source encoding. I tried "US-ASCII", "windows-1252", "ISO-8859" and "UTF-8" as well as a few variants on these in case I was getting the syntax wrong (forgive me if my vocabulary is wrong... I'm not that familiar with this). When I opened the files in FBReader on my Ubuntu desktop, the default conversion worked fine so long as I changed it to ASCII, so I know that it isn't fundementally a font issue. The problem is that when I bring them to my ebook reader and openinkpot, I still get the similar strange symbols, despite trying all sorts of encoding in different files. In fact, I could tolerate it, but one of the symbols not represented is the apostrophe, which is variously replaced as the Euro symbol and a trademark symbol. I downloaded some of the openinkpot fonts, just to confirm, but from the font description, I can see that the apostrophe is there, so clearly that isn't the issue. So I guess I know it is an encoding issue. My question is whether or not there is a default encoding in Openinkpot's FBReader. I suspect not, because otherwise one of the multiple encodings I tried should have worked no? Which then leads me to believe that it is set to automatically detect the encoding, and keeps getting it wrong, no matter what the encoding. Is this possible? I guess I really need to go learn how to modify the books.xls file... If someone can confirm that there is no way to solve the problem through encoding source files, I'd appreciate it. Thanks,

02-16-2009, 05:54 AM	#9
lunohod OpenInkpot developer Posts: 211 Karma: 5627 Join Date: Mar 2008 Location: Stuttgart, Germany Device: N516,V3,PRS505,iLiad,eSlick,E60,K3,K4NT,M90,PB912,KoboMini	Those issues will be solved in the upcoming snapshot. I've packaged the new fbreader version and the autodetection seems to work better. If it fails you can change the language and encoding settings via gui. Last edited by lunohod; 02-16-2009 at 06:02 AM.

03-01-2009, 12:02 PM	#10
llhots Junior Member Posts: 7 Karma: 10 Join Date: Feb 2009 Device: Hanlin (Bebook)	Working I just wanted to let everyone know that I downloaded the new snapshot and everything is working perfectly! Thanks Lunohod, and all developers of Openinkpot. The new release is excellent. I haven't found a single issue so far. The encoding works, the languages work, the fonts work. Really, I can't thank you enough. Congratulations on some excellent work and best of luck.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
pdb to epub (encoding issue)	lamzor	Calibre	2	05-13-2013 06:38 PM
Encoding	prusaks	Recipes	0	09-27-2010 06:25 AM
Language and Encoding Question	llhots	Calibre	1	02-11-2009 10:01 PM
FBReader fixes character encoding problem	jbenny	News	1	10-18-2007 10:50 PM
iLiad Enabling Korean encoding (EUC-KR) with FBReader (0.8.2b) for iLiad	wagnerian	iRex Developer's Corner	2	05-11-2007 10:03 PM

02-11-2009, 08:44 PM	#2
wallcraft reader Posts: 6,975 Karma: 5183568 Join Date: Mar 2006 Location: Mississippi, USA Device: Kindle 3, Kobo Glo HD	One thing you could try is opening the file in the Desktop FBReader. This will at least tell you what format and encoding FBReader thinks the file is using. You can also try other encodings (and fonts) to confirm that the problem can be fixed. Note that there are two potential reasons for "boxes" in FBReader, the encoding could be wrongly selected or the font could be missing the needed symbols. If you are getting random bad characters (vs boxes), then it is almost certainly the encoding. Similarly, if the ePub version is working then it can't be the font and must be the encoding. Note the encoding isn't "English", it is probably windows-1252 or UTF-8. If you can get SSH over USB working (possible from a Linux Desktop) then access to the books.xml is relatively easy, otherwise it isn't.

02-11-2009, 09:42 PM	#3
llhots Junior Member Posts: 7 Karma: 10 Join Date: Feb 2009 Device: Hanlin (Bebook)	Thanks very much Wallcraft! I appreciate your answer. I suspect you're right, because when I open the same file on in the EPUB format, it turns out fine (I would use EPUB, it's just that my index doesn't work). I think I was able to confirm this because I went and downloaded FBReader. It turns out that the file is encoded in UTF-8, but when I open it the language is set to "Other" as opposed to English. I have a desktop running Ubuntu, so I will see if I can get access to the reader through USB... (I don't know what SSH is, but will look into it). Will keep everyone posted should I have any luck. Thanks again.

02-11-2009, 11:22 PM	#5
llhots Junior Member Posts: 7 Karma: 10 Join Date: Feb 2009 Device: Hanlin (Bebook)	Thanks again Wallcraft! You've really been helpful and I appreciate it. I'm going to read over the SSH access material over the weekend and will post my results in case it might be helpful for anyone else (us noobs). I really hope it will work, because I feel like I'm so close. Worst case, I'll simply use the EPUB files and scroll through the whole newspaper (given that the index doesn't work using my EPUB files in OpenInk with FBRearder-- always tough to have it all). Thanks again for your help

02-13-2009, 07:05 PM	#7
lunohod OpenInkpot developer Posts: 211 Karma: 5627 Join Date: Mar 2008 Location: Stuttgart, Germany Device: N516,V3,PRS505,iLiad,eSlick,E60,K3,K4NT,M90,PB912,KoboMini	send me a sample file, please.

02-13-2009, 08:17 PM	#8
llhots Junior Member Posts: 7 Karma: 10 Join Date: Feb 2009 Device: Hanlin (Bebook)	Hi Lunohod, I sent it to an email I found on Openinkpot. I hope you get it. Thanks for your help!

Advert

Advert