Libprs500 unicode supporting help!!!!!

[leebok21]

Dear Kovid Goyal,

First of all, I want to appreciate you to share excellent program, Libprs500 for Sony reader users. I am using it daily base for transferring News and other files. BTW, I'd like to ask you a favor for all asian language users.



505 does not contain fonts for showing Asian languages (Korean, Japanese and Chinese and so on). So to read Korean text (in my case), I used to make Korean-font-embeded PDF files and convert to LRF using PDFLRF tool. Which is a "OK", except taking chunk of time and effort. About 3 weeks ago, we all got a great news of unicode font supporting patch has been released without firmware flashing. Ever since, many Korean user started to rename fonts for prs-505, and it was pretty successful. Korean TXT and RTF file formats could be used in the reader using tt0011m_.ttf and tt0003m_.ttf, respectively (since korean fonts containing lots of character set, it usually sized about 500kb - 6MB or more). Especially RTF can retain all the styles that was set in the word processor. It looks awsome. That was big change, since we don't have to take longcut anymore, like making PDF and convert to LRF. By the way, downside of reading TXT and RTF directly in the reader make machine sluggish. Sometimes, it spent 10 minutes to formatting (performing layout) one small txt file. RTF converting, however, make small txt files more than 10 times bigger ('cause, font embeding), and reader doesn't like big file size. So, either way, we have pretty annoying limitations.

I was wondering if libprs500 can convert this unicode txt file into LRF format while embdeing appropriate fonts. The answer is NO. I got message below.

XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
UnicodeDecodeError: 'utf16' codec can't decode byte 0x00 in position 10: truncated data
Failed to perform job: Convert book:libtest
Detailed traceback:
Traceback (most recent call last):
File "parallel.py", line 95, in run_job
File "libprs500\ebooks\lrf\any\convert_from.pyo", line 166, in main
File "libprs500\ebooks\lrf\any\convert_from.pyo", line 143, in process_file
File "libprs500\ebooks\lrf\txt\convert_from.pyo", line 90, in process_file
File "libprs500\ebooks\lrf\html\convert_from.pyo", line 1725, in process_file
File "libprs500\ebooks\lrf\__init__.pyo", line 286, in Book
File "libprs500\ebooks\lrf\__init__.pyo", line 248, in find_custom_fonts
File "libprs500\ebooks\lrf\__init__.pyo", line 229, in find_family
File "ttfquery\describe.pyo", line 29, in shortName
File "encodings\utf_16_be.pyo", line 16, in decode
UnicodeDecodeError: 'utf16' codec can't decode byte 0x00 in position 10: truncated data
Log:
[INFO] convert_from.pyo:60: Converting text to HTML...
C:\Program Files\libprs500\library.zip\libprs500\ebooks\lrf\_ _init__.py:230: UnicodeWarning: Unicode unequal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

I have no idea about any kind of software develoment, but it looks like convert TXT to HTML first, and HTML2LRF take over after that. Unicode is definitely NOT supported, right? From Looks&Feel menu, I have chosen fonts that is implented in my reader for "embeded fonts", but it didn't work.

To read Korean text file, I had to change original encoding to UTF-8. Even though I set the encoding of test file as UTF-8, error description keep saying "utf16" somehow.

I don't know how hard it is, but can you make libprs500 as unicode compatible (maybe UTF-8?)? I am pretty sure that unicode enabled libprs500 will be able to convert korean (Japanes and Chinese) text correctly. We also want to get news feed from Korean newspaper company directly and carrying in PRS-505, which will be making a lot of people pleased.


Or, any other suggestions?

Lots of thanks in advance.

[leebok21]

I made it by adding a "UTF-8" in the source encoding line. This is easiest way to convert korean TXT to LRF directly.

Thanks Kovid,