[Kindle Touch] Localization

[JustAMan]

ixtab,
I tried your latest tool... still crashes.
Here's the trace:

[justaman@localhost kindle-touch-l10n]$ java -jar tool/kt-l10n.jar dist -f -k ~/kindletool/KindleTool/kindletool -s src/5.0.0/ -t update_5.0.0-my-LOCALE_TYPE.bin ru_RU -g tool/KTFrameWorkLocale/glibc-locales/ 
Exception in thread "main" java.lang.RuntimeException: java.io.IOException: Bad file descriptor
   at ixtab.ktlocale.DistributionTask.run(Unknown Source)
   at ixtab.ktlocale.LocaleTool.main(Unknown Source)
Caused by: java.io.IOException: Bad file descriptor
   at gnu.java.nio.channels.FileChannelImpl.write(libgcj.so.12)
   at java.io.FileOutputStream.write(libgcj.so.12)
   at java.io.BufferedOutputStream.flush(libgcj.so.12)
   at java.io.FilterOutputStream.close(libgcj.so.12)
   at java.util.zip.GZIPOutputStream.close(libgcj.so.12)
   at java.io.FilterOutputStream.close(libgcj.so.12)
   at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:155)
   at ixtab.ktlocale.util.TGZCreator.createTGZFile(Unknown Source)
   at ixtab.ktlocale.DistributionTask.createKindleInstallFiles(Unknown Source)
   at ixtab.ktlocale.DistributionTask.createKindlePackages(Unknown Source)
   at ixtab.ktlocale.DistributionTask.createDistribution(Unknown Source)
   at ixtab.ktlocale.DistributionTask.run(Unknown Source)
   ...1 more

eureka,
I think I'll give your JS tool a try tomorrow... want to have some sleep once in a while

[eureka]

Originally Posted by ixtab

About the naming: IMO your second proposal (src/5.0.3/{framework,locale,pillow,waf}) is best. Since you're the "master of slug-naming tools" , could you update the repo and transifex accordingly?

Done.

[ixtab]

@eureka: Thanks!

@JustAMan: oh, I see. And now I also see what you meant by "GNU java". Don't use gcj, it's horrendously buggy.

If you want to use an open-source Java implementation, use openjdk instead

Otherwise, you're of course free to debug the tool to see what initially causes the problem, and to change it accordingly to make it work with gcj.

[eureka]

@ixtab, so, I see that Pillow and WAF resources are being translated on Transifex. Nice. Let me know if you'll find js_resources's README incomplete or obscure or not helpful.

[ixtab]

@eureka:

Yeah, I'm at it right now... I noticed everything's setup on transifex and tool-wise, and I'm actually looking into the integration of it into the dist chain.

My observation is: it's almost working.

I.e., it seems to be working correctly when compiling the en_US locale, but not for others.

Here's what I get when compiling the de locale:

$ python tool/js_resources/ compile --source-dir ../kindle-touch-l10n-tx-full/src/5.0.3/pillow/ --dest-dir /tmp/loc-js/ --source-locale en_US --locale de
Traceback (most recent call last):
  File "/usr/lib/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/home/cl/kindle-touch/localization/kindle-touch-l10n/tool/js_resources/__main__.py", line 132, in <module>
    args.cmd.execute(args)
  File "tool/js_resources/lib/cli_commands.py", line 157, in execute
    resources = properties.read(compiled_file, base=base_file)
  File "tool/js_resources/lib/format/properties.py", line 71, in read
    with codecs.open(metadata_file, encoding='utf8') as md:
  File "/usr/lib/python2.6/codecs.py", line 870, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 2] No such file or directory: '../kindle-touch-l10n-tx-full/src/5.0.3/pillow/de/strings/wifi_wizard_dialog_strings.metadata.json'

The solution is trivial: metadata should not be looked for in the translated locale directory ("de"), but in the source locale directory ("en_US"). As I pointed out multiple times, I'm a complete legasthenic when it comes to python, so... could you please fix this?

Update:
P.S. The README is fine, it's just that it still has the old commands in it (if I'm not mistaken). For compile, I understand that the shortcut for "--source-locale" is now "-b", not "-m". Not a huge issue, as I'm using the long flags for now anyway, but probably just a small glitch that should be fixed.

[eureka]

Originally Posted by ixtab

The solution is trivial: metadata should not be looked for in the translated locale directory ("de"), but in the source locale directory ("en_US"). As I pointed out multiple times, I'm a complete legasthenic when it comes to python, so... could you please fix this?

Update:
P.S. The README is fine, it's just that it still has the old commands in it (if I'm not mistaken). For compile, I understand that the shortcut for "--source-locale" is now "-b", not "-m". Not a huge issue, as I'm using the long flags for now anyway, but probably just a small glitch that should be fixed.

My bad. Fixed and README updated.

UPD Ooops, no, it wasn't fixed in the strict sense, but rather stupidly avoided. But now it should be of no problem if en_US is used as source locale. I'll fix it for real soon.

[JustAMan]

ixtab,
So when you're planning to release de_DE JS locale part? I think I may have to include that into my tool actions, e.g., report & blacklist ones.

[ixtab]

@JustAMan: The german translation is, in fact, almost finished for these.

I'm currently coding the logic for including pillow/waf bundles into translations. I have already looked at all all the translatable strings (and translated most of them, diba is currently proof-reading and fixing the german translation).

In short, no blacklisting is required for these resources, because they *only* contain strings that were actually meant to be localized. (As in: Java ResourceBundles are the exception, not the rule. They contain a mix of behavioral and localizable resources. In contrast, pillow/waf/blanket resources only contain things relating to translation).

I intend to finish a first version of the tool tonight, so maybe we can have bundles that could be usable for end users by.... well, say, alpha version tonight, beta version tomorrow, final version in 3 days or so.

[diba]

ixtab, I have gone through the lastest files around 18:00 hrs, send a PN concerning a missing / in google.de/search and changed two or three wordings. Could not go through a final one, was blocked by you. But everything else is looking very good.

[JustAMan]

Okay, so I think I'll look into doing the same "combine/expand" action for JS, wait a bit for Transifex ru_RU people for translate it, then proof-read and try on my device

[diba]

done for all files, only thing I am not shure is the "Article mode", where is this to be found on the KT? Made "Artikel-Modus" assuming this is what it should be. Fixed the missing / as well now.

[eureka]

Originally Posted by JustAMan

Okay, so I think I'll look into doing the same "combine/expand" action for JS, wait a bit for Transifex ru_RU people for translate it, then proof-read and try on my device

Nobody from ru_RU people knows about WAF subproject (it was created 2 days ago). And there is no ru_RU team at the Pillow subproject.

So I'm expecting no ru_RU translations for JavaScript resources. You should announce these subprojects to Russian translators (I can do it, but have no desire at the moment). Or create a team by yourself and start translating.

Originally Posted by diba

done for all files, only thing I am not shure is the "Article mode", where is this to be found on the KT? Made "Artikel-Modus" assuming this is what it should be. Fixed the missing / as well now.

As far as I remember, it's a "Readability mode" of browser. This menu item could be found when Menu is invoked while browsing.

[diba]

I have sent the actual Sueddeutsche Zeitung, a german newspaper, via Calibre news to my KT. Here we have english leftovers as well. I am very sure having them read in the german loc, but they are not translated or at another place as well. Is "Sections" what should be "Rubriken" and "Articles" which should be Artikel. But not everywhere ...
Some expressions too long for the buttons, will have a look in later.


UPDATE:
Reviewed all files I identified having something to do with periodicals. There everything is ok, seems there are more files for the periodicals like for the browser. Either they should come to pillows or WAF, here I am out. This is something for ixtab and eureka.

[JustAMan]

eureka,
Hmm, makes sense. I think I'll try translating those when I get time then

[eureka]

@ixtab, please, don't include Pillow and WAF resources to your [automatically built] snapshots yet!!

(However, localization bundles built for testing purposes with manual checking of translated Pillow/WAF resources before building are OK.)

Pillow and WAF strings are used in HTML. They are possible source of XSS and I've overlooked this fact. I must implement some solution to filter these strings and leave only allowed HTML tags (so far, I've found two such tags in original JS files: <br> and <p>).

Also I've missed WAF's browser/readability/messages.js in extraction process. There is <a> tag in this file with embedded JavaScript as onClick handler. This embedded JavaScript probably should go into metadata...