View Single Post
Old 02-24-2014, 11:55 AM   #18
Doitsu
Grand Sorcerer
Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.
 
Doitsu's Avatar
 
Posts: 5,731
Karma: 24031401
Join Date: Dec 2010
Device: Kindle PW2
Quote:
Originally Posted by DaleDe View Post
Actually UTF-16 is preferred for some languages. UTF-8, as you point out, is perfect for English and Latin based languages but it is a variable length and will actually produce a larger file than UTF-16 for some languages which is why it is in the standard.
Chinese utf-16 files are on average only 30% smaller than the corresponding utf-8 files. However, English utf-16 files are on average twice the size of the corresponding utf-8 files.

I.e., the size advantage isn't that great, even for Chinese texts who benefited the most from the introduction of the utf-16 standard.
Doitsu is offline   Reply With Quote