Developing Ebooks For Kindle In Bangla

[Nabodita]

Some time back, I started this thread about creating epub books using Bangla (Bengali) fonts. Since then I've (more or less) managed to create epubs which display correctly on devices which allow the user to enable publisher formatting.

Now, I need to create books in Bangla for the Kindle. Unfortunately, Kindle does not support Bangla natively. On the several different occasions I have spoken with Amazon, I got the stock reply about how they don't currently support Bangla but are continually working to expand their reach. In other words, I'm on my own.

As far as I can see, there are a few possible solutions:

1. Amazon could provide me language support. Now we all know how improbable that is!

2. Newer Kindles support font embedding. I could take advantage of this feature to create books in a manner similar to the one I used for epub. More on this later.

3. I could install the font on my Kindle, create the ebook without font embedding and see how that goes.

4. I could distribute the epub itself along with a market / custom app for reading the epub but in this case, I can't take advantage of Amazon's ebook distribution network.

Now that I've gotten the background information out of the way, here's how far I've gotten.

I started by trying to embed the Bangla font using Calibre and generating an azw3 file. Transferred to my first gen Kindle Fire and opened it and well... let's just say it didn't display correctly. I then created the file using the latest KindleGen and faced the same problem. Then I went and bought a Kindle Fire HD and the same file displayed perfectly. (Yeah!). I'll be getting a new Kindle in a couple of days (eInk - I don't know about 2/3/4 yet but I'm assuming it will be the latest model since I ordered this one from Amazon) and will be able to test the file then but I don't see why it shouldn't work.

So I have a couple of problems. First, my old Kindle Fire doesn't render the embedded font correctly even though it supports azw3. Further, AFAIK, the older eInk models don't support azw3. So, even if I can somehow get the rendering issue sorted out - which I doubt - people with older Kindles will not be able read the book.

Do you think it would be possible to write some code (an android developer would do this, not me) so that a Kindle user could download and install the Bangla font onto their device? No hacks or jailbreaks for the end user, just a file to download from the website which will install itself. And how do you think Amazon will react to this? (I will talk to them about this, of course).

Of course, before I proceed along this line, I need to jailbreak my old Kindle Fire and install the font on my device (thereby bypassing the need for font embedding) and test. It may be that the font won't render correctly even when installed on the device.

Although there are extensive wikis for jailbreaks and font hacks, I couldn't find something for my old Kindle Fire. Any help would be appreciated.

A very long post. Just wanted to get my thoughts down and take opinions and suggestions. So please, opinions, suggestions, questions and any help would be sincerely appreciated. If this post is in the wrong section, could one of the mods please move it to the appropriate place?

Thanks
Nabodita

[knc1]

One of the reasons you didn't find any information about the Kindle Fire (of any age) here, is this is the Kindle E-Ink (grayscale) only forum.

Not only do the color model Kindles have different hardware, they also have a different operating system.

So at a minimum - you need to split your project into two projects.
One for each product line.

[NiLuJe]

Is Bengali an RTL script, or a script heavily relying on combined glyphs/diacriticals marks?

FWIW, KF8 support goes as far back as the K3 (Kindle Keyboard) on the e-Ink line, so that's not too bad nowadays.

As knc1 said, can't help you at all on the Fire (Android) front, but I doubt anything custom would be possible without a root/jailbreak/hack. Which, quite frankly, will also mostly be the case on the eInk line if you forgo the KF8+publisher's font way.

[Nabodita]

Originally Posted by knc1

One of the reasons you didn't find any information about the Kindle Fire (of any age) here, is this is the Kindle E-Ink (grayscale) only forum.

Not only do the color model Kindles have different hardware, they also have a different operating system.

So at a minimum - you need to split your project into two projects.
One for each product line.

So two projects it is. Are the E-Ink Kindles Android based?

I've been googling a bit about how to install fonts on Kindle Fire. I'll do some testing and report back on my progress.

I'm perfectly happy to restrict this discussion to the E-Ink devices, since I have to get that sorted as well!

Originally Posted by NiLuJe

Is Bengali an RTL script, or a script heavily relying on combined glyphs/diacriticals marks?

Left to Right language and yes, complex glyphs.

Originally Posted by NiLuJe

FWIW, KF8 support goes as far back as the K3 (Kindle Keyboard) on the e-Ink line, so that's not too bad nowadays.

That's good to know. Under the circumstances, do you think a publishing house can release Kindle ebooks which are not compatible with older Kindle models? If I can't figure out a way to make this work, I might need to distribute ebooks with this caveat (assuming Amazon will allow that).

Originally Posted by NiLuJe

I doubt anything custom would be possible without a root/jailbreak/hack. Which, quite frankly, will also mostly be the case on the eInk line if you forgo the KF8+publisher's font way.

Can you think of any other way it might be possible to create Bangla ebooks? I prefer to embed the fonts but unfortunately that won't be supported by ALL Kindles.

In this post, Doitsu said that "all current eInk Kindles (K3 and higher) should support them, because the Bengali Unicode range is covered by the system fallback font."
Would that be true of earlier Kindles as well? I have no means of testing this.



FYI: Bangla is the 7th most spoken language in the world. Amazon, get a clue!

[knc1]

So two projects it is. Are the E-Ink Kindles Android based?

No.
They are Amazon-Lab126 based (a custom Linux O.S.).

[Nabodita]

Originally Posted by knc1

No.
They are Amazon-Lab126 based (a custom Linux O.S.).

Ok. Do you have any idea how I could go about testing on the older (pre K3) Kindles? I don't have access to this hardware.

Another thought: How many people still actually use Kindles that old? I suppose I also need to find out which Kindle generations support KF8.

[NiLuJe]

@Nabodita: You can forget about pre-K3 devices. They don't support KF8 files, and, IIRC, the few tests I did with Doitsu a while back with custom fonts & custom freetype libs lead to mostly useless results in the case of RTL and/or complex scripts. Since Bengali appears to be both, I can pretty safely say that, nope, not gonna happen .

I was going to say "see if Doitsu looked at Bengali before", and you did, so the good news appears to be that code2000 (the fallback font w/ the widest unicode support on eInk Kindles) handles it. With KF8 handling RTL & complex script shaping (mostly? I dimly remember some weird issues w/ some complex scripts in some circumstances? Take that with a grain of salt, I might be mixing up stuff), that should mean almost zero hard work on the eInk front (provided you forget about the K1 & the K2, which you can't do anything about anyway, IMHO).

[knc1]

Originally Posted by Nabodita

FYI: Bangla is the 7th most spoken language in the world. Amazon, get a clue!

Interesting, but Amazon is a for-profit company.
And (e-) books are for reading.

So what percentage of the population over 15 can read and write?
57 %
http://en.wikipedia.org/wiki/List_of..._literacy_rate

Using the GNI(PPP) measure (Gross National Income per capita at purchasing power parity) as an indicator of the ranking of potential e-books and e-book readers, where does this population stand?
147th.
http://en.wikipedia.org/wiki/List_of...%29_per_capita

Using the HDI measure (Human Development Index - which includes both education and income measures) as the same ranking indicator, where does this population stand?
145th.
http://en.wikipedia.org/wiki/Bangladesh

The language is spoken in other countries of that area of the world, but stats other than "per country" are difficult to research.
The above figures are based on that of Bangladesh, the single largest population using the language.

A rough guess at how many people have enough education and income (say, 10%) to buy and read an e-book (with a purchased e-book reader) puts the users of this language somewhere around the population of Los Angeles, USA.
http://en.wikipedia.org/wiki/Bangladesh

A rather small market to target with support for a language.
Amazon may indeed have a very good clue.

Going by my own reasoning above, Amazon is more likely to support Navajo before they support Bangla.

- - - -
Who? Me? Take Amazon's side in something? Geesh, I must be drunk and/or asleep.

[Nabodita]

@knc1: Incredible! I'm mildly insulted but I'm not sure why? Perhaps, regional pride? I'm a Bengali (Indian, not Bangladeshi), can read, write and speak 4 languages and a very, very avid reader (although to be honest, I only read English books!) But I have to say,

I'm just curious, though... is it so hard to provide language support? I find that simply embedding a font file in an epub and enabling publisher formatting allows a device to display Bengali correctly... Please don't flame me.. I know that I don't know, well, a lot!

Bengali appears to be supported on so many devices and platforms today. And the depth and width of Bengali literature is astounding. Besides, Amazon is retailing its Kindles in India... it would make sense for them to support at least the major languages. (I don't know about Bangladesh). Of course, by this logic, Kindle would have to support the language of any country in which Kindles are sold and that's plain unrealistic!

Will post back in a little while with updates.

[Nabodita]

@NiLuJe

Well, I rechecked Doitsu's test file and there's good news and there's bad news. The good news is that it's rendering perfectly on his K3. The bad news is that his test file is not rendering correctly on my old Kindle Fire (2011). Neither is my test file.

Here's the thing. I installed Aldiko on my Kindle Fire and tested an epub with the font embedded and it rendered perfectly. I took the same epub, ran it through KindleGen (converted without errors) and then tested the mobi and it didn't render correctly. What could be wrong? Obviously, the device is capable of rendering the font correctly otherwise the epub would have been off too... I feel like

BTW, the same mobi renders correctly on my new Kindle Fire HD.

[NiLuJe]

@Nabodita: I can't help you on the Fire front, I never had one of those, and have approximately zero idea of what mess they're using as the renderer on the different models, sorry .

The only thing that comes to mind is checking how Kindle For PC/Kindle for Mac & the Kindle Previewer handle this?

[knc1]

Originally Posted by Nabodita

@knc1: Incredible! I'm mildly insulted but I'm not sure why? Perhaps, regional pride? I'm a Bengali (Indian, not Bangladeshi), can read, write and speak 4 languages and a very, very avid reader (although to be honest, I only read English books!) But I have to say,

No insult intended.

Bengali is common in what became the Eastern part of Pakistan when it was created after the war. (WWII that is)

So you are in a much different demographic than those of what is now Bangladeshi.
It is almost a second (third?) national language.

A good friend of mine (a native of Pakistan) claims you can always recognize a well to do person in Bangladeshi - that is the fellow with rubber tires on his donkey cart.
(again, no offense intended).

It just isn't a real hot-bed market for e-book readers and e-books.
At least not enough to attract a profit oriented company like Amazon.

Hmm...
Would Delhi or Karachi have as many (or more) readers of Bengali than all of Bangladeshi?

nope - will not take any bets on that one, either one or the other or both might.

[nasser]

Hello knc1..
back in this forum after a long break..
was doing some real reading on my PW.. instead of endlessly hacking..
I live in Bangalore..
Quite a few Bengalis here.. especially in the software shops..
I meet them mostly in the sweet shops.. i like bengali sweets.. all dripping with sugar syrup..
Don't know about their e-book reading habits though..
Anyway.. হ্যালো to Nabodita.. Ate any sondesh or rosho golla recently..?

[Nabodita]

@knc1: Oh pooh! I know there's no insult intended... I'm laughing at myself. I never thought of myself as particularly patriotic!

The former East Pakistan is now Bangladesh. In India, there is a huge Bengali speaking community in West Bengal. During partition (pre-independence) Bengali Hindus moved to West Bengal, India, Bengali Muslims moved/stayed in Bangladesh. I won't do a history lesson now. Suffice it to say, rough estimates put the population of the state of West Bengal at around 91 million while that of Bangladesh around 152 million. Of course, this is not taking into account the millions of Bengalis scattered throughout rest of India. The reason why Bengali is associated with Bangladesh is because it is a national language while in India, its a regional language.

As it so happens, there is a very large Bengali speaking community in Delhi. Mostly people who have settled there during partition. Other than that, Bengalis (like other communities in India) are settled all over India.

When I started this project, I really didn't think about GNI(PPP) or HDI (in fact, Economics was never my forte!). I calculated that if 1% of the entire Bengali speaking community in the world (181 million according Encyclopedia Brittanica) could be tapped thats 1.8 million people. Not bad. Wait. That doesn't sound right. If there are 152 million people in Bangladesh and 91 million people in West Bengal.... Oh I give up.

I know this is way off topic but I'm having too much fun!

[Nabodita]

Originally Posted by nasser

Ate any sondesh or rosho golla recently..?

Me, I'm a Roshogolla person, preferably made of Gur (Jaggery)!

Nasser, you are the man. Please help!