Kobo firmware still sucks

[Quoth]

Originally Posted by Sarmat89

That's not stupidity. EPUB format is too complex to be parsed and rendered by an embedded device, so it must be preprocessed into a less taxing form, like predictable-structure kepub or lightweight binary KFX.

I think you don't understand epub or kepub, or the sort of HW in readers. Even 25 years ago a cpu less powerful than in any ereader since 2005 could render web pages (HTML) including graphics. The Kepub adds extras to epub. Both are a zip archive with essentially the ingredients of a web page as separate files for index (NCX), each image, the cover, subset of HTML for the content broken into many files (ideally) and a css file. There may also be font files.

There would be very little difference in speed or RAM usage for epub2, epub3, kepub, the mobi-KF8 or ordinary web pages. The mobi 6 or PRC probably needs less RAM and cpu, especially if there are no images.
The slowest bit is the eink screen rendering, especially if there is 256 shade grey images. The true-colour RGB simply adds RAM overhead per image.

It's recommended that epubs/kepubs of any kind have no more than about 280K per part. The images are separate anyway and ought not to be larger dimensions than about 5" x 6" at 300 dpi (which is lunatic big) and jpeg. Even a 3" x 4" RGB JPEG is ten times more RAM than a typical "part" (< 0.3M), as that is 3 M uncompressed. The jpeg file has to be uncompressed, converted to 256 shades grey, then processed for the eink display (most only do 16 shades = Black, White and 14 x greys).


The original part files should split from the source at paragraph, heading, chapter or page boundary. Some books don't have chapter or page breaks in the source. Then the text is formatted according to the css file and possibly embedded fonts used. The part file may be many screen pages, the pagination is done according to styles, font faces, text sizes, margins etc. It's more work than what a web browser does. Actually older kindles used to process web pages like this.

So the differences of epub and kepub are some mysterious idea to suit Kobo, nothing to do with cpu / RAM load of parsing.

KFX (if's it's true about being smaller) would be MORE CPU work than any epub/kepub/mobi-KF8 of the same content. It would have to be unpacked into css, XML/HTML, images, fonts and index to be rendered for the screen.

I've not even addressed links, footnotes or javascript. Even ancient mobi/prc from Mobi-pocket creator mentions javascript support. The old PRC also could have an extra tag in HTML source that meant "You can't turn page past here". That meant subsequent parts could only be reached via internal links or the TOC/Index. I'd the idea of using this to build a "plot your own text adventure" book. I never did find out what javascript was supported. The original prc/mobi also supported some sort of database. I still might do a simple adventure ebook, using just links. I have written (in 1992!) a source text to build a PC graphic adventure. I found the barrier was getting artwork done. So I might do it as a text only adventure. Using the ebook formats is easier to build, more portable and easier to distribute than an Android version (basically Java apps in all but name).


As an aside, Codex means a book, i.e. instead of a scroll over 2000 years ago they hit on the idea of cutting up the material into pages, writing on both sides and binding. The way most software works on screen since the 1960s is a backward step. I'd love a web browser that worked like ereader page rendering, that you could view the huge scrollable web "pages" (should be called Web Scrolls?) as pages, using tap, swipe, volume up/down, PgUp/PgDn, or < > keys etc to flip the pages like an ereader app does. Maybe there is a plugin. I must look!

[DNSB]

Originally Posted by FrustratedReader

KFX (if's it's true about being smaller) would be MORE CPU work than any epub/kepub/mobi-KF8 of the same content. It would have to be unpacked into css, XML/HTML, images, fonts and index to be rendered for the screen.

KFX doesn't get split into css, etc. unless you are unpacking it on your computer and even there, quite a bit of information is lost from the original input to the KFX generator. The way I see it, KFX is more like an old bytecode compiler's output than the compressed archives that epub/azw3 use.

[Quoth]

Originally Posted by DNSB

KFX doesn't get split into css, etc. unless you are unpacking it on your computer and even there, quite a bit of information s lost from the original input to the KFX generator.

Something in the Kindle ereader or a kindle app has to read the KFX and then render the book the same appearance as azw/KF8. So while there isn't an archive with human readable content, it can't be losing anything or the book wouldn't look the same. I've compared the visual appearance of "direct by WiFi from Amazon KFX" on a PW3, with the "Download to PC, Transfer via USB for <device target>" which at last use was still and AZW and it looks identical. So obviously the KFX has two aims at least, stronger DRM and dramatically reduce traffic cost for Amazon if someone has the 3G model of a Kindle. They'd not care about broadband + Wifi, or people using WiFi points with a mobile dongle, only their own account traffic. Obviously the KFX has not obvious archive text files, however it must have the same structure of content, styles, images and index.

[DNSB]

Originally Posted by FrustratedReader

Something in the Kindle ereader or a kindle app has to read the KFX and then render the book the same appearance as azw/KF8. So while there isn't an archive with human readable content, it can't be losing anything or the book wouldn't look the same. I've compared the visual appearance of "direct by WiFi from Amazon KFX" on a PW3, with the "Download to PC, Transfer via USB for <device target>" which at last use was still and AZW and it looks identical. So obviously the KFX has two aims at least, stronger DRM and dramatically reduce traffic cost for Amazon if someone has the 3G model of a Kindle. They'd not care about broadband + Wifi, or people using WiFi points with a mobile dongle, only their own account traffic. Obviously the KFX has not obvious archive text files, however it must have the same structure of content, styles, images and index.

Perhaps you missed the bytecode compiler reference? Expecting a KFX to have the same structure as an epub/azw3 is like expecting a Pascal p-code file to have the same structure as the Pascal source code file.

[Quoth]

Originally Posted by DNSB

Perhaps you missed the bytecode compiler reference? Expecting a KFX to have the same structure as an epub/azw3 is like expecting a Pascal p-code file to have the same structure as the Pascal source code file.

No I did not. Since I have actually WRITTEN interpreters and and compiler I'm familiar with the concepts. However it's NOT the same idea at all as p-code or bytecode as that is a program executed on a virtual machine. The KFX is a compression of DATA, it's mostly content. Text & Images. Even the styling information originating in CSS isn't instructions in the the bytecode / pcode sense. The KFX is really entirely specifying data like a Word 95 doc file, it's not instructions in the sense of VB, or Java program.

Obviously you can't open a KFX and see the same thing as in an AZW, it's encoded. However ultimately there is only storage space saving. The data has to be interpreted to produce the same results. It's originated as css, HTML, index (all no doubt in an intermediate stage encoded to XML, but I don't know). The images will be separate blocks of data. Then no doubt some sort of tokens for the non-image data and encryption for DRM.
It's absolutely not a program, just a different representation of the data.
Like bytecode/p-code to Java, J++, C#, VB or UCSD-Pascal, you can get back a human readable source, but it will have lost all non-essential to the page content and rendering information and be hard to read.
At "run time" the Kindle or Kindle app must be converting KFX data to the equivalent of text, styles, index and separate blocks of images. Nothing else makes any sense. It's ENCODING, not byte code or p-code in the sense of a program. It makes it VERY much more compact than general purpose zip archiving can and thus any encryption (DRM) is more effective.

I totally agree there in no concept of "seeing" NCX, CSS, HTML and JPEGs in a KFX, nevertheless it's encoded and thus might have a higher CPU load than AZW/KF8, epub, kepub as the aim is lower transmission cost and more security.

The original point was that kepub certainly isn't any significant reduction in reader cpu requirement, neither is KFX. Probably the reverse as they are newer formats to aid distribution.

[geek1011]

Overall, I feel the Kobos have more pros than cons. I would always choose a Kobo over basically any other brand.

Originally Posted by John F

Did you read the title of my first post? The firmware works very well for how I use it. It is the best, for how I use it. It doesn't suck, and it isn't perfect.

I agree with this completely. Also, I prefer Kobo hardware over Kindle as well (in general).

Originally Posted by rashkae

Kobo, on the other hand, routinely introduces new bugs, and often times, they are there for what feels like a year and over 6 firmware releases before they are fixed. (I could cite examples, but I would only be venting my own pet peeves.)

I also wish there was a bit more structure to the releases, as there are sometimes hours and sometimes months between releases. Also, I still can't figure out for the life of me what that '-s' version was all about. Also, the thing which happens every now and then with them releasing a buggy version, then a new one shortly after (just after I finish the patches) bothers me a bit.

Originally Posted by rashkae

1. I'll start with the issue that started this thread even. Adjustment of text selection with the 'handles.' Once text is selected, the selection can not be adjusted unless you know the trick to moving the handles. (drag once to 'unlock' them, then once the pop up menu is gone, drag again to actually move.) This was a regression added in an update at least 3 versions ago.

Once I got used to that, I actually liked it better, as I'm not constantly trying to see under my finger when I start dragging. It also helps because the touchscreen is a bit offset for me (that's my number one bug), so I can properly estimate the offset I need.

2. New regression affecting Epub files introduced with 4.15. In some books, if there are two TOC entries pointing to the the same internal file, (ex: Heading and subheading,), at some points in the book, putting the kobo to sleep or going back to home will loose your current page. Bookmarks and highlights are also impossible to create in these locations.

That is a bit of a issue, but I am not going to complain about that. Keeping track of position is one of the most complicated things to do with EPUBs. The standard seems to be CFIs, but every implementation is buggy (including my own attempts at them).

Originally Posted by Lucas Malor

3. the virtual keyboard is highly unresponsive. it takes seconds to display the touched character

I think that's more of a hardware issue (but I'm not 100% certain). It feels instant on my Kobo Mini which uses the infrared touchscreen, but I agree with the lag on my Aura Edition 2.

when I have the bad idea to read a pdf, I usually turn it 90 degrees and I zoom it a little, so I can read them better. The problem is that, when I reach the end of the page and I touch to change page, the next page is displayed at the same level of the previous page, ie the bottom...

I'm not entirely sure what you mean by that (I don't read too many PDFs), but if you mean that you want the pan/zoom position preserved between pages, there's a patch for that (it was missing for a while, but I redid it a month or two ago).

5. sometimes, if I connect the Kobo to the pc and I edit a text file, like the nightmode ini, when I safely disconnect the device, it look like is completely reset. No books, nothing. I have to restart to restore the normal situation, after some heart attack and some profanity.

There are many possible causes for this. Does your Kobo ever disconnect randomly (I have that issue intermittently)?

Originally Posted by Sarmat89

That's not stupidity. EPUB format is too complex to be parsed and rendered by an embedded device, so it must be preprocessed into a less taxing form, like predictable-structure kepub or lightweight binary KFX.

I disagree that the EPUB format is too complex to be parsed and rendered by an embedded device, but agree it is complex in general. The part about the kepubs does not make it any less taxing, but it makes many things simpler to implement. I'm not sure whether I'd call KFX lightweight or simple though. Structured, maybe, but not simple.

Originally Posted by DNSB

HUH???? Kepub is epub format with added spans making the kepub structure, if anything, more complex than epub structure. So all these years I thought I was using RMSDK or ACCESS on my Kobo ereaders to parse and render epub2 and epub3 files never happened?

Technically, you're correct about the spans making it more complex, but it really makes things simpler in every way which matters. If all EPUBs had those spans, developing tools for working with EPUBs would be way simpler. Those spans mean that there is a consistent way to track the position of bits of text in books without resorting to the other common methods: CFIs (which are extremely difficult to parse correctly), scroll percentage (which is not consistent between readers, and is lossy), word/character number (which doesn't work with books mainly composing of images, and what about hidden stuff), and display page (which is not consistent between font sizes, screen sizes, and so on). With those spans, all you need to do is find the first and last one visible for the range you need. Also, since in general, the spans don't contain other elements, it is easier to get a specific piece of the document without any ambiguity.

Originally Posted by FrustratedReader

So the differences of epub and kepub are some mysterious idea to suit Kobo, nothing to do with cpu / RAM load of parsing.

It's just about simplicity for development and consistency between devices.

Originally Posted by FrustratedReader

No I did not. Since I have actually WRITTEN interpreters and and compiler I'm familiar with the concepts. However it's NOT the same idea at all as p-code or bytecode as that is a program executed on a virtual machine. The KFX is a compression of DATA, it's mostly content. Text & Images. Even the styling information originating in CSS isn't instructions in the the bytecode / pcode sense. The KFX is really entirely specifying data like a Word 95 doc file, it's not instructions in the sense of VB, or Java program.

...

Like bytecode/p-code to Java, J++, C#, VB or UCSD-Pascal, you can get back a human readable source, but it will have lost all non-essential to the page content and rendering information and be hard to read.

Yep. That's correct.

[davidfor]

Originally Posted by geek1011

I also wish there was a bit more structure to the releases, as there are sometimes hours and sometimes months between releases. Also, I still can't figure out for the life of me what that '-s' version was all about. Also, the thing which happens every now and then with them releasing a buggy version, then a new one shortly after (just after I finish the patches) bothers me a bit.

You would prefer them to leave the buggy version out there? That usually happens when Kobo consider the bug bad enough to really cause a problem. And they will usually stop rolling out the bad version first. And occasionally, they have put an update out a early when a problem was discovered a bit later.

There are also some other impacts. I think one of the recent releases was for some other product release or change, possibly "Kobo Plus" in the countries that have it. I think they had planned to include the Parental Controls with that, but couldn't finish it in time. So they hid it, and had another release when they were happy with it.

Once I got used to that, I actually liked it better, as I'm not constantly trying to see under my finger when I start dragging. It also helps because the touchscreen is a bit offset for me (that's my number one bug), so I can properly estimate the offset I need.

Overall, I think the selection has improved recently. And using the handles as well. The initial "grab" of the handles takes an extra step, but, it seems to be more accurate once you get going.

[geek1011]

Originally Posted by davidfor

You would prefer them to leave the buggy version out there?

Sometimes, yes . Many of the bugs haven't affected me too much, and it's a bit boring to update the same patches twice in a day. I wish they would catch the bugs before the rollout though. Or maybe, I should add a delay to my firmware notifications for myself (I currently have my script checking every 5 minutes)!

Overall, I think the selection has improved recently. And using the handles as well. The initial "grab" of the handles takes an extra step, but, it seems to be more accurate once you get going.

Yep. Although, one idea I just had is to have arrows beside the selection to expand it that direction until the next non-alphabetic character. This would make it easier to precisely select things near the edge of the screen.

[lumpynose]

Originally Posted by FrustratedReader

Like bytecode/p-code to Java, J++, C#, VB or UCSD-Pascal, ...

I can't help but wonder how old you are since you know about UCSD-Pascal, where I first learned programming, on a Terak which was a PDP-11 on a chip. Brings back fond memories of 8 inch floppies.

[DNSB]

Originally Posted by lumpynose

I can't help but wonder how old you are since you know about UCSD-Pascal, where I first learned programming, on a Terak which was a PDP-11 on a chip. Brings back fond memories of 8 inch floppies.

Makes me wonder if anyone else around here ever used a WD-900 Pascal MicroEngine chipset based computer. A neat idea that used a Western Digital microprocessor that executed USCD Pascal p-code as it's native language. One company I did some work for purchased 3 of WD-90s (computer based around that chipset) and we had fun (for certain values of fun ) getting them to work.

That same chipset with different microcode chips, sold as the MCP-1600, was the basis for PDP's LSI-11 computer.

Ahhh... the good old days of microcomputing.

[NiLuJe]

@geek1011: See this PR for an example of that kind of behavior in KOReader.

(It could be polished to apply to the initial selection process, instead of only the edit of an existing highlight, and to re-position the popup so it doesn't intrude on the actually highlighted content. But, yeah, the concept works pretty well, and definitely makes sense.).

[PeterT]

Originally Posted by lumpynose

I can't help but wonder how old you are since you know about UCSD-Pascal, where I first learned programming, on a Terak which was a PDP-11 on a chip. Brings back fond memories of 8 inch floppies.

LSI 11/03 with OMSI Pascal (after 4 years of APL (both IBM and then IP/SHARP) and 1 year of HP Basic (9830A))

Sent from my moto g(7) power using Tapatalk

[rashkae]

Originally Posted by NiLuJe

@geek1011: See this PR for an example of that kind of behavior in KOReader.

[geek1011]

Originally Posted by NiLuJe

@geek1011: See this PR for an example of that kind of behavior in KOReader.

(It could be polished to apply to the initial selection process, instead of only the edit of an existing highlight, and to re-position the popup so it doesn't intrude on the actually highlighted content. But, yeah, the concept works pretty well, and definitely makes sense.).

. Taking it a step further, I wonder if it would be usable as swipe gestures in the highlighted area too. Then, it wouldn't clutter the toolbar, and you'd still have more room to move stuff around in. I might try some stuff out with JS and the Kobo browser when I get a chance.

[Lucas Malor]

Originally Posted by geek1011

There are many possible causes for this. Does your Kobo ever disconnect randomly (I have that issue intermittently)?

Nope.

Originally Posted by NiLuJe

@geek1011: See this PR for an example of that kind of behavior in KOReader.

I think it's definitively a good idea. It should be added to Android too.

---

About KFX, for what I know it's not HTML+CSS anymore. It's simply JSON:

https://wiki.mobileread.com/wiki/KFX

I think this is an absurdity. HTML and CSS are mature languages for displaying the content of a book. Epub 3 has more than enough to be a better competitor of KFX:

1. It's based on HTML5, so you can add videos or audio files. You can have a text and audio book at the same time.
2. Supports comic books, no more need of CBZ.
3. it supports annotations internally, so you can read the epub with the same annotations even on another device. This IMMENSELY useful. If I want to share a book, I'm interested to share my annotations too.
4. For scientists and geeks, it supports SVG and MathML.

I think that if Calibre will finally support the conversion to the Epub 3 format, there will be no story for Amazon closed and proprietary formats. For now you have to edit the book, click Tools and Update the internals. Not so intuitive.

PS: well, Epub 3 has also JS. Nothing is perfect