Page Numbers in electronix text formats

[bob_ninja]

This is a continuation from "News from Bookeen about firmware upgrade"
I thought this topic deserves a separate thread.

The original discussion has take a rather negative tone, so I would like to hit a reset button and look at this issue from a more constructive point.

In general I would agree with demands to show page number *IF* the device already has the information as implied by the "Go to ..." (page) command. However, let us examine what exactly is involved in computing page numbers for electronic formats.

In the old days it was simple. As soon as you print physical pages you had numbers by default. You simply assign numbers as you go. Then came SGML, followed by simple HTML and other derivatives like MobiPocket.

The critical difference is that new formats like HTML define layout instructions (<p>age break, line <br>eak, etc.) and not the actual page/diaplay content. In fact, the original HTML before CSS style sheets didn't even define font information. What font to use for <h1>eader? Well user could choose anything. Furthermore as displays have many different dimensions and windows can be resized, the physical page size is really unknown, undefined. Instead, HTML content was designed to "flow" into whatever display container is used.

Thus emerged PDF which decided to nail down every last detail, from fonts to spacing, images, etc. even page dimensions. Once page dimensions is fixed then it is easy to divide content into pages in PDF *REGARDLESS* of display device or window dimensions. Keep in mind what is being lost here. This is a clear compromise being made here. You gain fidelity, consistency while loosing flexibility. In some case, like for smaller screens, the lost flexibility can be painful.

So back to the question of page numbers. The 1st question is what is a page in the 1st place. Well it is the display device, or browser window, whatever is displaying content. Thus page is defined by our display. Today virtually all displays are simply a 2D matrix of pixels. The process of converting a certain format (like HTMK or MobiPocket) into pixels that can be displayed on a physical display (in this case eInk 6" display) is called "rendering". In short, a rendering engine (like MobiPocket renderer in Cybook) reads the content, executes the instructions like <p> and <br> and converts them into pixels according to a set of rules.

So to actually build a page for display you take some HTML content and render it into pixels. Those pixels (image) is an actual page that is ready for display. There are several important characteristics of rendering:
- it consumers significant computing resources
(more for more complex formats and less for simpler ones)
- thus it tends to consume significant energy (battery power)
- also tends to take relatively long time

Typically generated pixels (images) are much bigger than the input content, so they are only used for display and not kept anywhere. For instance, our typical 1 Mb and smaller MobiPocket books produce many magabytes of images when rendered. Hence rendered page images are only generated on demand, in general.

The HTML content is simply a long sequence of bytes, which in turn consist of a mix of words and rendering instructions (like heading, paragraph, line break, etc.) The Cybook computer simply knows that it is display content from byte #23,859 for instance. No page number. It just keeps reading HTML content from this position until Cybook page is filled. Simple.

Say you want to figure out what is the current page number. This is *THE QUESTION*

Well, the simple approach might be to start rendering content from the 1st byte (in memory only, it doesn't actually get displayed) and keep rendering until you reach the current position, byte #23,859. Say we rendered 31 pages to get to this byte position. Thus we learned that the current page number is 31.

The similar process could be used by the "Go to ..." (page) command to select the new byte position to render from. Again start rendering from byte 0 until the page count reaches the desired page, at which point you display the last rendered page.

There are a lot of problems with this approach. If you are trying to get to page 1,000 then the Cybook processor has to do a lot of work, consumer a lot of batter energy and worst of all would take a very long time (maybe minutes even). In computer speak, this approach doesn't scale. It may work for the initial 10-20 pages, but thereafter becomes too slow and uses too much battery power.

Then you go to work trying to figure out faster ways to manage the page numbers. Maybe the computer keeps track of pages as they are rendered and maps each page number to a corresponding byte position. That also doesn't work since I can jump from TOC to a middle of a book without rendering the initial pages (which could be many). And so on.

In effect, the Cybook programmers (as well as others trying to render HTML or MobiPocket) resort to taking shortcuts, or having Cybook make some guess or approximations to determine a page number without doing so much work. The compromise is lost precision (as reported in the other thread) for far less computation (and corresponding less power usage and better spead).

Of course, I have no clue what is the actual algorithm used and how precise it is (not). Some of you expressed interest in a 100% precise page number. Again, you need to step into Cybook maker's shoes.

SIDE NOTE: Before you scream, it safe to say that page numbers are not important to the majority of users that use browsers and other pageless systems every day without complaints. Sure you may be keen to have them 100% correct, but you are a minority. That being said ....

Bookeen is looking at this marginal feature of a low priority and assigning to it the proportional engineering effort. Of course, they have deadlines and limited resources, so it is just common sense that some manager will decide to assign 10 dev-days, for instance. So this single programmer has only 10 days to implement page "features". Thus they decide they can only do a single "Go to ..." (page) command using some less than great approximation for a barely/reasonably functional go to page command. They just don't have the time/resources to do more. Perhaps due to lower priority they decide they'll assign more effort to it for another release.

So the bottom line is they will spend as much time/effort as X units @350E / @$350 buys them. Simple business decision. You cannot spend more money than you earn.

That is why I said that users with higher expectations should get iRex product as it costs about 2x more. Therefore, iRex has more resources to implement page features better.

In summary,
Page numbers is a difficult feature to implement for certain formats. Such small devices like Cybook lack processing power for precise pagination. It will take both software engineering effort and likely more processing power to obtain more/better page features in the future. How far in the future is hard to say.

This feature like any other tends to increase costs and causes corresponding device price increase. Cybook walks a fine line between providing the desired features like pagination and keeping device price low enough for wider appeal.

Bottom line is, give them a break

[HarryT]

Hi Bob,

Thank you for your useful summary.

I think, however, the significant thing to note is that the way that page numbers behave on the Gen3 is entirely consistent with the way they work on every other MobiPocket device. This is not some nasty bug that Bookeen have introduced - it appears to be behaviour which is inherent to the Mobi reader.

Personally - as I've stated before - I consider page numbers to be an irrelevent hang-over from paper books which have no place in the eBook world except, perhaps, as a method of making large "jumps" to an approximate position in a book.

[Sparrow]

Originally Posted by bob_ninja

The 1st question is what is a page in the 1st place. Well it is the display device, or browser window, whatever is displaying content. Thus page is defined by our display.

Maybe one approach could be to break the connection between a 'page' and the screen display, so that a page could span more (or less) than one screen and rendering calculations aren't required.
If pagination was somehow embedded in the file it would produce stable and consistent page numbering.
Of course then an attempt to 'go to page 136' might only display the first part of the page on the screen; but it would only be a key press or two to get to the bit of the page you wanted (probably no less efficient than the current method).
This approach would also maintain consistency of page numbering regardless of display format and device, and reduce the workload for the device.

[tompe]

Originally Posted by Sparrow

Maybe one approach could be to break the connection between a 'page' and the screen display, so that a page could span more (or less) than one screen and rendering calculations aren't required.
If pagination was somehow embedded in the file it would produce stable and consistent page numbering.
Of course then an attempt to 'go to page 136' might only display the first part of the page on the screen; but it would only be a key press or two to get to the bit of the page you wanted (probably no less efficient than the current method).
This approach would also maintain consistency of page numbering regardless of display format and device, and reduce the workload for the device.

That seems to be how FBReader works.

As has been pointed out in the other threads it is not the page numbers that are important. What is important is that you know where you are in the book and the absolute total length. For a text file that would be trivial to implement. For a HTML file you have to ignore markup that does not generate text. If you have this measure you will soon learn how long time it take to read things and how many forward pagings are required for a certain length.

It seems like FBReaders approach is something similar to this.

[nikben]

Just throwing in a torch in the hayloft here....(instead of introducing myself in the appropriate forum...). I just loved the pagination in Microsofts Reader. It was my preferred reading software before I found the fullscreen-option in Mobipocket. . The pagination saves time whenever I forget to bookmark.

[HarryT]

I'm not at all sure that Microsoft Reader does proper pagination, actually. Try the experiment I've described elsewhere with MobiPocket: ie page forward from the start of a book until you get to the start of a certain chapter, and make a note of the page number. Then go back to the start of the book and do a direct jump to that same page number. See if you finish up at the same place. In MobiPocket you don't; I'd be interested to know whether or not you do in Microsoft Reader!

[nikben]

...you might have a point....It has happened that the pagenumber I thought I was at, was not accurate, but I always put that down to my "goldfish-memory"....I will test it...

[DaleDe]

Originally Posted by HarryT

Hi Bob,

Thank you for your useful summary.

Actually the summary did not cover all the cases. A real book has forced page breaks for things like chapter boundaries and widow and orphan rules can change the page break also. In addition images will cause page breaks since the image can't be split very easily. Every time the font is changed, and the cybook lets you pick your own, the entire book page scheme is thrown out the window. Different fonts kern differently so even if you have the same size the same number of words won't fit on a page.

When a book has specialized text like a quote with different margins or a poem embedded in it then the rules change once again.

Not only is pagination computed in the forward direction it has to be computed backwards as well. If you look at a real mobipocket implementation of their reader and walk past a chapter and then back up you will notice that they attempt to keep the partial page intact (which is done by remembering it from when you went forward). If you go to a new chapter and then backup the history isn't there and the partial page is rendered improperly.

Book readers that support real pagination have a limit to fonts and font sizes and then they store every page boundary in a separate database. This effort is best done on a computer before the book is ever downloaded since it is very compute intensive. This removes flexibility like changing font sizes or font families on the fly after the fact.

Dale

[tompe]

I think page numbers are used for two things. One is for referencing positions in the book. The other is so you know how long the text is and were you are in the text. For this second purpose the exact pagination is not so interesting and the functionality ought to be simpler to implement then real stable pagination.

[JSWolf]

Originally Posted by nikben

Just throwing in a torch in the hayloft here....(instead of introducing myself in the appropriate forum...). I just loved the pagination in Microsofts Reader. It was my preferred reading software before I found the fullscreen-option in Mobipocket. . The pagination saves time whenever I forget to bookmark.

But if you notice that in order to get the page numbers, it has to paginate internally. Do you want the gen3 to do that every time you change the font or the font size or the first time you open the book?

[kkingdon]

The "Palm" reader does the pagination in the background the first time you open the document. Until the background process finishes, all gotos and progress bars are only approximate. There is some UI feedback -- the progress bar is "greyer" until the pagination is complete. Like Mobipocket, the Palm ereader format compresses the text. I don't know how the Palm ereader deals with embedded markup.

[jgray]

Originally Posted by HarryT

I'm not at all sure that Microsoft Reader does proper pagination, actually. Try the experiment I've described elsewhere with MobiPocket: ie page forward from the start of a book until you get to the start of a certain chapter, and make a note of the page number. Then go back to the start of the book and do a direct jump to that same page number. See if you finish up at the same place. In MobiPocket you don't; I'd be interested to know whether or not you do in Microsoft Reader!

I tried your experiment. It takes me to the same page number as the one I found by manually paging. For some reason, the Alt-G keyboard shortcut for "goto page" didn't work for me, but clicking on a particular spot on the scrollbar at the bottom of MS Reader took me to the same page number every time. It looks like Microsoft got this feature right.

[jgray]

Originally Posted by JSWolf

But if you notice that in order to get the page numbers, it has to paginate internally. Do you want the gen3 to do that every time you change the font or the font size or the first time you open the book?

Since screen sizes are different, the page numbers have to come from somewhere. So what's wrong with generating them when you load the book? If the way MS Reader does it works and doesn't cause any noticeable slowdown in reading, what is the problem?

[JSWolf]

Originally Posted by jgray

Since screen sizes are different, the page numbers have to come from somewhere. So what's wrong with generating them when you load the book? If the way MS Reader does it works and doesn't cause any noticeable slowdown in reading, what is the problem?

Sony does page numbers with BBeB. But to do so eBook Library does the pagination when it put the eBook on the reade ror if not, the reader will do it. But once done, you have valid working page number that do mean something.

To be honest, I think that's really the only way to do it.

[Bimble]

In the thread that inspired this, one person mentioned more than once that his desire for page numbers had a lot to do with reading short stories in a compilation. Without page numbers (and a table of contents that reflects them, I assume), he couldn't tell how far along he was in a short story.

One approach to solving a problem like that might be a change to the file format to store chapter bookmarks, then a change to the reader to display progress within a chapter in place of/in addition to the progress display for the document as a whole. That would involve less work on the reader device's part than pagination, anyway. Since I haven't seen a format that provides a dynamic table of contents with page numbers (just with hyperlinks), accurate page numbers in a list of chapters/stories would involve a change to reader software anyway. It seems like right now the best approach is to select a chapter in the table of contents, note the page it takes you to, then go back and select the following chapter, so you can compare the page numbers.