[Kindle, periodical] Remember last article selected

[tylau0]

It looks to me that the periodicals generated by calibre news recipe (in .mobi format) can't remember the last article being selected in the "Sections & Articles" view of Kindle 3. That is, every time when I select a section (say the 3rd section) and then pick an article, read it, and then click on "View Sections & Articles" to go back to the "Sections & Articles" view, the cursor always returns to the first section, no matter what my previous choice on the section last time is.

This may make browsing periodical content an inconvenient experience, as one may wish to stay in the section he has selected.

I find that it does not happen with periodicals purchased from Amazon (in .azw format), so I believe that's an issue with the Calibre software, or the .mobi file format itself.

I would be glad if the developer of Calibre can look into this issue. Thanks!

[markvdvelde]

I'm having exactly the same problem. Thanks

[janvanmaar]

Yes, this is a known problem many users complain about - there are at least 2 other threads and 2 bug reports already within the relatively short time from the release of the Kindle 3.1 software:

https://www.mobileread.com/forums/sho...d.php?t=120613
https://www.mobileread.com/forums/sho...d.php?t=122361
http://bugs.calibre-ebook.com/ticket/9147
http://bugs.calibre-ebook.com/ticket/9201

From the Calibre sources, it is not clear to me what could be the problem, it may require some additional reverse-engineering - which is may be the main reason why it is not sorted out.
It is perhaps promising that there exist third party produced mobi periodicals not suffering by the problem: Instapaper generated Kindle periodicals (except when Instapaper is used via Calibre - then the problem appears). Which means that either Marco reverse-engineered the format himself too or he uses some mobi generation tools that do not suffer by the problem.

At the moment, the only way to avoid the problem is disabling upgrade to Kindle 3.1 software - unless you already have it, of course.

[kovidgoyal]

Yeah it's going to require reverse engineering, and GRiker, the guy who originally wrote the periodicals code no longer uses a Kindle, so this is languishing. I may take it up at some point, but, as you can imagine, I have a thousand other things clamoring for my attention. So if you want it done quickly, you're going to have to do it yourself.

I'm guessing Instapaper uses kindlegen, IIRC there is a way to get kindlegen to produce periodcal output.

[nickredding]

I'm working on this. However, after several days of effort I must say this is an extremely difficult problem. I suspect Amazon is formatting periodicals in a way that is not MOBI-compliant, that is they have introduced a wrinkle that the Kindle depends on for the "Sections & Articles" view and the BACK button to work correctly, but that is not necessary for a fully MOBI-compliant reader to work properly.

Note to Kovid: I emailed Marco about how he does his Kindle formatting for Instapaper, and he wouldn't tell me, claiming he didn't want to "irritate Amazon." If he is using kindlegen, I'd be interested in how he is getting it to generate periodical fomats--I haven't been able to figure out how to make it do that. Any clues?

[kovidgoyal]

I may be misremembering, at least a bit of quick googling didn't turn up anything. Try PMing GRiker in case his memory is better

[janvanmaar]

they have introduced a wrinkle that the Kindle depends on for the "Sections & Articles" view and the BACK button to work correctly, but that is not necessary for a fully MOBI-compliant reader to work properly.

I suspect this is the case indeed.

I emailed Marco about how he does his Kindle formatting for Instapaper, and he wouldn't tell me, claiming he didn't want to "irritate Amazon." If he is using kindlegen, I'd be interested in how he is getting it to generate periodical fomats--I haven't been able to figure out how to make it do that. Any clues?

This is a speculation but I guess that Marco is using Amazon/Mobipocket tools indeed - Instapaper was reported to work with the new layout a day after the very first 3.1 candidate was released - not impossible but rather unlikely that he would have noticed and reverse-engineered the issue within a day. However, I did a quick search for mobigen periodicals generation and did not find anything either. Although I am still on 3.0.3 and not planning to upgrade to 3.1 soon, so this issue has low priority for me at the moment...

[tylau0]

As this affects all Kindle 3 users, may I suggest the following temporary workaround?

Disable generation of periodical (instead generate a e-book) using the Kindle profile

It turns out to be changing one line in the implementation of the following function of calibre.web.feeds.news

def create_opf(self, feeds, dir=None):

Turn the line

mi.publication_type = 'periodical:'+self.publication_type+':'+self.short_title()

to

mi.publication_type = self.publication_type+':'+self.short_title()

Then we do not need to change any existing recipe to cater for Kindle 3. Currently I have to do it in the recipe-level (See here for an example).

What do you think?

[nickredding]

Originally Posted by janvanmaar

This is a speculation but I guess that Marco is using Amazon/Mobipocket tools indeed - Instapaper was reported to work with the new layout a day after the very first 3.1 candidate was released - not impossible but rather unlikely that he would have noticed and reverse-engineered the issue within a day.

I doubt that Amazon actually changed their format for the 3.1 release--rather, they are probably exploiting a proprietary "feature" of their format that has always been there but not created any issues until they introduced the "Sections & Articles" view.

Note that if you select "Articles List" from the "Sections & Articles" view you get the linear list of articles as was available prior to 3.1, and the BACK button works properly if you select an article from this list and the press BACK. Interestingly, how you scroll to the "Article List" affects whether it works properly. If you scroll down the left side of the "Sections & Articles" page (through the section list) it works properly. However, if you scroll down the right side (via the article list) you get a wierd view of the article list (one per screen with the article title at the top, and the select button takes you back to the "Sections & Articles" view instead of the article).

I'm guessing the Kindle isn't doing page navigation for periodicals in a pure browser-style fashion (if it was, these problems wouldn't occur). Rather, I suspect the Kindle is building a linked structure representing the periodical contents and some of those links end up missing (in the case of going BACK to the "Sections & Articles" view from an article, so section 1 article 1 is the default) or they are pointing somewhere that doesn't make sense (in the case of scrolling the "Articles List" via the right side of the "Sections & Articles" view). Prior to 3.1, this structure was simpler. Now, it is more complex and there is something critical in the Kindle format that Calibre is missing, hence the problems.

[kovidgoyal]

I don't think that this bug is severe enough to dump the periodical structure. The periodical structure provides many other benefits, like auto categorization on the Kindle with expiry of old issues, the ability to view section lists. Not to mention all the people with Kindle 2s and those who chose not to update to 3.1

[nickredding]

Originally Posted by tylau0

As this affects all Kindle 3 users, may I suggest the following temporary workaround?

Disable generation of periodical (instead generate a e-book) using the Kindle profile

I don't see how this would solve anything. If you format periodicals as books, you'll lose all the periodical navigation (sections lists and article lists). You might get an HTML-style table of contents at the beginning or end of the document, but using that is a real pain compared to using Kindle's periodical naviagtion.

The periodical navigation in 3.1 isn't broken for calibre periodicals, it just has a couple of irritating issues (see my previous message).

[janvanmaar]

I don't think that this bug is severe enough to dump the periodical structure.

I guess that tylau0 meant a non-default option rather than dumping it completely... but even then I have to agree with Kovid and Nick - it sounds rather useless to me too...

[tylau0]

Originally Posted by nickredding

I don't see how this would solve anything. If you format periodicals as books, you'll lose all the periodical navigation (sections lists and article lists). You might get an HTML-style table of contents at the beginning or end of the document, but using that is a real pain compared to using Kindle's periodical naviagtion.

The periodical navigation in 3.1 isn't broken for calibre periodicals, it just has a couple of irritating issues (see my previous message).

If we format the document as an e-book, at least we have a way to navigate between articles which is intuitive and familiar to the Kindle users:

The first time we read the document, we can go to the "Table of Content" using the Kindle menu. From the table of content, we click on an article to read. Then when one is finished, he can click the "Back" button of the Kindle to go back to exactly where he left from the table of content.

We don't have such an alternative if the document is formatted as a periodical - first, the "go to TOC" function does not appear in the Kindle menu if the document is a periodical; second, the TOC of the periodical is at the end of the document. In Kindle there is no way to go to that end directly.

The "article view", I think, is not that intuitive to the users. Kindle users expect everything can be done perfectly with the "Sections & Articles" view. Also, we are limited to 3 article titles per each page in the article view. This does not give a better overview on what is inside the section than displaying the titles as indented text in TOC.

The above are why I suggested generating an e-book rather than a periodical in my previous post.

[tylau0]

Indeed those additional benefits you mentioned are why I am always wanting to push the periodical version of my recipe (I did push one this month but I have decided to drop it for the time being, for the navigation problem together with a few CJK display issues in Kindle 3).

But the navigation problem really affects the readability of the document... hope we can solve this problem very soon!

Originally Posted by kovidgoyal

I don't think that this bug is severe enough to dump the periodical structure. The periodical structure provides many other benefits, like auto categorization on the Kindle with expiry of old issues, the ability to view section lists. Not to mention all the people with Kindle 2s and those who chose not to update to 3.1

[tylau0]

Is it possible to provide the profile according to the Kindle firmware version?

Originally Posted by janvanmaar

I guess that tylau0 meant a non-default option rather than dumping it completely... but even then I have to agree with Kovid and Nick - it sounds rather useless to me too...