Automatic sending of RSS news feeds to Kindle as blog/ebook

[janvanmaar]

Currently, any subsequent updates of an RSS feeds sent from Calibre to Kindle *on the same day* suffer by the problem of displaying/sorting incorrectly in Kindle:
https://www.mobileread.com/forums/sho...37#post1218937

Currently, the news sent from Calibre are marked as periodicals (called "issues" in Kindle). However, the default behaviour for standard ebooks (called "documents" in Kindle) or blogs should fix this problem: the latest issue would simply replace the previous one in Kindle, without the feature of saving "older" issues as is the case with periodicals which is broken for updates on the same day.

So my question is: How can I set the type of the document for Kindle in Calibre? I could not find it either in the metadata or in the html. And is it possible to change the type for the news delivered automatically by Calibre?

[kovidgoyal]

It isn't. If you do that, you lose all the news navigation features.

[janvanmaar]

For testing purposes, I have subscribed to a blog at Amazon which is updated many times a day. Kindle recognizes it as blog, has no problems with many updates of it in the same day and all the news navigation features are present (in fact, the navigation seems very similar to the feeds generated and sent by Calibre).

So it seems it should be possible to achieve. I don't mind playing with it a bit to get this working. If you could tell me how to change the document type, I can start playing and let you know how things go...

[kovidgoyal]

the document type is set calibre.ebooks.mobi.writer

[janvanmaar]

thanks, I will have a look at it.

[janvanmaar]

I did not get very far: after some effort I managed to compile from sources, however I get

    for href in cssutils.getUrls(item.data):
  File "install_files/calibre/calibre-src/calibre/lib/calibre/calibre/ebooks/oeb/base.py", line 1066, in fget
    data = self._parse_css(data)
  File "install_files/calibre/calibre-src/calibre/lib/calibre/calibre/ebooks/oeb/base.py", line 1012, in _parse_css
    import_rules = list(data.cssRules.rulesOfType(CSSRule.IMPORT_RULE)
AttributeError: 'CSSRuleList' object has no attribute 'rulesOfType'

error with every news fetching. The version of the downloaded source is the same as the downloaded binary version that works fine though (0.7.28). Does it mean that I have problems with my installation from sources or are the sources different (and contain an CSS bug), just the version has not been updated (yet)?

[kovidgoyal]

you dont need to compile from sources, just setup a development environment as described in the user manual.

As for your specific error, the version of cssutils on your system is outdated.

[janvanmaar]

Well, I have compiled from sources already... and after cssutils update it works fine.

But to the results: It turned out that Kindle's blog = 0x102 = news feed with flat navigation. Which is certainly less of a problem for me than the broken Kindle behaviour of 0x103 (after all, the next/previous section navigation by arrows is supported by 0x102). So I set Calibre to produce 0x102 instead of 0x103 and I was happy - until I found out that subsequent "blogs" are actually not replaced but all instances are placed to Home, even if exactly the same "blog" is sent repeatedly...

This is rather strange as
1) when I copy over USB, the problem is gone (only one instance is kept)
2) when I receive the subscribed blog from Amazon, one instance is kept
So it seems to be a problem of sending via wifi (I use gmail)... and I have no idea why this is happening...

[kovidgoyal]

it may be that with email delivery amazon rewites some internal metadata in the file causing this behavior.

An alternative is to setup the content server and get your files from that.

[janvanmaar]

The problem with that solution is that I would either have to always manually download the file from the server or to jailbreak the Kindle (which I plan to do but only after a few months as it voids the warranty). At least I do not know how to send a file to Kindle without jailbreaking.

[kovidgoyal]

yeah...well that's the problem with buying devices from totally closed companies like Amazon

[janvanmaar]

indeed... well, it is just my first reader, I will certainly take this into account when choosing the next one
Anyway, it seems that I am stuck now.

[janvanmaar]

Just a small update: it turned out that the sending through Amazon is actually the main reason of this issue - 0x103 behaves just fine when sent over usb too. And the reason is quite clear when looking at the filenames returned by Amazon - they are all different, containing some asin sequences and more... So this whole thread and my testing was rather pointless, I should have noticed the real problem immediately.
EDIT: However, of course it does not diminish your help, Kovid - thanks a lot for that! If nothing else then at least I learnt something again.

[janvanmaar]

Another update: Downloading via Calibre content server does not work either. From some reason (may both feature or bug), the files are uploaded to the server with _number suffix, where number is the internal Calibre unique number of a document (Calibre Quick Start Guide gets number 1 and every added document a number increased by 1).
This means that they show up all as different instances in Kindle, creating the same problems as in case of delivery via Amazon. So either a Calibre content server modification or a separate server would be needed for this to work properly.
Oh well, I am getting too tired of this issue now and finally giving up.

[kovidgoyal]

Yes, the content server always appends db id to filenames to ensure that two records with the same metadata don't clobber each others files.

If you don't care about that, it should be fairly easy to tweak the server to not do this, you'd need to modify server.content and server.mobile