Recipe for Ukrainian Economic / Legal news sites.

[Dereks]

Hi folks,

I have to tell that I'm a total rookie here. For half a year I've been using this shamefully primitive self-made recipe to download news from Ukrainian economic sites.

Here is the recipe itself:

class AdvancedUserRecipe1268599504(BasicNewsRecipe):
    title          = u'Economics UA'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'Publications', u'http://www.epravda.com.ua/rss/id_433/'), (u'Columnists', u'http://www.epravda.com.ua/rss/id_432/'), (u'Finance.UA articles', u'http://feed43.com/6441846012758810.xml'), (u'Liga:News', u'http://news.ligazakon.ua/news_rss/tape_clauses.xml')]

    def print_version(self, url):
	if url.startswith('http://www.epravda.com.ua'): 
   	     return url + 'view_print/'
	else: 
	     return url

Generally it worked Ok, excluding the fact, that sometimes fonts looked funny, but that was fine by me.
But since inclusion of periodics script for Sony Readers this recipe started to act funny. When opened on my PRS-650, it either freezes, slows or crashes the device, even though random pages, I've manged to load seem to look ok. This problems seems to be specific only to this feed - other standard calibre recipes or self-made recipes are handled perfectly by the reader.
I do realize that throwing in several feeds into one recipe is not a good thing, so don't object splitting it up, just don't know how to do it properly.
And yeah, the funny RSS-feed address at feed43.com is actually a service I use, which allows you to create an RSS-feeds from almost any site, since that site don't have specific RSS-feeds for the column I want to read.
http://news.finance.ua/ua/~/2 - that's the page it fetches news from, if it's of any importance.
Calibre itself seems to handle this issue fine and fetches the full article, but maybe it's the reason for reader crashes.
Thanks in advance for any help!

[kovidgoyal]

generally speaking, you should remove as much extra guff from the downloaded articles as possible. Use the remove_tags and keep_only_tags features for this.

[Dereks]

ok. At least I figured where the problem lies with glitches. It now seems to load quite well, but the other thing poped up.
Since it's Ukrainian, the alphabet is Cyrillic and when loaded to the Reader or Reader Library all the Cyrillic letters are replaced by question marks. My Reader uses custom firmware which supports Cyrillic and normally all the lrf or epub files are displayed correctly.
So I don't really know where the problem may lie?
I have a slight suspicion, that it may have something to do with the periodicals, because before that, epub or lrf recipe files as simple ebooks where displayed properly.

[kovidgoyal]

well if you change your output profile to say cybook opus then the epubs will no longer be periodicals, see if that helps.

[Dereks]

It does help - changed to LRF and everything worked perfect.
But still I find Periodicals menu quite useful and don't like all the book folder messed up by news recipes...
also, just notices that it didn't create a proper TOC for the Periodical EPUB. It's just Front Page there and that's it...
Update: just checked with epub file - non-periodical outlaying doesn't display fon't properly either. So it's something about the encoding (in the calibre viewer everything is fine).
I will check with the author of the firmware, but I think it uses Unicode. Can it be the case, that calibre converter uses something different?
Will also try to convert some regular books into epub in Calibre and will tell the result...
Final Update:
Yep, it's definitely some problem with Calibre Epub converter (maybe not a problem, just me being lame with customizations) - all calibre-created epubs show all cyrcillic letters as question marks, while external load on reader properly.