Reversing articles order in a custom news recipe?

[mairabc]

Hello,

first of all, thank you for the wonderful software! Calibre is great, I love it!

I tried to understand the API and looked for something in the forum that would point in the direction I wanted, but with no luck. But in any case, I am sorry if this is already explained somewhere and I missed it.

Now, to the question:

I tried to create a News Source from a website that posts every day a new chapter of a detective story. So, it makes sense to me to have the posts ordered from oldest to newest, and have each post as a chapter of a book. I am pretty sure I can achieve what I want using a custom recipe in Python, but I couldn't find out how. Which method should I modify to get the articles in reverse order in the index?

Any help is highly appreciated

Best,
Maira

[kovidgoyal]

You would have to re-implement the parse_feeds method in your recipe. Have a look at the original in BasicNewsRecipe, should be failrly obvious how to modify it.

[JayCeeEll]

I have been playing around with trying to do this and I think that I need to reverse the entries within the feed.

As far as I can tell the best place to do this would be populate_from_feed in __init__.py from web.feeds

Currently populate_from_feed is:

PHP Code:

    def populate_from_feed(self, feed, title=None, oldest_article=7,
                           max_articles_per_feed=100):
        entries = feed.entries
        feed = feed.feed
        self.title        = feed.get('title', _('Unknown feed')) if not title else title
        self.description  = feed.get('description', '')
        image             = feed.get('image', {})
        self.image_url    = image.get('href', None)
        self.image_width  = image.get('width', 88)
        self.image_height = image.get('height', 31)
        self.image_alt    = image.get('title', '')

        self.articles = []
        self.id_counter = 0
        self.added_articles = []

        self.oldest_article = oldest_article

        for item in entries:
            if len(self.articles) >= max_articles_per_feed:
                break
            self.parse_article(item) 


I have tried adding entries.reverse before the for item in entries but this does not work.

In VB I could just replace the for with a loop from entries.count -1 to 0, but I am not sure how to do this in Python.

Any suggestions?

[kovidgoyal]

for item in reversed(entries):

[JayCeeEll]

I have created a new ticket #4095 which includes a bazaar patch file to allow this behavior.

If the patch gets included in calibre then adding "reverse_article_order = True" to a custom recipe will reverse the article order in the output file.

Next I need to figure out how calibre loads/saves data from the SQLite database so that I can include a new field on the news table and only include articles in the output if they are later than the current article stored in the DB.

[mtg]

When I try using "reverse_article_order = True", I get the following exception on both Mac Leopard and Windows XP:

InputFormatPlugin: Recipe Input running Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Applications/calibre.app/Contents/console.app/Contents/Resources/lib/python2.6/site-packages.zip/calibre/utils/ipc/worker.py", line 90, in main
File "calibre/gui2/convert/gui_conversion.pyo", line 21, in gui_convert
File "calibre/ebooks/conversion/plumber.pyo", line 736, in run
File "calibre/customize/conversion.pyo", line 208, in __call__
File "calibre/web/feeds/input.pyo", line 80, in convert
File "calibre/web/feeds/news.pyo", line 598, in download
File "calibre/web/feeds/news.pyo", line 742, in build_index
TypeError: argument to reversed() must be a sequence

I set up a development environment on my Mac, but when I try to modify the code in news.py, it appears that it always runs the compiled app code rather than the code from the source tree, so I haven't been able to debug it. Is there a way to make workers run the code in the source tree rather than the compiled code in the app?