Substack recipe: Download older posts?

[reasonablehabit]

Hi there,
I've set up Calibre with the Substack recipe and it works great, including my paid subscriptions.

Now the challenge I have is that I want to also download and read older posts from specific Substacks but the RSS feed only has the last ~25 posts or so.
Is there a way to also download older posts?
I'd be fine copying/pasting all the URLs somewhere if that has to be done but how would I have to adjust the recipe to accept URLs pointing to individual articles rather than an RSS feed?
Thanks for any pointers!

RH

[kovidgoyal]

You implement parse_index in your recipe see https://manual.calibre-ebook.com/news.html

[dunderwerk]

Hey reasonablehabit,

Could you please share your recipe for reading Substack archives via Calibre?
I have tried, but not being a programmer, I can't get it to work.

It would be very helpful!

[dunderwerk]

Namaskar Kovidgoyal,

Thank you for your answer to reasonablehabit's question!

Not being a programmer myself (and many like me), could you please show how to implement the code into this recipe?

[ender7]

Originally Posted by dunderwerk

Namaskar Kovidgoyal,

Thank you for your answer to reasonablehabit's question!

Not being a programmer myself (and many like me), could you please show how to implement the code into this recipe?

Google led me to this thread. In my case I just needed something quick to run once to get all the posts that were years old. I used ChatGPT to parse the html for the page containing links to all the posts. Used it to convert the html to a list of dictionaries. Created a parse_html function like so. This would be for one feed containing 2 posts.

PHP Code:

def parse_index(self):
        feeds = [
            {"title": "UI Part 1: The Interaction Medium", "url": "https://www.example.com/first-post"},
            {"title": "UI Part 2: Every Single Frame", "url": "https://www.example.com/second-post"}
        ]
        return [('articles', feeds)] 


Add that to your existing recipe. If you need more details they are in the link provided by kovid. Specifically the recipe for the Atlantic news site.