The Diplomat

[unkn0wn]

feeds from https://thediplomat.com

This is not a recipe for the diplomat's magazine edition https://magazine.thediplomat.com, although I make it fetch cover_url.

So oldest article can be any no. of days. I've used 14 as its neither too big nor too small.. make it 30.

Whatever is fetched through the feeds isn't available in the magazine edition and vice-versa.

[unkn0wn]

I wanted to make the magazine edition but there's no links available in the html, somehow it does open links as i click through..

for example this is the link to the current edition which can be fetched from default page (thediplomat.com).

But article links aren't present in the html but i'm able to click through and it takes me to another link within the page.

If we are able to fetch all those article links, the whole text of the article is present within the html (hidden behind paywall).

[kovidgoyal]

That's because its loaded via javascript, you can use the developer tools in a browser to follow what happens, most likely it makes some ajax call to load the content when you click on the text.

[unkn0wn]

so its still possible to make a recipe that could fetch those links? Are there any other recipes that do this?

[kovidgoyal]

its possible you would need to make the ajax requests yourself to get the data. I cant think of an existing recipe that does exactly this, but many recipes make manual requests. Search for the usage of Request in the recipes.