Problem with The Guardian & The Observer recipe?

[dinwood]

I've had problems with this recipe since updating from Calibre 2.5 to 2.8, but reverting back to 2.5 doesn't fix it. At first the recipe returned very little (0.2Mb vs the usual ~10Mb on a weekend), then it stopped running with the error message "NoneType' object has no attribute 'findAll'. I contacted Calibre with a bug report but they redirected me here.
Can anyone shed any light?

[dhell]

[ Reasons right now mean I can only use Calibre 0.7.28, last version, for Mac OS X 10.5.8 ].

I have downloaded The Guardian/Observer for many weeks with this. Last week, about Oct 27, 28 only some pages became available to read, the remainder the URLs only, those entries always at the end of each page. Thereafter I received only the URLs, no pages at all. I have tried various ways to resolve that, but not being so softwarish, and finding 'python' hard to understand (and apparently anyway requires indents. Hmm?), I was not successful. Oh AND the Guardian web sights are now rather different from what they were up to a week ago, that is ... changed.
I can see each subject title being downloaded in sequence. The resulting table of contents set works fine. But click on a page title and get only the URL.

I tried to set up the feed[], based on the Daily Telegraph feed [] set, but observe the guardian.recipe is currently rather different from the telegraph.recipe, and the Guardian I tried, in my lack of knowledge and meaning, says 'no!'. This is all perhaps a significant montypython software construct.

It has been easy to set the ignore section[] -- in my case 'Sport', although seemed not to work for 'Observer Sport' -- but there is no way I see to ADD other sections I want, presumably not "basic" enough for the aroma of beautiful.soup <s>

So please, software pythons wind around these things and squeeze them to rights.

[gpauk]

I'm finding a similar problem -- frequently articles in the index are not in the file.. Eg, today the link to the article "DNA Scientist James Watson sells Nobel prize medal" fails -- the article is not there. Sometimes up to 25% of the articles fail in this way....

[AlanD]

Working through today's paper a pattern becomes obvious. This link fails: http://www.theguardian.com/science/2...el-prize-medal but this does not: http://www.theguardian.com/politics/...-neoliberalism

It depends on the value of XXX in www.theguardian.com/XXX/. All links are OK if XXX is world, or business, or commentisfree, or us-news, or uk-news, or politics, or society (or a few others). All links fail if XXX is stage, music, science, books, media, film, money, technology (and a few others).

Any suggestions on how to fix it? How can I look at any intermediate files that get built and then purged?

[PeterT]

I sense there is a new design coming to the Guardian web site and at some point all content will be like stage / music / ....

[kovidgoyal]

This commit, should allow extracting content from the new design, however, you will have to wait till that website stabilizes before this recipe can be updated properly.

https://github.com/kovidgoyal/calibr...23e1f52a82958e

[AlanD]

Thanks, Kovid, for your quick response. At first sight it works OK with the UK version of the website.

Alan

[gpauk]

Yes -- looks good so far! Thanks