Calibre News Source 'The Hindu' Recipe needs correction

[snji]

The Hindu - OpEd - Yes/No/It'sComplicated format articles are not being fetched completely, only 1/3 rd parts are being downloaded.

Ex: http://www.thehindu.com/opinion/op-e...le22828585.ece

These type of articles have 3 parts - Yes, No, It's Complicated.

I am suspecting calibre is discounting duplicates in title and hence after downloading 'Yes' Part, disregard other 2 parts.

Is it possible to update the recipe? or at least let me know what code I can put in the recipe.

I can edit the recipe. That is the extent of my IT capabilities.

Thanks

[kovidgoyal]

Since that is a single aritcle, not three, I doubt that duplicate removal is the problem, but it is easy to test,simple remove the

ignore_duplicate_articles = {'title', 'url'}


from the recipe.

[snji]

Hi, kovid

Thanks for response. I have updated the code.
Sadly it is weekly article so Paper of yesterday did not contain 'Yes, No ...'
And I do not how to download specific date paper or 'x days' back paper.
Is there any easy code, I can add in recipe for this ?

If nothing is there, I will update on 3rd March

[bobbysteel]

to test edit the recipe where the line has index_to_soup and a url with 'todays-paper' (i think 63) change that URL to the date you want
like
http://www.thehindu.com/archive/print/2018/03/01/

[snji]

Originally Posted by bobbysteel

to test edit the recipe where the line has index_to_soup and a url with 'todays-paper' (i think 63) change that URL to the date you want
like
http://www.thehindu.com/archive/print/2018/03/01/

Hi, Thanks for the suggestion. I tried the modification as follows

# Modified on 2018Mar06 To download specific date paper        
        soup = self.index_to_soup('http://www.thehindu.com/archive/print/2018/03/02/')
        #soup = self.index_to_soup('http://www.thehindu.com/todays-paper/')
        nav_div = soup.find(id='subnav-tpbar-latest')
        section_list = []

But Calibre is not downloading any article with the modification.
I have compared code of pages on 'todays-paper' and 'print/2018/03/02/' & it looks like the id = subnav-tpbar-latest contains complete URL in first case but only links in 2nd case which is causing Calibre to not able to find any article

I have attached Calibre logs with relevant screenshots. Requesting your further help in clarification if possible . Thanks

[snji]

Originally Posted by kovidgoyal

Since that is a single aritcle, not three, I doubt that duplicate removal is the problem, but it is easy to test,simple remove the

ignore_duplicate_articles = {'title', 'url'}


from the recipe.

Hi Kovid,
Commenting out the code did not work. Only Yes part is being downloaded.
Any other suggestion ?

[kovidgoyal]

I just tried it and it works for me, see screenshot

[snji]

Originally Posted by kovidgoyal

I just tried it and it works for me, see screenshot

Howwww !!!

Just now, I have removed the line 25 from the original recipe all together & kept everything same as original. Still I am not getting the other two parts...

Have you modified the original recipe or it was a trial code only ?

[kovidgoyal]

Nope, no modifications.

[0Arvind]

Same issue, only 1 of the 3 part getting downloaded and removing 25th line in code is not helping.
https://www.thehindu.com/opinion/op-...le24646941.ece
Only 'YES' part gets downloaded leaving 'NO', and 'it's complicated'.

[0Arvind]

Originally Posted by kovidgoyal

Nope, no modifications.

Hello Kovid,
I'm really grateful to you for making such wonderful software which makes reading awesome, and turning e-reading devices cost effective.

Issue:-
https://www.thehindu.com/opinion/op-...le24646941.ece. Only yes part is being downloaded, leaving No and it's complicated. Such kind of Article comes every week on Friday, in "The Hindu" Newspaper.

I'm not from IT background, but I tried removing the the code, ignore_duplicate_articles = {'title', 'url'}
It isn't working for me as well, I guess I'm missing something out there.

I have read your articles that you work 80 hours a week and you must be very busy, but if you can take out time to upload the recipe here with modifications, That would be helpful, or tweak the inbuilt recipe.

[kovidgoyal]

I'm afraid I cannot fix the recipe for an issue I cannot reproduce, sorry.

[Subhash]

What to change in the recipe of " The Hindu" to remove image ?