Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 02-22-2018, 10:41 PM   #1
snji
Junior Member
snji began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Feb 2018
Device: Kindle Paperwhite
Calibre News Source 'The Hindu' Recipe needs correction

The Hindu - OpEd - Yes/No/It'sComplicated format articles are not being fetched completely, only 1/3 rd parts are being downloaded.

Ex: http://www.thehindu.com/opinion/op-e...le22828585.ece

These type of articles have 3 parts - Yes, No, It's Complicated.

I am suspecting calibre is discounting duplicates in title and hence after downloading 'Yes' Part, disregard other 2 parts.

Is it possible to update the recipe? or at least let me know what code I can put in the recipe.

I can edit the recipe. That is the extent of my IT capabilities.

Thanks
snji is offline   Reply With Quote
Old 02-23-2018, 04:00 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,727
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Since that is a single aritcle, not three, I doubt that duplicate removal is the problem, but it is easy to test,simple remove the

ignore_duplicate_articles = {'title', 'url'}


from the recipe.
kovidgoyal is offline   Reply With Quote
Old 02-24-2018, 01:05 AM   #3
snji
Junior Member
snji began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Feb 2018
Device: Kindle Paperwhite
Hi, kovid

Thanks for response. I have updated the code.
Sadly it is weekly article so Paper of yesterday did not contain 'Yes, No ...'
And I do not how to download specific date paper or 'x days' back paper.
Is there any easy code, I can add in recipe for this ?

If nothing is there, I will update on 3rd March
snji is offline   Reply With Quote
Old 03-02-2018, 06:40 AM   #4
bobbysteel
Big Poppa
bobbysteel began at the beginning.
 
Posts: 67
Karma: 10
Join Date: Jul 2010
Device: Nook
to test edit the recipe where the line has index_to_soup and a url with 'todays-paper' (i think 63) change that URL to the date you want
like
http://www.thehindu.com/archive/print/2018/03/01/
bobbysteel is offline   Reply With Quote
Old 03-06-2018, 12:11 AM   #5
snji
Junior Member
snji began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Feb 2018
Device: Kindle Paperwhite
Quote:
Originally Posted by bobbysteel View Post
to test edit the recipe where the line has index_to_soup and a url with 'todays-paper' (i think 63) change that URL to the date you want
like
http://www.thehindu.com/archive/print/2018/03/01/
Hi, Thanks for the suggestion. I tried the modification as follows

Code:
# Modified on 2018Mar06 To download specific date paper        
        soup = self.index_to_soup('http://www.thehindu.com/archive/print/2018/03/02/')
        #soup = self.index_to_soup('http://www.thehindu.com/todays-paper/')
        nav_div = soup.find(id='subnav-tpbar-latest')
        section_list = []
But Calibre is not downloading any article with the modification.
I have compared code of pages on 'todays-paper' and 'print/2018/03/02/' & it looks like the id = subnav-tpbar-latest contains complete URL in first case but only links in 2nd case which is causing Calibre to not able to find any article

I have attached Calibre logs with relevant screenshots. Requesting your further help in clarification if possible . Thanks
Attached Thumbnails
Click image for larger version

Name:	Calibre_01.png
Views:	97
Size:	69.3 KB
ID:	162666   Click image for larger version

Name:	Calibre_02.png
Views:	86
Size:	35.8 KB
ID:	162667  
Attached Files
File Type: txt Calibre_03.txt (4.3 KB, 88 views)
snji is offline   Reply With Quote
Old 03-08-2018, 10:39 PM   #6
snji
Junior Member
snji began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Feb 2018
Device: Kindle Paperwhite
Quote:
Originally Posted by kovidgoyal View Post
Since that is a single aritcle, not three, I doubt that duplicate removal is the problem, but it is easy to test,simple remove the

ignore_duplicate_articles = {'title', 'url'}


from the recipe.
Hi Kovid,
Commenting out the code did not work. Only Yes part is being downloaded.
Any other suggestion ?

snji is offline   Reply With Quote
Old 03-09-2018, 01:37 AM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,727
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I just tried it and it works for me, see screenshot
Attached Thumbnails
Click image for larger version

Name:	Screenshot_20180309_110641.png
Views:	94
Size:	822.2 KB
ID:	162710  
kovidgoyal is offline   Reply With Quote
Old 03-09-2018, 04:00 AM   #8
snji
Junior Member
snji began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Feb 2018
Device: Kindle Paperwhite
Quote:
Originally Posted by kovidgoyal View Post
I just tried it and it works for me, see screenshot
Howwww !!!

Just now, I have removed the line 25 from the original recipe all together & kept everything same as original. Still I am not getting the other two parts...

Have you modified the original recipe or it was a trial code only ?
snji is offline   Reply With Quote
Old 03-09-2018, 04:01 AM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,727
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Nope, no modifications.
kovidgoyal is offline   Reply With Quote
Old 08-10-2018, 03:18 AM   #10
0Arvind
Junior Member
0Arvind began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Aug 2018
Device: Kindle 8th Gen
Hindu Paper

Same issue, only 1 of the 3 part getting downloaded and removing 25th line in code is not helping.
https://www.thehindu.com/opinion/op-...le24646941.ece
Only 'YES' part gets downloaded leaving 'NO', and 'it's complicated'.
Attached Thumbnails
Click image for larger version

Name:	Untitled   7777.png
Views:	26
Size:	46.9 KB
ID:	165509  
0Arvind is offline   Reply With Quote
Old 08-11-2018, 02:44 AM   #11
0Arvind
Junior Member
0Arvind began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Aug 2018
Device: Kindle 8th Gen
Hindu Paper recipe modification.

Quote:
Originally Posted by kovidgoyal View Post
Nope, no modifications.
Hello Kovid,
I'm really grateful to you for making such wonderful software which makes reading awesome, and turning e-reading devices cost effective.

Issue:-
https://www.thehindu.com/opinion/op-...le24646941.ece. Only yes part is being downloaded, leaving No and it's complicated. Such kind of Article comes every week on Friday, in "The Hindu" Newspaper.

I'm not from IT background, but I tried removing the the code, ignore_duplicate_articles = {'title', 'url'}
It isn't working for me as well, I guess I'm missing something out there.

I have read your articles that you work 80 hours a week and you must be very busy, but if you can take out time to upload the recipe here with modifications, That would be helpful, or tweak the inbuilt recipe.



0Arvind is offline   Reply With Quote
Old 08-11-2018, 09:14 PM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,727
Karma: 10215946
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I'm afraid I cannot fix the recipe for an issue I cannot reproduce, sorry.
kovidgoyal is offline   Reply With Quote
Old 10-06-2018, 12:11 AM   #13
Subhash
Junior Member
Subhash began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Oct 2018
Device: Kindle 8th gen
What to change in the recipe of " The Hindu" to remove image ?
Subhash is offline   Reply With Quote
Reply

Tags
calibre 3.17

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
The Hindu Recipe(Better than the default provided with calibre) sexymax15 Recipes 7 04-26-2017 12:43 PM
"The Hindu" Fetch News Builtin Recipe Not Working gids Recipes 1 01-13-2017 08:09 AM
How to manually set the language for each source of the news recipe? mendesitba Recipes 6 04-02-2015 07:42 PM
NME recipe URL correction scissors Recipes 0 05-17-2013 11:41 AM
the hindu recipe Dr. Ankala Mulle Recipes 0 04-24-2013 04:29 PM


All times are GMT -4. The time now is 05:36 AM.


MobileRead.com is a privately owned, operated and funded community.