customizing recipes

NSILMike · 10-30-2021, 10:15 AM

Is there a way to customize a recipe (in my case The Boston Globe) so that it only downloads articles from that day? The std recipe is downloading articles from up to a few days earlier, so I see the same thing I saw yesterday.

kovidgoyal · 10-30-2021, 11:06 AM

You would need to modify parse_section to check the date and ignore tool old articles.

NSILMike · 10-30-2021, 11:50 AM

I found this but I don't see anything that looks like the date...
def parse_index(self):
feeds = []
for sec in 'metro sports nation world business opinion lifestyle arts'.split():
articles = list(parse_section(self.index_to_soup(absolutize_u rl('/' + sec), raw=True).decode('utf-8')))
if articles:
self.log(sec.capitalize())
self.log(pprint.pformat(articles))
feeds.append((sec.capitalize(), articles))
if self.test:
del articles[self.test[1]:]
if len(feeds) >= self.test[0]:
break

kovidgoyal · 10-30-2021, 12:25 PM

parse_section not parse_index and there will be no date there already, you will have to search for it in the dtructure parse_section is parsing.

NSILMike · 10-30-2021, 12:37 PM

Quote:

Originally Posted by kovidgoyal

parse_section not parse_index and there will be no date there already, you will have to search for it in the dtructure parse_section is parsing.

I found this:
def parse_section(raw_html):
data = extract_json(raw_html)['content-feed']

def text(e):
if not e:
return ''
return e.get('basic') or e.get('native', '')

but I have no idea how to modify per your suggestion.

kovidgoyal · 10-31-2021, 02:27 AM

https://github.com/kovidgoyal/calibr...832ed5f3bebc7b

NSILMike · 10-31-2021, 07:40 AM

Quote:

Originally Posted by kovidgoyal

https://github.com/kovidgoyal/calibr...832ed5f3bebc7b

That fails with:
File "<string>", line 94, in parse_section
NameError: name 'now' is not defined

kovidgoyal · 10-31-2021, 09:47 AM

Works fine for me. And there is no now at line 94 of the builting recipe.

NSILMike · 10-31-2021, 10:02 AM

Quote:

Originally Posted by kovidgoyal

Works fine for me. And there is no now at line 94 of the builting recipe.

well, it still fails for me...

NSILMike · 11-14-2021, 09:10 AM

Why does it still download old news when the std recipe has this command:
oldest_article = 1 # days, includes articles that were published no more than the specified number of days ago

NSILMike · 12-30-2021, 02:31 PM

Quote:

Originally Posted by NSILMike

Why does it still download old news when the std recipe has this command:
oldest_article = 1 # days, includes articles that were published no more than the specified number of days ago

Still waiting for an answer....

NSILMike · 02-04-2022, 09:49 AM

Quote:

Originally Posted by NSILMike

Still waiting for an answer....

and still waiting...

10-30-2021, 10:15 AM	#1
NSILMike Guru Posts: 735 Karma: 35936 Join Date: Apr 2011 Location: Shrewsury, MA Device: Lenovo Android Tablet	customizing recipes Is there a way to customize a recipe (in my case The Boston Globe) so that it only downloads articles from that day? The std recipe is downloading articles from up to a few days earlier, so I see the same thing I saw yesterday.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Customizing the library	centuryx476	Library Management	3	05-09-2013 09:31 PM
Free (iTunes) Tasting Chef's Recipes: Summer 2012 Cookbook [Enhanced Recipes]	ATDrake	Deals and Resources (No Self-Promotion or Affiliate Links)	3	08-15-2012 11:15 PM
Asian Recipes - 50 Tasty & Easy Unique Exotic Recipes (With Images Of Each Dish And C	asiafoodguru	Self-Promotions by Authors and Publishers	1	08-10-2012 05:01 AM
Customizing Calibre	lunixer	Calibre	5	08-23-2010 01:21 PM
customizing built in recipes	jrasmussen	Calibre	2	02-15-2010 11:26 PM

10-30-2021, 11:06 AM	#2
kovidgoyal creator of calibre Posts: 45,343 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You would need to modify parse_section to check the date and ignore tool old articles.

10-30-2021, 11:50 AM	#3
NSILMike Guru Posts: 735 Karma: 35936 Join Date: Apr 2011 Location: Shrewsury, MA Device: Lenovo Android Tablet	I found this but I don't see anything that looks like the date... def parse_index(self): feeds = [] for sec in 'metro sports nation world business opinion lifestyle arts'.split(): articles = list(parse_section(self.index_to_soup(absolutize_u rl('/' + sec), raw=True).decode('utf-8'))) if articles: self.log(sec.capitalize()) self.log(pprint.pformat(articles)) feeds.append((sec.capitalize(), articles)) if self.test: del articles[self.test[1]:] if len(feeds) >= self.test[0]: break

10-30-2021, 12:25 PM	#4
kovidgoyal creator of calibre Posts: 45,343 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	parse_section not parse_index and there will be no date there already, you will have to search for it in the dtructure parse_section is parsing.

10-31-2021, 02:27 AM	#6
kovidgoyal creator of calibre Posts: 45,343 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	https://github.com/kovidgoyal/calibr...832ed5f3bebc7b

10-31-2021, 09:47 AM	#8
kovidgoyal creator of calibre Posts: 45,343 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Works fine for me. And there is no now at line 94 of the builting recipe.

11-14-2021, 09:10 AM	#10
NSILMike Guru Posts: 735 Karma: 35936 Join Date: Apr 2011 Location: Shrewsury, MA Device: Lenovo Android Tablet	Why does it still download old news when the std recipe has this command: oldest_article = 1 # days, includes articles that were published no more than the specified number of days ago

Advert

Advert