South China Morning Post (SCMP) recipe only headers but no articles

hongho71 · 09-26-2022, 10:01 AM

The South China Morning Post (SCMP) recipe only download headers but no articles. Anybody have a fix for this? The recipe doesn't ask for login information but noticed that it had one in past recipes. Thanks!

unkn0wn · 09-27-2022, 02:39 AM

don't know why self.tag_to_string works and not .text and also datetime formatting isn't working.

kovidgoyal · 09-27-2022, 02:46 AM

SCMP was already fixed here: https://github.com/kovidgoyal/calibr...cff72aaca2ea03 are your fixes in addition to that?

unkn0wn · 09-27-2022, 02:56 AM

oh 3 hours ago.. didn't check.

he used .contents instead.

I also added a cover url.. but wouldn't matter.

unkn0wn · 09-27-2022, 03:05 AM

but recipe still fails.. line 171
datetime formatting doesn't work in html_output

File "<string>", line 171, in preprocess_raw_html
ValueError: Invalid format string

I sort of.. fixed this.

kovidgoyal · 09-27-2022, 03:18 AM

I'm not getting a failure for the recipe, but I added a tyr/except block to ignore failures to format the datetime. And I added your cover code.

unkn0wn · 09-27-2022, 03:38 AM

not recipe failure but it skips articles by saying failed to download article.

Quote:

Fetched https://www.scmp.com/economy/china-e...ource=rss_feed in 1.469000 seconds
Could not fetch link https://www.scmp.com/economy/china-e...ource=rss_feed
Traceback (most recent call last):
File "calibre\web\fetch\simple.py", line 546, in process_links
File "calibre\web\fetch\simple.py", line 194, in get_soup
File "calibre\web\feeds\news.py", line 625, in preprocess_raw_html_
File "<string>", line 171, in preprocess_raw_html
ValueError: Invalid format string

anyhow its fixed now.

09-26-2022, 10:01 AM	#1
hongho71 Connoisseur Posts: 55 Karma: 16 Join Date: Feb 2015 Device: Kindle Paperwhite	South China Morning Post (SCMP) recipe only headers but no articles The South China Morning Post (SCMP) recipe only download headers but no articles. Anybody have a fix for this? The recipe doesn't ask for login information but noticed that it had one in past recipes. Thanks!

09-27-2022, 03:05 AM	#5
unkn0wn Guru Posts: 616 Karma: 85520 Join Date: May 2021 Device: kindle	but recipe still fails.. line 171 datetime formatting doesn't work in html_output File "<string>", line 171, in preprocess_raw_html ValueError: Invalid format string I sort of.. fixed this. Last edited by unkn0wn; 09-27-2022 at 03:07 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
South China Morning Post recipe not working	cupcakeg2	Recipes	0	08-11-2015 12:00 PM
build-in scmp recipe didn't download images	timoy_kindle	Recipes	1	09-27-2014 07:40 AM
South China Morning Post recipe broke?	mwerth1	Recipes	1	09-05-2012 12:04 AM
South China Morning Post (SCMP) - Hong Kong - Fixed	llam	Recipes	0	07-02-2011 10:48 PM
Recipe Request - South China Morning Post	mobilewilier	Calibre	1	05-02-2010 11:42 PM

09-27-2022, 02:46 AM	#3
kovidgoyal creator of calibre Posts: 45,347 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	SCMP was already fixed here: https://github.com/kovidgoyal/calibr...cff72aaca2ea03 are your fixes in addition to that?

09-27-2022, 02:56 AM	#4
unkn0wn Guru Posts: 616 Karma: 85520 Join Date: May 2021 Device: kindle	oh 3 hours ago.. didn't check. he used .contents instead. I also added a cover url.. but wouldn't matter.

09-27-2022, 03:18 AM	#6
kovidgoyal creator of calibre Posts: 45,347 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I'm not getting a failure for the recipe, but I added a tyr/except block to ignore failures to format the datetime. And I added your cover code.

Advert

Advert