Le Devoir update possible?

franzca · 12-18-2012, 02:38 PM

Hello,

Le Devoir (Canadian) changed something on its website last week, and since then the previously flawless recipe collects only headlines and summaries. Anybody have a quick fix? I wish I knew something about Python to be able to help...

Thanks!

Krittika Goyal · 12-19-2012, 03:40 AM

here is a modified recipe. i think it should work. But i do not hjave a username and password so i could not test it. see if it works. IF it does not then pm me a user name and password and I will try again

Krittika Goyal · 12-20-2012, 03:51 AM

try this now. turns out i did not need the login deatails and neither should you. see if it is complete

franzca · 12-20-2012, 07:02 AM

On the site of Le Devoir, about half the articles are freely available, whereas the rest you only get the first two paragraphs, followed by a message that the rest of the article is reserved for subscribers.

Can you add the login back in?

Thanks.

Krittika Goyal · 12-20-2012, 10:21 AM

i put the login back but i am not sure that will fix it. can you check and let nme know?

franzca · 12-20-2012, 10:47 AM

I get the error message

'NoneType' object has no attribute 'Title'

when trying to import the modified recipe, so can't test.

franzca · 12-20-2012, 10:54 AM

Sorry, it seems I created the last problem myself by attempting to save the modified recipe a second time.

The latest version does work perfectly.

Thanks!

odaigle · 03-07-2013, 10:14 AM

Hi,
I have updated once more the built-in recipe of Le Devoir. I sent it to Kovid for its inclusion in the next release of Caliber. Here it is.

I've tested it with and without an account. The previous version was not downloading some articles as they don't have all the same tags.

Regards,

Olivier

odaigle · 03-10-2013, 02:10 PM

The newest version of Calibre (0.9.22) changed something in the API of the fetch news scripts. Here is an updated version of the recipe for Le Devoir. Calibre 0.9.22 seems to create larger files than before. I am checking into it to get smaller files (it jumped from 5 Mb to 20 Mb for a Saturday edition).

kovidgoyal · 03-10-2013, 02:42 PM

The API change you are referring to was in 0.9.16. While I cant say for sure I'd guess your size increase is related to images, earlier calibre used to force convert all downloaded images to jpeg, it no longer does this, preserving the original image from the website.

odaigle · 03-10-2013, 03:09 PM

I was using 0.9.10 before updating to 0.9.22, hence the bug I got. Regarding file size, they are jpg on the website, but perhaps higher quality. Is there an easy way of reducing the resolution of images ?

kovidgoyal · 03-10-2013, 11:10 PM

calibre's conversion pipeline will automatically resize images to the screen size of the output profile you have selected. If you want to do the resizing in the recipe itself, you can implement image_url_processor, download the image yourself, resize it and then returh the url to the downloaded image.

odaigle · 03-17-2013, 08:57 PM

I've updated the recipe to save images in jpg at 30% quality. It seems that the images on the site are at 100% quality. This allowed the file for a Saturday edition to go from 26 Mb to a more manageable 3.4 Mb.... The recipe can be downloaded at http://www.nuvucameras.com/download/...ir_1007.recipe . Kovid, can you apply the new recipe to the next version of Calibre ? Thanks.

kovidgoyal · 03-18-2013, 01:59 AM

@odaigle: You should probably wait a bit and use the new image downsampling code as discussed here: https://www.mobileread.com/forums/sho...d.php?t=208427

odaigle · 03-23-2013, 02:59 PM

Works great! Modified the recipe. It can be fetched at http://www.nuvucameras.com/download/...ir_1008.recipe .

Olivier

12-18-2012, 02:38 PM	#1
franzca Junior Member Posts: 4 Karma: 10 Join Date: Dec 2012 Device: Sony	Le Devoir update possible? Hello, Le Devoir (Canadian) changed something on its website last week, and since then the previously flawless recipe collects only headlines and summaries. Anybody have a quick fix? I wish I knew something about Python to be able to help... Thanks!

12-20-2012, 07:02 AM	#4
franzca Junior Member Posts: 4 Karma: 10 Join Date: Dec 2012 Device: Sony	Sorry, Le Devoir _does_ need login On the site of Le Devoir, about half the articles are freely available, whereas the rest you only get the first two paragraphs, followed by a message that the rest of the article is reserved for subscribers. Can you add the login back in? Thanks.

12-20-2012, 10:47 AM	#6
franzca Junior Member Posts: 4 Karma: 10 Join Date: Dec 2012 Device: Sony	error trying to import recipe I get the error message 'NoneType' object has no attribute 'Title' when trying to import the modified recipe, so can't test.

12-20-2012, 10:54 AM	#7
franzca Junior Member Posts: 4 Karma: 10 Join Date: Dec 2012 Device: Sony	Correction -- it does work Sorry, it seems I created the last problem myself by attempting to save the modified recipe a second time. The latest version does work perfectly. Thanks!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
won't update...after completing the update cycle edge returns to the first screen	WeAreBorrg	enTourage Archive	2	03-18-2011 06:39 PM
Wrong rendering of captions in "Le Devoir" news feed	PoP	Recipes	2	01-29-2011 02:29 PM
How can I update creation time for a title when I update the book info or convert it	setherd	Calibre	3	10-30-2010 01:51 PM
Firmware Update Can I reinstall the official amazon update after the update file from this site ?	yuenslhk	Amazon Kindle	2	06-20-2010 11:24 AM
PRS-500 What file to change to re-update reader with latest update?	pitolee	Sony Reader Dev Corner	2	05-24-2007 11:56 PM

03-10-2013, 02:42 PM	#10
kovidgoyal creator of calibre Posts: 43,858 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	The API change you are referring to was in 0.9.16. While I cant say for sure I'd guess your size increase is related to images, earlier calibre used to force convert all downloaded images to jpeg, it no longer does this, preserving the original image from the website.

03-10-2013, 03:09 PM	#11
odaigle Junior Member Posts: 5 Karma: 10 Join Date: Feb 2012 Device: Kindle keyboard 3G	I was using 0.9.10 before updating to 0.9.22, hence the bug I got. Regarding file size, they are jpg on the website, but perhaps higher quality. Is there an easy way of reducing the resolution of images ?

03-10-2013, 11:10 PM	#12
kovidgoyal creator of calibre Posts: 43,858 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	calibre's conversion pipeline will automatically resize images to the screen size of the output profile you have selected. If you want to do the resizing in the recipe itself, you can implement image_url_processor, download the image yourself, resize it and then returh the url to the downloaded image.

03-17-2013, 08:57 PM	#13
odaigle Junior Member Posts: 5 Karma: 10 Join Date: Feb 2012 Device: Kindle keyboard 3G	I've updated the recipe to save images in jpg at 30% quality. It seems that the images on the site are at 100% quality. This allowed the file for a Saturday edition to go from 26 Mb to a more manageable 3.4 Mb.... The recipe can be downloaded at http://www.nuvucameras.com/download/...ir_1007.recipe . Kovid, can you apply the new recipe to the next version of Calibre ? Thanks.

03-18-2013, 01:59 AM	#14
kovidgoyal creator of calibre Posts: 43,858 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	@odaigle: You should probably wait a bit and use the new image downsampling code as discussed here: https://www.mobileread.com/forums/sho...d.php?t=208427

03-23-2013, 02:59 PM	#15
odaigle Junior Member Posts: 5 Karma: 10 Join Date: Feb 2012 Device: Kindle keyboard 3G	Works great! Modified the recipe. It can be fetched at http://www.nuvucameras.com/download/...ir_1008.recipe . Olivier