MobileRead Forums - View Single Post

Alexander Turcic · 09-10-2004, 05:57 AM

aranpura,

Great work. I am happy you discovered the wonderful art of site scraping

Can be a lot of fun. One suggestion: The Nation also offers a print version of each article. So when you parse the main content page, simply replace all article links with the print version.

E.g.
http://www.thenation.com/doc.mhtml?i=20040927&s=legum
becomes
http://www.thenation.com/docprint.mh...040927&s=legum

(a small regex replacing /doc\.mhtml with /docprint\.mhtml should be sufficient.)

Anyways, keep up the great work!

09-10-2004, 05:57 AM	#2
Alexander Turcic Fully Converged Posts: 18,175 Karma: 14021202 Join Date: Oct 2002 Location: Switzerland Device: Too many to count here.	aranpura, Great work. I am happy you discovered the wonderful art of site scraping Can be a lot of fun. One suggestion: The Nation also offers a print version of each article. So when you parse the main content page, simply replace all article links with the print version. E.g. http://www.thenation.com/doc.mhtml?i=20040927&s=legum becomes http://www.thenation.com/docprint.mh...040927&s=legum (a small regex replacing /doc\.mhtml with /docprint\.mhtml should be sufficient.) Anyways, keep up the great work!