Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 11-08-2011, 04:57 PM   #1
fw190a8
Junior Member
fw190a8 began at the beginning.
 
Posts: 7
Karma: 10
Join Date: Aug 2011
Device: Kindle 3
Lightbulb Request for generic 'multiple pages' code

Being a programmer but a bit new to python and the world of feed parsing, could anybody help out with feeds that lead to stories with multiple pages? I am looking at the PC Gamer site first of all, to give you some context, and this post in particular is an example of one that has multiple pages.

Not all posts have multiple pages, and those that do, simply have 'next' buttons, rather than a full pager, so it's impossible to say from the first page how many pages are in a given article. It would be a case of saying: while (there's a 'next' button on the page) keep appending the page content and getting the next page - unfortunately my python's not good enough.

I'd like to ask for some help for a generic solution so I can adapt this for other purposes. I'll post any recipes I create using this, in case others are interested. Thanks in advance!
fw190a8 is offline   Reply With Quote
Old 11-08-2011, 09:35 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,908
Karma: 5035037
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You use match_regexps to match only the next page link and set recursions to some large number, say 10.
kovidgoyal is offline   Reply With Quote
Old 11-09-2011, 09:22 AM   #3
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by fw190a8 View Post
Being a programmer but a bit new to python and the world of feed parsing, could anybody help out with feeds that lead to stories with multiple pages? I am looking at the PC Gamer site first of all, to give you some context, and this post in particular is an example of one that has multiple pages.

Not all posts have multiple pages, and those that do, simply have 'next' buttons, rather than a full pager, so it's impossible to say from the first page how many pages are in a given article. It would be a case of saying: while (there's a 'next' button on the page) keep appending the page content and getting the next page - unfortunately my python's not good enough.

I'd like to ask for some help for a generic solution so I can adapt this for other purposes. I'll post any recipes I create using this, in case others are interested. Thanks in advance!
It's been written many times. Kovid's explained how it works. See the Adventure Gamers recipe that's builtin for a sample recipe. Search for "multipage" in this forum for comments on it and other examples.

Also, see here.

Last edited by Starson17; 11-09-2011 at 10:03 AM.
Starson17 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Sidebars Across Multiple Pages Rand Brittain Sigil 0 01-08-2011 12:11 AM
Skipping multiple pages on Kindle DX brianpc Amazon Kindle 8 12-13-2010 12:13 AM
Multiple Cover Pages Desertway Calibre 0 09-12-2010 03:16 PM
Creating eBooks from multiple web pages tichkiir Workshop 1 10-08-2007 06:16 PM


All times are GMT -4. The time now is 03:40 PM.


MobileRead.com is a privately owned, operated and funded community.