Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 11-02-2010, 09:34 AM   #1
janvanmaar
Addict
janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.
 
Posts: 215
Karma: 404
Join Date: Nov 2010
Device: Kindle 3G, Samsung SIII
mechanize.py error when logging in to subscribed articles

I try to use the following plugin to fetch subscribed articles:
Code:
from calibre.web.feeds.news import BasicNewsRecipe

class TyzdenRecipe(BasicNewsRecipe):
    title          = u'.tyzden'
    oldest_article = 7
    max_articles_per_feed = 100
    needs_subscription    = True
    language = 'sk'

    feeds          = [(u'.tyzden - clanky z tlacenych vydani', u'http://www.tyzden.sk/rss/clanky-z-tlacenych-vydani.xml')]

    LOGIN = 'http://www.tyzden.sk/prihlasenie.html'

    def get_browser(self):
        br = BasicNewsRecipe.get_browser()
        if self.username is not None and self.password is not None:
            br.open(self.LOGIN)
            br.select_form(nr=1)
            br['user'] = self.username
            br['pass'] = self.password
            br.submit()
        return br
It gets to the right form and finds the right fields but then crashes when trying to submit the form:
Quote:
InputFormatPlugin: Recipe Input running
Traceback (most recent call last):
File "/tmp/init.py", line 48, in <module>
File "/home/kovid/build/calibre/src/calibre/utils/ipc/worker.py", line 107, in main
File "/home/kovid/build/calibre/src/calibre/gui2/convert/gui_conversion.py", line 24, in gui_convert
File "/home/kovid/build/calibre/src/calibre/ebooks/conversion/plumber.py", line 836, in run
File "/home/kovid/build/calibre/src/calibre/customize/conversion.py", line 216, in __call__
File "/home/kovid/build/calibre/src/calibre/web/feeds/input.py", line 101, in convert
File "/home/kovid/build/calibre/src/calibre/web/feeds/news.py", line 616, in __init__
File "/tmp/calibre_0.7.26_tmp_bsIqcm/calibre_0.7.26_u79lhp_recipes/recipe0.py", line 29, in get_browser
br.submit()
File "/usr/lib64/python2.6/site-packages/mechanize/_mechanize.py", line 547, in submit
File "/usr/lib64/python2.6/site-packages/mechanize/_mechanize.py", line 209, in open
File "/usr/lib64/python2.6/site-packages/mechanize/_mechanize.py", line 261, in _mech_open
mechanize._response.httperror_seek_wrapper: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Found
I don't understand why this is happening... Any idea?
I should mention that login works fine when I fill the form in a browser. I double checked that correct username and password are inputted in the scheduler also...
janvanmaar is offline   Reply With Quote
Old 11-02-2010, 11:47 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,834
Karma: 4370273
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
that means mechanize thinks there is an infinite redirect loop, which typically means you aren't filling in the form correctly. There may be hidden fields that need to be filled in. Use Tamper data in firefox to see what they might be.
kovidgoyal is offline   Reply With Quote
Old 11-02-2010, 12:51 PM   #3
janvanmaar
Addict
janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.
 
Posts: 215
Karma: 404
Join Date: Nov 2010
Device: Kindle 3G, Samsung SIII
Thanks for the Tamper suggestion!
It did not reveal any required hidden fields. But it showed that after the form has been submitted, the browser asked for logout and then for login again with a cookie... and when I disabled cookies then login did not work.
I guess this is very difficult to do in Calibre/python, isn't it... but at least I know where the problem is.
janvanmaar is offline   Reply With Quote
Old 11-02-2010, 01:02 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,834
Karma: 4370273
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
mechanize handles cookies just fine. You should set handle refresh to false on the mechanize browser and simulate the logout/login steps manually with br.open
kovidgoyal is offline   Reply With Quote
Old 11-02-2010, 01:33 PM   #5
janvanmaar
Addict
janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.
 
Posts: 215
Karma: 404
Join Date: Nov 2010
Device: Kindle 3G, Samsung SIII
Hm, I guess I should have said it is difficult for me as somebody without much knowledge of HTTP protocol.
Anyway, whatever I do, I get the same problem when submitting the form. Setting handle refresh to false does not change that. And I cannot do other br.open as it crashes before...
janvanmaar is offline   Reply With Quote
Old 11-02-2010, 01:41 PM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,834
Karma: 4370273
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
use a try catch to catch the exception and ignore it.
kovidgoyal is offline   Reply With Quote
Old 11-02-2010, 02:04 PM   #7
janvanmaar
Addict
janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.
 
Posts: 215
Karma: 404
Join Date: Nov 2010
Device: Kindle 3G, Samsung SIII
well, that enabled me to pass through the submit (which does not mean that registration was successful of course), however, I got a segfault when creating the ebook then:
calibre-parallel: line 11: 17597 Segmentation fault $loader "$@"
EDIT: I am using binaries - perhaps I should try to compile...

Last edited by janvanmaar; 11-02-2010 at 02:08 PM.
janvanmaar is offline   Reply With Quote
Old 11-02-2010, 03:45 PM   #8
janvanmaar
Addict
janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.
 
Posts: 215
Karma: 404
Join Date: Nov 2010
Device: Kindle 3G, Samsung SIII
Guess what: it worked! I would not say that ignoring the error could be working but it is. Thanks for all your help!
janvanmaar is offline   Reply With Quote
Old 11-02-2010, 03:46 PM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 24,834
Karma: 4370273
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
the segfault can be ignored, it happens during interpreter shutdown and is an artifact of the process of creating the binaries that I have never been motivated enough to track down and fix.
kovidgoyal is offline   Reply With Quote
Old 11-02-2010, 04:10 PM   #10
janvanmaar
Addict
janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.janvanmaar has a complete set of Star Wars action figures.
 
Posts: 215
Karma: 404
Join Date: Nov 2010
Device: Kindle 3G, Samsung SIII
Takehome message from this thread: ignore errors and conquer.
janvanmaar is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Search within subscribed threads! thinkpad Feedback 3 08-18-2010 09:44 AM
I don't get email notifications of updates to threads I'm subscribed to Ea Feedback 4 03-26-2010 02:20 PM
calibre-0.6.31, mechanize and lxml taurnil Calibre 5 01-01-2010 07:47 AM
Views subscribed threads pilotbob Feedback 6 02-28-2009 02:49 PM
'mechanize' module? itimpi Calibre 2 01-22-2009 03:42 PM


All times are GMT -4. The time now is 11:04 AM.


MobileRead.com is a privately owned, operated and funded community.