Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 01-28-2017, 11:29 AM   #1
keyi.fang
Junior Member
keyi.fang began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jan 2017
Device: Kindle PaperWhite 3.0
Recipe for Science Magazine

Hi All,

Can someone help me setup the recipe for Science Magazine?
I am trying to download the RSS Feeds from 'http://science.sciencemag.org/rss/twis.xml'
but encounter problems logging in using the following code.

Spoiler:


Following is the code I mm working on:

#!/usr/bin/env python2

__license__ = 'GPL v3'
__copyright__ = '2008, Darko Miletic <darko.miletic at gmail.com>'
'''
sciencemag.org
'''
from calibre.web.feeds.news import BasicNewsRecipe


class ScienceAAS(BasicNewsRecipe):
title = u'Science AAAS - Work In Progress'
__author__ = u'Darko Miletic'
language = 'en'

description = u'The best in science news, commentary, and research'
oldest_article = 7
max_articles_per_feed = 100
no_stylesheets = True
use_embedded_content = False
simultaneous_downloads = 1
delay = 1
timefmt = ' [%A, %d %B, %Y]'
needs_subscription = True

def get_browser(self):
br = BasicNewsRecipe.get_browser(self)
if self.username is not None and self.password is not None:
br.open('https://signin.aaas.org/oxauth/login')
br.select_form('loginForm')
br['username'] = self.username
br['password'] = self.password
br.submit()
return br

keep_only_tags = [dict(name='div', attrs={'id': 'content-block'})]

feeds = [('Research highlights from the current issue of Science Magazine',
'http://science.sciencemag.org/rss/twis.xml')]

and the Following is the error returned:

Fetch news from Science AAAS - Work In Progress
Resolved conversion options
calibre version: 2.78.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x0455BE50>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindlePaperWhiteOutp ut object at 0x04561230>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Using custom recipe
Python function terminated unexpectedly
no form matching name 'loginForm' (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 26, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1065, in run
File "site-packages\calibre\customize\conversion.py", line 245, in __call__
File "site-packages\calibre\ebooks\conversion\plugins\recipe_ input.py", line 118, in convert
File "site-packages\calibre\web\feeds\news.py", line 887, in __init__
File "<string>", line 30, in get_browser
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize\_mechanize.py", line 524, in select_form
mechanize._mechanize.FormNotFoundError: no form matching name 'loginForm'



Last edited by Dr. Drib; 02-17-2017 at 09:55 AM.
keyi.fang is offline   Reply With Quote
Old 01-28-2017, 10:36 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,408
Karma: 10205094
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Use select_form(nr=0) (might need to change 0 to 1 or 2 to get the right form.
kovidgoyal is offline   Reply With Quote
Advert
Old 01-30-2017, 02:19 AM   #3
keyi.fang
Junior Member
keyi.fang began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jan 2017
Device: Kindle PaperWhite 3.0
i tired nr=0, 1 and 2 all returning the following error msg:

Spoiler:


Fetch news from Science AAAS - Work In Progress
Resolved conversion options
calibre version: 2.78.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x044AAE70>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindlePaperWhiteOutp ut object at 0x044B0250>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Using custom recipe
Python function terminated unexpectedly
no form matching nr 2 (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 26, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1065, in run
File "site-packages\calibre\customize\conversion.py", line 245, in __call__
File "site-packages\calibre\ebooks\conversion\plugins\recipe_ input.py", line 118, in convert
File "site-packages\calibre\web\feeds\news.py", line 887, in __init__
File "<string>", line 30, in get_browser
File "site-packages\mechanize-0.2.5-py2.7.egg\mechanize\_mechanize.py", line 524, in select_form
mechanize._mechanize.FormNotFoundError: no form matching nr 2


Last edited by Dr. Drib; 02-17-2017 at 09:56 AM.
keyi.fang is offline   Reply With Quote
Old 01-30-2017, 03:46 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,408
Karma: 10205094
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
They cannot all be returning that error, since that error refers to nr=2. If you are getting a form not found error with all values of nr, it means there is no form on the page.
kovidgoyal is offline   Reply With Quote
Old 02-03-2017, 01:17 AM   #5
keyi.fang
Junior Member
keyi.fang began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jan 2017
Device: Kindle PaperWhite 3.0
yes. I have tired all values of nr.

Studying the website using the development tools there is a form titled 'loginForm' in 'https://signin.aaas.org/oxauth/login'

How to specify the form?

[Image violates Posting Guidelines for size - MODERATOR]

Last edited by Dr. Drib; 02-17-2017 at 09:57 AM.
keyi.fang is offline   Reply With Quote
Advert
Old 02-03-2017, 01:24 AM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 33,408
Karma: 10205094
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
What you see in a browser and what you get in mechanize can be two very different things. Either save the html mechanize gets and check that, or just manually send the post request that the form would generate, which you can get by using the dev tools in your browser to intercept the POST request.
kovidgoyal is offline   Reply With Quote
Old 02-10-2017, 11:39 AM   #7
keyi.fang
Junior Member
keyi.fang began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jan 2017
Device: Kindle PaperWhite 3.0
Thanks for your reply.

hmm..Let me explore your suggestions abit.
keyi.fang is offline   Reply With Quote
Old 02-17-2017, 09:57 AM   #8
Dr. Drib
Grand Sorcerer
Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.Dr. Drib ought to be getting tired of karma fortunes by now.
 
Dr. Drib's Avatar
 
Posts: 41,382
Karma: 29182977
Join Date: Jan 2007
Location: San Borja (Lima), Peru
Device: Kindle: Oasis 3G, Voyage WiFi;FireHD; Kobo: Aura One, Glo HD
Moderator Notice

Please take a few moments to read our Posting Guidelines, in particular the part about the maximum image size allowed (600x600 pixels). Images that violate those guidelines may be deleted.

It is every member's responsibility to read our Posting Guidelines and follow them.

Thanks.


[COLOR="Green"]
Dr. Drib is offline   Reply With Quote
Reply

Tags
aaas, form, login, recipe, science magazine

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Time Magazine recipe NSILMike Recipes 6 07-04-2016 05:09 AM
Fantasy & Science Fiction Magazine - Free Sampler for Kindle rogerVA Deals and Resources (No Self-Promotion or Affiliate Links) 21 08-04-2011 05:43 PM
Recipe for Self Magazine (US) (need help) xXxXxXxXxXx Recipes 1 05-17-2011 05:16 PM
SPIN Magazine recipe Quistopher Recipes 0 01-27-2011 09:04 PM
Help me find story from Isaac Asimov's Science Fiction Magazine Renaldo Reading Recommendations 6 12-06-2009 11:55 AM


All times are GMT -4. The time now is 11:49 PM.


MobileRead.com is a privately owned, operated and funded community.