Skeptical Enquirer - Error fetching

[d.r.l]

Dear team,

I'm trying to setup calibre to download the Skeptical Enquirer magazines I'm subscribed to.
I've setup the schedule for download, first day of the month, never delete older news and keep at most: all issues.

Then, hitting download now or ok does not prompt for my logon information.
First question: how can I add my logon information?

I have the following error message:

calibre, version 5.13.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Skeptical Inquirer

Fetch news from The Skeptical Inquirer
Conversion options changed from defaults:
  verbose: 2
  output_profile: 'generic_eink'
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x000002963C786A00>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.GenericEink object at 0x000002963C786D00>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:skeptical_enquirer
Trying to get latest version of recipe: skeptical_enquirer
Using user agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 216, in main
  File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
  File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1105, in run
  File "calibre\customize\conversion.py", line 245, in __call__
  File "calibre\ebooks\conversion\plugins\recipe_input.py", line 139, in convert
  File "calibre\web\feeds\news.py", line 1051, in download
  File "calibre\web\feeds\news.py", line 1231, in build_index
ValueError: No articles found, aborting

Any idea?

Thanks,

[d.r.l]

So, I kept digging and realised the recipe was a tad outdated and missed the code to log in as a browser before downloading the issues.

I have an error in the customized recipe I tried to come up with.

Here's my recipe as it stands

from calibre.web.feeds.news import BasicNewsRecipe
import re

def get_browser(self):
    br = BasicNewsRecipe.get_browser(self)
    if self.username is not None and self.password is not None:
        br.open('https://skepticalinquirer.org/member-login/')
        br.select_form(name='login')
        br['HIDDEN']   = self.username
        br['HIDDEN'] = self.password
        br.submit()
    return br


class TheSkepticalInquirer(BasicNewsRecipe):
    title = u'The Skeptical Inquirer'
    description = 'Investigation of fringe science and paranormal claims.'
    language = 'en'
    __author__ = 'Starson17'
    oldest_article = 31
    cover_url = 'https://skepticalinquirer.org/wp-content/uploads/sites/29/2019/03/SI-logo-tag-line-w.png'
    remove_empty_feeds = True
    remove_javascript = True
    max_articles_per_feed = 50
    no_stylesheets = True

    keep_only_tags = [dict(name='div', attrs={'id': ['content', 'bio']})]

    remove_tags = [
        dict(name='div', attrs={'id': ['socialMedia']}),
    ]

    preprocess_regexps = [
        (re.compile(r'\.\(JavaScript must be enabled to view this email address\)',
                    re.DOTALL | re.IGNORECASE), lambda match: ''),
    ]

    def parse_index(self):
        feeds = []
        for title, url in [("The Skeptical Inquirer", "https://skepticalinquirer.org/archive/")]:
            articles = self.make_links(url)
            if articles:
                feeds.append((title, articles))
        return feeds

    def make_links(self, url):
        soup = self.index_to_soup(url)
        title = ''
        current_articles = []
        for item in soup.findAll(attrs={'class': ['article-single bigger']}):
            page_url = url + str(item.a["href"])
            title = str(item.a.string)
            current_articles.append(
                {'title': title, 'url': page_url, 'description': '', 'date': ''})
        return current_articles

    extra_css = '''
                    h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;}
                    h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal;font-size:small;}
                    p{font-family:Arial,Helvetica,sans-serif;font-size:small;}
                    body{font-family:Helvetica,Arial,sans-serif;font-size:small;}
    '''


calibre_most_common_ua = 'Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko'

Here's the result

calibre, version 5.13.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Skeptical Inquirer

Fetch news from The Skeptical Inquirer
Conversion options changed from defaults:
  verbose: 2
  output_profile: 'tablet'
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x000002415DD06A00>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.TabletOutput object at 0x000002415DD2B340>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: custom:1000
Using user agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 216, in main
  File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
  File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1105, in run
  File "calibre\customize\conversion.py", line 245, in __call__
  File "calibre\ebooks\conversion\plugins\recipe_input.py", line 139, in convert
  File "calibre\web\feeds\news.py", line 1051, in download
  File "calibre\web\feeds\news.py", line 1231, in build_index
ValueError: No articles found, aborting

To clarify what I'm trying to achieve is to have Calibre download all the past issues I have access to, which are in PDF format and listed by date in the website with a link pointing to each issue. Here's a snipped of few links
https://skepticalinquirer.org/wp-con...1-proof-v2.pdf
https://skepticalinquirer.org/wp-con...21-reduced.pdf

I'm sure something is wrong with my code, I'll keep on searching for a solution in the meantime someone helps out here (not a complaint, I simply can't help myself )

Originally Posted by d.r.l

Dear team,

I'm trying to setup calibre to download the Skeptical Enquirer magazines I'm subscribed to.
I've setup the schedule for download, first day of the month, never delete older news and keep at most: all issues.

Then, hitting download now or ok does not prompt for my logon information.
First question: how can I add my logon information?

I have the following error message:

calibre, version 5.13.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Skeptical Inquirer

Fetch news from The Skeptical Inquirer
Conversion options changed from defaults:
  verbose: 2
  output_profile: 'generic_eink'
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x000002963C786A00>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.GenericEink object at 0x000002963C786D00>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:skeptical_enquirer
Trying to get latest version of recipe: skeptical_enquirer
Using user agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 216, in main
  File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
  File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1105, in run
  File "calibre\customize\conversion.py", line 245, in __call__
  File "calibre\ebooks\conversion\plugins\recipe_input.py", line 139, in convert
  File "calibre\web\feeds\news.py", line 1051, in download
  File "calibre\web\feeds\news.py", line 1231, in build_index
ValueError: No articles found, aborting

Any idea?

Thanks,

[d.r.l]

Alright, I've used a recipe's code I found on the website and tried to adapt it to fit my needs.

Despite the fact that I do define my username and password, the error message I have states it needs it

import re, zipfile, os
from calibre.ptempfile import PersistentTemporaryDirectory
from calibre.ptempfile import PersistentTemporaryFile
from urllib.parse import urlparse, urlsplit
GET_MOBI=False
GET_PDF=True
GET_AUDIO=False
GET_MAGAZIN=False

class TheSkepticalInquirer(BasicNewsRecipe):
    title = u'The Skeptical Inquirer'
    description = 'Investigation of fringe science and paranormal claims.'
    language = 'en' 
    needs_subscription = True 
    
def get_browser(self):
    br = BasicNewsRecipe.get_browser(self)
    if self.username is not None and self.password is not None:
        br.open('https://skepticalinquirer.org/member-login/')
        br.select_form(name='login')
        br['HIDDEN']   = self.username
        br['HIDDEN'] = self.password
        br.submit()
    return br

def build_index(self):
        browser = self.get_browser()

        # find the links (Adjust to your needs!)
        #epublink = browser.find_link(text_regex=re.compile('.*Download ePub.*'))
        #mobilink = browser.find_link(text_regex=re.compile('.*Download Mobi.*'))
        pdflink = browser.find_link(text_regex=re.compile('https://skepticalinquirer.org/archive/'))

        # Cheat calibre's recipe method, as in post from Starsom17
        self.report_progress(0,_('downloading epub'))
        response = browser.follow_link(epublink)
        dir = PersistentTemporaryDirectory()
        epub_file = PersistentTemporaryFile(suffix='.epub',dir=dir)
        epub_file.write(response.read())
        epub_file.close()
        zfile = zipfile.ZipFile(epub_file.name, 'r')
        self.report_progress(0.1,_('extracting epub'))
        zfile.extractall(self.output_dir)
        epub_file.close()
        index = os.path.join(self.output_dir, 'content.opf')
        self.report_progress(0.2,_('epub downloaded and extracted'))


        #
        # Now, download the remaining files
        #
        if (GET_MOBI):
           self.report_progress(0.3,_('downloading mobi'))
           mobi_file = PersistentTemporaryFile(suffix='.mobi',dir=dir)
           browser.back()
           response = browser.follow_link(mobilink)
           mobi_file.write(response.read())
           mobi_file.close()

        if (GET_PDF):
           self.report_progress(0.4,_('downloading pdf'))
           pdf_file = PersistentTemporaryFile(suffix='.pdf',dir=dir)
           browser.back()
           response = browser.follow_link(pdflink)
           pdf_file.write(response.read())
           pdf_file.close()

        # Get all formats into Calibre's database as one single book entry
        self.report_progress(0.6,_('Adding files to Calibre db'))
        cmd = "calibredb add -1 " + dir
        os.system(cmd)

        return index

The error message is the following:

calibre, version 5.13.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Skeptical Inquirer

Fetch news from The Skeptical Inquirer
Conversion options changed from defaults:
  verbose: 2
  output_profile: 'tablet'
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x000001F176AC6A00>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.TabletOutput object at 0x000001F176AEB340>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:skeptical_enquirer
Trying to get latest version of recipe: skeptical_enquirer
Using user agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 216, in main
  File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
  File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1105, in run
  File "calibre\customize\conversion.py", line 245, in __call__
  File "calibre\ebooks\conversion\plugins\recipe_input.py", line 139, in convert
  File "calibre\web\feeds\news.py", line 1051, in download
  File "calibre\web\feeds\news.py", line 1231, in build_index
ValueError: No articles found, aborting

[d.r.l]

Still modified the code, scheduled the recipe for download, adding the logon and password directly in the GUI as requested.

The code now is:

import re, zipfile, os
from calibre.ptempfile import PersistentTemporaryDirectory
from calibre.ptempfile import PersistentTemporaryFile
from urllib.parse import urlparse, urlsplit
GET_MOBI=False
GET_PDF=True
GET_AUDIO=False
GET_MAGAZIN=False

class TheSkepticalInquirer(BasicNewsRecipe):
    title = u'The Skeptical Inquirer'
    description = 'Investigation of fringe science and paranormal claims.'
    language = 'en' 
    needs_subscription = True 

def build_index(self):
        browser = self.get_browser()

        # find the links (Adjust to your needs!)
        #epublink = browser.find_link(text_regex=re.compile('.*Download ePub.*'))
        #mobilink = browser.find_link(text_regex=re.compile('.*Download Mobi.*'))
        pdflink = browser.find_link(text_regex=re.compile('https://skepticalinquirer.org/archive/'))

        # Cheat calibre's recipe method, as in post from Starsom17
        self.report_progress(0,_('downloading epub'))
        response = browser.follow_link(epublink)
        dir = PersistentTemporaryDirectory()
        epub_file = PersistentTemporaryFile(suffix='.epub',dir=dir)
        epub_file.write(response.read())
        epub_file.close()
        zfile = zipfile.ZipFile(epub_file.name, 'r')
        self.report_progress(0.1,_('extracting epub'))
        zfile.extractall(self.output_dir)
        epub_file.close()
        index = os.path.join(self.output_dir, 'content.opf')
        self.report_progress(0.2,_('epub downloaded and extracted'))


        #
        # Now, download the remaining files
        #
        if (GET_MOBI):
           self.report_progress(0.3,_('downloading mobi'))
           mobi_file = PersistentTemporaryFile(suffix='.mobi',dir=dir)
           browser.back()
           response = browser.follow_link(mobilink)
           mobi_file.write(response.read())
           mobi_file.close()

        if (GET_PDF):
           self.report_progress(0.4,_('downloading pdf'))
           pdf_file = PersistentTemporaryFile(suffix='.pdf',dir=dir)
           browser.back()
           response = browser.follow_link(pdflink)
           pdf_file.write(response.read())
           pdf_file.close()

        # Get all formats into Calibre's database as one single book entry
        self.report_progress(0.6,_('Adding files to Calibre db'))
        cmd = "calibredb add -1 " + dir
        os.system(cmd)

        return index

The error message now is:

Fetch news from The Skeptical Inquirer
Conversion options changed from defaults:
  verbose: 2
  output_profile: 'tablet'
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x0000022086166A30>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.TabletOutput object at 0x000002208618B370>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: custom:1000
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.182 Safari/537.36
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 216, in main
  File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
  File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1105, in run
  File "calibre\customize\conversion.py", line 245, in __call__
  File "calibre\ebooks\conversion\plugins\recipe_input.py", line 139, in convert
  File "calibre\web\feeds\news.py", line 1051, in download
  File "calibre\web\feeds\news.py", line 1228, in build_index
  File "calibre\web\feeds\news.py", line 1659, in parse_feeds
  File "calibre\web\feeds\news.py", line 474, in get_feeds
NotImplementedError

I feel like I'm getting there, yet not...

Thanks for your help!

[d.r.l]

Tried removing all the other format, more particularly epub as their lines were commented out, thinking it could be the source of the error.

Code

import re, zipfile, os
from calibre.ptempfile import PersistentTemporaryDirectory
from calibre.ptempfile import PersistentTemporaryFile
from urllib.parse import urlparse, urlsplit

class TheSkepticalInquirer(BasicNewsRecipe):
    title = u'The Skeptical Inquirer'
    description = 'Investigation of fringe science and paranormal claims.'
    language = 'en' 
    needs_subscription = True 

def build_index(self):
        browser = self.get_browser()

        pdflink = browser.find_link(text_regex=re.compile('https://skepticalinquirer.org/archive/'))

        # Cheat calibre's recipe method, as in post from Starsom17
        self.report_progress(0,_('downloading PDF'))
        response = browser.follow_link(pdflink)
        dir = PersistentTemporaryDirectory()
        pdf_file = PersistentTemporaryFile(suffix='.epub',dir=dir)
        pdf_file.write(response.read())
        pdf_file.close()

        # Get all formats into Calibre's database as one single book entry
        self.report_progress(0.6,_('Adding files to Calibre db'))
        cmd = "calibredb add -1 " + dir
        os.system(cmd)

        return index

Error:

Fetch news from The Skeptical Inquirer
Conversion options changed from defaults:
  verbose: 2
  output_profile: 'tablet'
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x0000023C025F6A30>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.TabletOutput object at 0x0000023C0261B370>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: custom:1000
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 216, in main
  File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
  File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1105, in run
  File "calibre\customize\conversion.py", line 245, in __call__
  File "calibre\ebooks\conversion\plugins\recipe_input.py", line 139, in convert
  File "calibre\web\feeds\news.py", line 1051, in download
  File "calibre\web\feeds\news.py", line 1228, in build_index
  File "calibre\web\feeds\news.py", line 1659, in parse_feeds
  File "calibre\web\feeds\news.py", line 474, in get_feeds
NotImplementedError

[kovidgoyal]

build_index and get_browser have to belong to the recipe class, which means they have to be indented correctly.

[d.r.l]

Hey Kovid, always a pleasure to see you personally helping out, it's very impressive. Thanks!

Your input helped out and I could move to steps forward. I am now stuck at the login page. I had to put back the 'open the login page first' before moving forward with the downloading of the code itself.

I'm now stuck with the login itself, I have the following error messages

mechanize._mechanize.FormNotFoundError: no form matching name 'login'

referring to the page's source code, I've tried to use the following tags (in red):

<div class="entry-content">
        <div class="login-form-container">
	
	<form name="loginform" id="loginform" action="https://skepticalinquirer.org/wp-login.php" method="post">
			
			<p class="login-username">
				<label for="user_login">Email or Username</label>
				<input type="text" name="log" id="user_login" class="input" value="" size="20">
			</p>
			<p class="login-password">
				<label for="user_pass">Password</label>
				<input type="password" name="pwd" id="user_pass" class="input" value="" size="20">
			</p>

While writing this, I changed used the name of the form itself loginform

And now have the following error message:

mechanize._form_controls.ControlNotFoundError: no control matching name 'EMAILADDRESS'

Here's the updated code as it stands now

import re, zipfile, os
from calibre.ptempfile import PersistentTemporaryDirectory
from calibre.ptempfile import PersistentTemporaryFile
from urllib.parse import urlparse, urlsplit

class TheSkepticalInquirer(BasicNewsRecipe):
    title = u'The Skeptical Inquirer'
    description = 'Investigation of fringe science and paranormal claims.'
    language = 'en' 
    needs_subscription = True 

    def build_index(self):
        br = self.get_browser()
        if self.username is not None and self.password is not None:
            br.open('https://skepticalinquirer.org/member-login/')
            br.select_form(name='loginform')
            br['HIDDEN']   = self.username
            br['HIDDEN'] = self.password
            br.submit()
        return br

        pdflink = br.find_link(text_regex=re.compile('https://skepticalinquirer.org/archive/'))

        # Cheat calibre's recipe method, as in post from Starsom17
        self.report_progress(0,_('downloading PDF'))
        response = br.follow_link(pdflink)
        dir = PersistentTemporaryDirectory()
        pdf_file = PersistentTemporaryFile(suffix='.pdf',dir=dir)
        pdf_file.write(response.read())
        pdf_file.close()

        # Get all formats into Calibre's database as one single book entry
        self.report_progress(0.6,_('Adding files to Calibre db'))
        cmd = "calibredb add -1 " + dir
        os.system(cmd)

        return index

Thanks again for your assistance, here !

[kovidgoyal]

look at the <input> tag's name attribute to get the control name which is log not EMAILADDRESS here.

[d.r.l]

DISREGARD THIS, I WAS THICK


If you mean changing
br.select_form(name='loginform')
to
br.select_form(name='log'),

It does not work, I have the following error message

calibre, version 5.13.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Skeptical Inquirer

Fetch news from The Skeptical Inquirer
Conversion options changed from defaults:
  verbose: 2
  output_profile: 'tablet'
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x00000200D09D7A30>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.TabletOutput object at 0x00000200D09FB370>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: custom:1000
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 216, in main
  File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
  File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1105, in run
  File "calibre\customize\conversion.py", line 245, in __call__
  File "calibre\ebooks\conversion\plugins\recipe_input.py", line 139, in convert
  File "calibre\web\feeds\news.py", line 1051, in download
  File "<string>", line 16, in build_index
  File "mechanize\_mechanize.py", line 681, in select_form
mechanize._mechanize.FormNotFoundError: no form matching name 'log'

[d.r.l]

Originally Posted by kovidgoyal

look at the <input> tag's name attribute to get the control name which is log not EMAILADDRESS here.

Indeed! I've changed the control code to log for the login and pwd for the password and it does something better!

Still not working though...

calibre, version 5.13.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Skeptical Inquirer

Fetch news from The Skeptical Inquirer
Conversion options changed from defaults:
  verbose: 2
  output_profile: 'tablet'
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'epub_flatten': False,
 'epub_inline_toc': False,
 'epub_toc_at_end': False,
 'epub_version': '2',
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x00000200F8776A30>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.TabletOutput object at 0x00000200F879B370>,
 'page_breaks_before': None,
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: custom:1000
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0
Parsing all content...
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 82, in <module>
  File "site.py", line 77, in main
  File "site.py", line 49, in run_entry_point
  File "calibre\utils\ipc\worker.py", line 216, in main
  File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
  File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
  File "calibre\ebooks\conversion\plumber.py", line 1115, in run
  File "calibre\ebooks\conversion\plumber.py", line 1313, in create_oebbook
  File "calibre\ebooks\oeb\reader.py", line 69, in __call__
  File "calibre\ebooks\oeb\base.py", line 553, in __init__
  File "ntpath.py", line 204, in splitext
TypeError: expected str, bytes or os.PathLike object, not NoneType

[kovidgoyal]

you are using the recipe system to download a pdf and add it directly to the db, that is never going to run to completion. Recipes are supposed to create HTML content.

[d.r.l]

I see!

Thanks for the info