New Recipe for ZEIT Premium download ALL

[achims]

Hi Ben,

with this configuration, the recipe should create 3 or 4 new book entries:
a) Beilage (1 pdf file, not in every edition)
b) Magazin (1 pdf file)
c) Zeit (1 mobi file and 1 epub file)
d) Zeit (1 epub file)
You might want to enter 'tags:Zeit' into the search bar in order to find them all.
Calibre shows the book formats in the 'Book Details' section of the graphical user interface.
Please confirm that this is what you get. If so, it would seem to me that this is a general issue of your calibre configuration and not specially related to the recipe. Have you tried sending different mobi files obtained from other sources? Does it still send epub? In this case you should post your problem on the Calibre - Devices forum.

Anyway, this is what I would do: take book entry c) and send to device. It should send the mobi file, since you set calibre to prefer mobi output. If it still sends an epub, delete the epub file in book entry c) and try again. If it still sends an epub, then I have no clue. I am not using mobi files, so I cannot test.

Best regards
Achim

[rogerben]

Hi,

you are right - it does indeed download the native mobi file but then choses to email the altered epub to my kindle (this is the way I usually transfer my news). When I manually send the mobi file from c), it is perfect. Do you happen to know if I can tell calibre to send this little file while batch processing, rather than the converted one?

Thanks so far, and: frohen Advent.

-br

[achims]

Hi Ben,

if have so far never used calibre's email function, so unfortunately I cannot answer your question.

I wish you good luck figuring out, and of course auch einen frohen Advent.

Achim

[Rince123]

Hmm, I tried the recipie tonight, but it wasn't able to fetch everything, but I get an error:

calibre, version 0.8.35
ERROR: Konvertierungsfehler: <b>Misslungen</b>: Nachrichten abrufen von Zeit Premium All Formats

Nachrichten abrufen von Zeit Premium All Formats
Resolved conversion options
calibre version: 0.8.35
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'dont_split_on_page_breaks': True,
'duplicate_links_in_toc': False,
'enable_heuristics': False,
'epub_flatten': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'flow_size': 260,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x108770190>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_default_epub_cover': False,
'no_inline_navbars': False,
'no_svg_cover': False,
'output_profile': <calibre.customize.profiles.OutputProfile object at 0x108770550>,
'page_breaks_before': None,
'password': 'wontworkhere',
'prefer_metadata_cover': False,
'preserve_cover_aspect_ratio': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'series': None,
'series_index': None,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'username': 'myownone',
'verbose': 2}
InputFormatPlugin: Recipe Input running
Found epub-link: http://contentserver.hgv-online.de/n...GbvVIq4sk3E%3D
Found Mobi-link: http://contentserver.hgv-online.de/n...FRtDnfayaWg%3D
Found pdf-link: https://premium.zeit.de/system/files...aper_03_12.pdf
Found audio-link: https://premium.zeit.de/system/files...IE_ZEIT_03.zip
Will try ZM-link: https://premium.zeit.de/system/files...aper_03_12.pdf
Will try BL-link: https://premium.zeit.de/system/files...aper_03_12.pdf
This edition is: 2012 - 03
output_dir is: /private/var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/Tlbxpz_plumber
DZdir is: /var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/DZ__mSzwH
ZMdir is: /var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/ZM_BwiPtb
BLdir is: /var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/BL_FPbRst
Python function terminated unexpectedly: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Found
Traceback (most recent call last):
File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 147, in main
return run_entry_point()
File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 116, in run_entry_point
return getattr(pmod, func)()
File "site-packages/calibre/utils/ipc/worker.py", line 191, in main
File "site-packages/calibre/gui2/convert/gui_conversion.py", line 25, in gui_convert
File "site-packages/calibre/ebooks/conversion/plumber.py", line 959, in run
File "site-packages/calibre/customize/conversion.py", line 204, in __call__
File "site-packages/calibre/web/feeds/input.py", line 105, in convert
File "site-packages/calibre/web/feeds/news.py", line 856, in download
File "/var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/jfCHhN_recipes/recipe0.py", line 273, in build_index
browser.back()
File "site-packages/mechanize/_mechanize.py", line 356, in back
File "site-packages/mechanize/_mechanize.py", line 343, in reload
File "site-packages/mechanize/_mechanize.py", line 255, in _mech_open
mechanize._response.httperror_seek_wrapper: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Found

So, Something goes wrong, even when it finds the links. Any idears here?

[achims]

Hi Rince123,

I have just tested the recipe and cannot reproduce your error (I even updated to your calibre version, just in case).

From the error log, the problem has to do with the 'browser.back()' statement just before downloading the audio files. Seems like a previous download failed (which should be the pdf file, if enabled).

Did you try to download the files manually from the web page? There might just be some temporally connection problem between your provider and the ZEIT server.

Hope this helps
Achim

PS: Just a note to help make the forum thread more readable: You should enclose larger code-snippets (like the error log) within '[ CODE][ SPOILER] [/SPOILER][/CODE]' marks (the '#' symbol and the one with the X'd eye).

[Rince123]

Hmm, I just tried it again, with another Provider, same result:

(and I just downloaded manually everything which worked fine..)

Spoiler:

calibre, version 0.8.35
ERROR: Konvertierungsfehler: <b>Misslungen</b>: Nachrichten abrufen von Zeit Premium All Formats

Nachrichten abrufen von Zeit Premium All Formats
Resolved conversion options
calibre version: 0.8.35
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': 'original',
 'chapter': None,
 'chapter_mark': 'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_download_recipe': False,
 'dont_split_on_page_breaks': True,
 'duplicate_links_in_toc': False,
 'enable_heuristics': False,
 'epub_flatten': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'flow_size': 260,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x108690190>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'no_chapters_in_toc': False,
 'no_default_epub_cover': False,
 'no_inline_navbars': False,
 'no_svg_cover': False,
 'output_profile': <calibre.customize.profiles.OutputProfile object at 0x108690550>,
 'page_breaks_before': None,
 'password': 'foobar',
 'prefer_metadata_cover': False,
 'preserve_cover_aspect_ratio': False,
 'pretty_print': True,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': '',
 'series': None,
 'series_index': None,
 'smarten_punctuation': False,
 'sr1_replace': '',
 'sr1_search': '',
 'sr2_replace': '',
 'sr2_search': '',
 'sr3_replace': '',
 'sr3_search': '',
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'username': 'barfoo',
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Using proxies: {'http': 'ivanova.notwork.de:8118', 'https': 'ivanova.notwork.de:8118'}
Using proxies: {'http': 'ivanova.notwork.de:8118', 'https': 'ivanova.notwork.de:8118'}
Found epub-link: http://contentserver.hgv-online.de/nodrm/fulfillment?distributor=zeit-online&orderid=zeit_online%3A132678931611786285811&resid=die_zeit-abo&customerno=77046&auth=U0bol1fHvoNIKhcWnuMKnmQQef4%3D
Found Mobi-link: http://contentserver.hgv-online.de/nodrm/fulfillment?distributor=zeit-online&orderid=zeit_online%3A132678931611786285811&resid=die_zeit-abo-mobi&customerno=77046&auth=KOupOhb%2F%2FF4tnWOv9WZOLBveG4s%3D
Found pdf-link: https://premium.zeit.de/system/files/epaper/DZ/pdf/DZ_ePaper_03_12.pdf
Found audio-link: https://premium.zeit.de/system/files/audio/DZ/2012/Audiofiles_DIE_ZEIT_03.zip
Will try ZM-link: https://premium.zeit.de/system/files/epaper/ZM/pdf/ZM_ePaper_03_12.pdf
Will try BL-link: https://premium.zeit.de/system/files/epaper/BL/pdf/BL_ePaper_03_12.pdf
This edition is: 2012 - 03
output_dir is: /private/var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/UabFwk_plumber
DZdir is: /var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/DZ_kZ65iq
ZMdir is: /var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/ZM_0CH5iV
BLdir is: /var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/BL_cF0MgR
Python function terminated unexpectedly: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Found
Traceback (most recent call last):
  File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 147, in main
    return run_entry_point()
  File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 116, in run_entry_point
    return getattr(pmod, func)()
  File "site-packages/calibre/utils/ipc/worker.py", line 191, in main
  File "site-packages/calibre/gui2/convert/gui_conversion.py", line 25, in gui_convert
  File "site-packages/calibre/ebooks/conversion/plumber.py", line 959, in run
  File "site-packages/calibre/customize/conversion.py", line 204, in __call__
  File "site-packages/calibre/web/feeds/input.py", line 105, in convert
  File "site-packages/calibre/web/feeds/news.py", line 856, in download
  File "/var/folders/xU/xU7s6ruOEQiHnUuqHijBqE+++TI/-Tmp-/calibre_0.8.35_tmp_7FJ_8B/JqyK02_recipes/recipe0.py", line 273, in build_index
    browser.back()
  File "site-packages/mechanize/_mechanize.py", line 356, in back
  File "site-packages/mechanize/_mechanize.py", line 343, in reload
  File "site-packages/mechanize/_mechanize.py", line 255, in _mech_open
mechanize._response.httperror_seek_wrapper: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.
The last 30x error message was:
Found

[achims]

@Rince123,

please try the following version. It does not use browser.back(), so I hope this will work.

Spoiler:

import sys, re, zipfile, os
from calibre.ptempfile import PersistentTemporaryDirectory
from calibre.ptempfile import PersistentTemporaryFile
from urlparse import urlparse
from calibre.ebooks.metadata import MetaInformation, string_to_authors
from calibre.library.cli import do_add_empty, send_message, write_dirtied, do_add
from calibre.utils.config import prefs
from calibre.library.database2 import LibraryDatabase2



GET_MOBI=False
GET_PDF=True
GET_AUDIO=True
GET_MAGAZIN=True
GET_BEILAGE=True
authors = 'Zeitverlag Gerd Bucerius GmbH und Co. KG'
tags = ['Die Zeit']
languages = ['de']
domain = "https://premium.zeit.de"

class ZeitPremiumAllFormats(BasicNewsRecipe):
    title          = u'Zeit Premium All Formats'
    description    = u'Lädt alle angebotenen E-Book Formate der aktuellen Woche aus dem Zeit Premium Bereich (kostenpflichtiges Abo): Die Zeit als epub, mobi, pdf und alle Audiofiles als zip. Sie werden in der Calibre Datenbank als ein einziges Buch eingetragen. Das Zeit Magazin und ggfls. die Beilage als pdf als je eigenständiges Buch. Aus technischen Gründen wird ein doppelter Bucheintrag der Zeit erstellt, der ein epub in einer abgewandelten Version erhält. Dieser Eintrag kann gelöscht werden. Alle Formate ausser epub können ein- oder ausgeschaltet werden. Anmerkung: Während der Umstellung auf eine neue Ausgabe (Mittwoch abends) werden nicht alle Formate gleichzeitig erneuert. Im Calibre Eintrag können dann die verschiedenen Formate zu verschiedenen Ausgaben gehören! Bei mehrfachem Aufruf werden Duplikate der Bucheinträge erstellt.'
    __author__ = 'Achim Schumacher'
    language = 'de'
    needs_subscription = True
    conversion_options = {
        'no_default_epub_cover' : True,
    }

    #
    # Login process required:
    # Override BasicNewsRecipe.get_browser()
    #
    def get_browser(self):
        br = BasicNewsRecipe.get_browser()
        # new login process
        response = br.open(domain)
        # Get rid of nested form
        response.set_data(re.sub('<div><form action=.*', '', response.get_data() ))
        br.set_response(response)
        br.select_form(nr=2)
        br.form['name']=self.username
        br.form['pass']=self.password
        br.submit()
        return br


    # Copies only those parts of the new metadata to the old metadata
    # which have actual data.
    def copy_metadata(self, new, old):
        mi = old
        if new.title:
            mi.title = new.title
        if new.authors:
            mi.authors = new.authors
        if new.isbn:
            mi.isbn = new.isbn
        if new.tags:
            mi.tags = new.tags
        if new.languages:
            mi.languages = new.languages
        return mi

    # Override calibre.library.import_book_directory
    # because it does not offer options to set metadata.
    # This version adds a new option mi: 
    # for all mi-fields which have data, the data is copied to ebook's metadata
    def import_book_directory(self, db, dirpath, mi2, callback=None):
        from calibre.ebooks.metadata.meta import metadata_from_formats
        dirpath = os.path.abspath(dirpath)
        formats = db.find_books_in_directory(dirpath, True)
        formats = list(formats)[0]
        if not formats:
            return
        mi = metadata_from_formats(formats)
        mi = self.copy_metadata(mi2, mi)
        if mi.title is None:
            return
        if db.has_book(mi):
            return [(mi, formats)]
        db.import_book(mi, formats)
        if callable(callback):
            callback(mi.title)


    # Override calibre.library.do_add,
    # because it does not offer options to set metadata.
    # This version adds a new option mi: 
    # for all mi-fields which have data, the data is copied to ebook's metadata
    # In this version: recurse=False, one_book_per_directory=True
    def do_add(self, db, paths, mi2, add_duplicates):
        from calibre.ebooks.metadata.meta import get_metadata
        orig = sys.stdout
        #sys.stdout = NULL
        try:
            files, dirs = [], []
            for path in paths:
                path = os.path.abspath(path)
                if os.path.isdir(path):
                    dirs.append(path)
                else:
                    if os.path.exists(path):
                        files.append(path)
                    else:
                        print path, 'not found'

            formats, metadata = [], []
            for book in files:
                format = os.path.splitext(book)[1]
                format = format[1:] if format else None
                if not format:
                    continue
                stream = open(book, 'rb')
                mi = get_metadata(stream, stream_type=format, use_libprs_metadata=True)
                if not mi.title:
                    mi.title = os.path.splitext(os.path.basename(book))[0]
                if not mi.authors:
                    mi.authors = [_('Unknown')]
                mi = self.copy_metadata(mi2, mi)
                formats.append(format)
                metadata.append(mi)

            file_duplicates = []
            if files:
                file_duplicates = db.add_books(files, formats, metadata,
                                               add_duplicates=add_duplicates)
                if file_duplicates:
                    file_duplicates = file_duplicates[0]
    

            dir_dups = []
            for dir in dirs:
#                if recurse:
#                    dir_dups.extend(db.recursive_import(dir, single_book_per_directory=one_book_per_directory))
#                else:
                    func = self.import_book_directory(db, dir, mi2)# if one_book_per_directory else db.import_book_directory_multiple
                    dups = func
                    if not dups:
                        dups = []
                    dir_dups.extend(dups)

            sys.stdout = sys.__stdout__

            if add_duplicates:
                for mi, formats in dir_dups:
                    mi = self.copy_metadata(mi2, mi)
                    db.import_book(mi, formats)
            else:
                if dir_dups or file_duplicates:
                    print >>sys.stderr, _('The following books were not added as '
                                          'they already exist in the database '
                                          '(see --duplicates option):')
                for mi, formats in dir_dups:
                    title = mi.title
                    if isinstance(title, unicode):
                        title = title.encode(preferred_encoding)
                    print >>sys.stderr, '\t', title + ':'
                    for path in formats:
                        print >>sys.stderr, '\t\t ', path
                if file_duplicates:
                    for path, mi in zip(file_duplicates[0], file_duplicates[2]):
                        title = mi.title
                        if isinstance(title, unicode):
                            title = title.encode(preferred_encoding)
                        print >>sys.stderr, '\t', title+':'
                        print >>sys.stderr, '\t\t ', path

            write_dirtied(db)
            send_message()
        finally:
            sys.stdout = orig



    # Do not fetch news and convert them to E-Books.
    # Instead, download the epub directly from the site.
    # For this, override BasicNewsRecipe.build_index()
    #
    def build_index(self):
        browser = self.get_browser()
        # Get the path to the db
        dbpath = prefs['library_path']
        # Get access to the database
        dbpath = os.path.abspath(dbpath)
        db = LibraryDatabase2(dbpath)


        # find the links
        epublink = browser.find_link(text_regex=re.compile('.*als Datei im ePub-Format.*'))
        mobilink = browser.find_link(text_regex=re.compile('.*im Mobi-Format.*'))
        pdflink = browser.find_link(text_regex=re.compile('.*Download der gesamten Ausgabe als PDF Datei.*'))
        browser.open("https://premium.zeit.de/abo/zeit-audio")
        audiolink = browser.find_link(text_regex=re.compile('.*Alle Audios der aktuellen ZEIT.*'))
        #edition = (urlparse(pdflink.url)[2]).replace('/system/files/epaper/DZ/pdf/DZ_ePaper_','').replace('.pdf','')
        edition_ = re.split('_', (urlparse(pdflink.url)[2]).replace('/system/files/epaper/DZ/pdf/DZ_ePaper_','').replace('.pdf','') )
        edition = '20' + edition_[1] + ' - ' + edition_[0]
        au_url = urlparse(pdflink.base_url)[0]+'://'+urlparse(pdflink.base_url)[1]+''+(urlparse(pdflink.url)[2]).replace('epaper/DZ/pdf/DZ_ePaper','ZM/pdf/ZM_ePaper')
        zm_url = urlparse(pdflink.base_url)[0]+'://'+urlparse(pdflink.base_url)[1]+''+(urlparse(pdflink.url)[2]).replace('DZ/pdf/DZ_ePaper','ZM/pdf/ZM_ePaper')
        bl_url = urlparse(pdflink.base_url)[0]+'://'+urlparse(pdflink.base_url)[1]+''+(urlparse(pdflink.url)[2]).replace('DZ/pdf/DZ_ePaper','BL/pdf/BL_ePaper')
        print "Found epub-link: %s" % epublink.url
        print "Found Mobi-link: %s" % mobilink.url
        print "Found pdf-link: %s" % pdflink.url
        print "Found audio-link: %s" % audiolink.url
        print "Will try ZM-link: %s" % zm_url
        print "Will try BL-link: %s" % bl_url
        print "This edition is: %s" % edition

        # The following part is from a recipe by Starson17
        #
        # It modifies build_index, which is the method that gets the 
        # masthead image and cover, parses the feed for articles, retrieves
        # the articles, removes tags from articles, etc. All of those steps 
        # ultimately produce a local directory structure that looks like an 
        # unzipped EPUB. 
        #
        # This part grabs the link to one EPUB, saves the EPUB locally,
        # extracts it, and passes the result back into the recipe system
        # as though all the other steps had been completed normally.
        #
        # This has to be done, even if one does not want to use this
        # calibre-modified epub. Otherwise, the recipe runs into an error.
        # This is the reason why there shows up a second Die Zeit entry
        # in calibre db.
        self.report_progress(0,_('downloading epub'))
        browser.open(domain);
        response = browser.follow_link(epublink)
        # We need two different directories for Die Zeit and Zeit Magazin
        DZdir = PersistentTemporaryDirectory(prefix='DZ_')
        ZMdir = PersistentTemporaryDirectory(prefix='ZM_')
        BLdir = PersistentTemporaryDirectory(prefix='BL_')
        epub_file = PersistentTemporaryFile(suffix='.epub',dir=DZdir)
        epub_file.write(response.read())
        epub_file.close()
        zfile = zipfile.ZipFile(epub_file.name, 'r')
        self.report_progress(0.1,_('extracting epub'))
        zfile.extractall(self.output_dir)
        epub_file.close()
        index = os.path.join(self.output_dir, 'content.opf')
        self.report_progress(0.2,_('epub downloaded and extracted'))

        #
        # Now, download the remaining files
        #
        print "output_dir is: %s" % self.output_dir
        print "DZdir is: %s" % DZdir
        print "ZMdir is: %s" % ZMdir
        print "BLdir is: %s" % BLdir

        if (GET_MOBI):
           self.report_progress(0.3,_('downloading mobi'))
           mobi_file = PersistentTemporaryFile(suffix='.mobi',dir=DZdir)
           browser.open(domain);
           response = browser.follow_link(mobilink)
           mobi_file.write(response.read())
           mobi_file.close()

        if (GET_PDF):
           self.report_progress(0.4,_('downloading pdf'))
           pdf_file = PersistentTemporaryFile(suffix='.pdf',dir=DZdir)
           browser.open(domain);
           response = browser.follow_link(pdflink)
           pdf_file.write(response.read())
           pdf_file.close()

        if (GET_AUDIO):
           self.report_progress(0.5,_('downloading audio'))
           audio_file = PersistentTemporaryFile(suffix='.mp3.zip',dir=DZdir)
           browser.open(domain);
           response = browser.follow_link(audiolink)
           audio_file.write(response.read())
           audio_file.close()

        # Get all Die Zeit formats into Calibre's database
        self.report_progress(0.6,_('Adding Die Zeit to Calibre db'))
        mi = MetaInformation(None)
        title="Die ZEIT "+edition
        mi.title = title
        mi.authors = string_to_authors(authors)
        mi.tags = tags
        mi.languages = languages
        self.do_add(db, [DZdir], mi, True)
        

        # Zeit Magazin has to be handled differently.
        # First, it has to be downloaded into it's own directory, since it
        # is a different book as Die Zeit.
        # Second, we know its url rather than its link.
        # Third, there is no Metadata present in the file itself.
        if (GET_MAGAZIN):
           self.report_progress(0.7,_('downloading ZM'))
           title="ZEIT Magazin "+edition
           ZM_file = PersistentTemporaryFile(suffix='.pdf',dir=ZMdir)
           try:
              response = browser.open(zm_url)
              ZM_file.write(response.read())
              ZM_file.close()
              # Get Zeit Magazin into Calibre's database
              self.report_progress(0.8,_('Adding Zeit Magazin to Calibre db'))
              mi.title = title
              self.do_add(db, [ZMdir], mi, True)
           except:
              self.report_progress(0.8,_('No Zeit Magazin found...'))

        # Zeit Beilage is technically the same as Zeit Magazin, but it is
        # not included in every edition. So, the use of try: is 
        # obligatory here.
        if (GET_BEILAGE):
           self.report_progress(0.9,_('downloading BL'))
           title="ZEIT Beilage "+edition
           BL_file = PersistentTemporaryFile(suffix='.pdf',dir=BLdir)
           try:
              response = browser.open(bl_url)
              BL_file.write(response.read())
              BL_file.close()
              # Get Zeit Beilage into Calibre's database
              self.report_progress(0.9,_('Adding Zeit Beilage to Calibre db'))
              mi.title = title
              self.do_add(db, [BLdir], mi, True)
           except:
              self.report_progress(0.9,_('No Zeit Beilage found...'))

        return index

[Rince123]

achims,

thank for the new version - this version works fine (takes around 15 minutes when I enable MOBI) and gets all versions - wonderful!

[achims]

Rince123,

your welcome. Nice to hear it works.

[rogerben]

Still haven't figured out how to have calibre send the proper (original) mobi file to my kindle - it is created, indeed, but calibre sends the converted epub. Could there be a way to have calibre fetch only the mobi file in the first place?

TIA!
Regards
b.r.

[achims]

@rogerben
Hi,

I have looked into the email function of calibre. Turns out that it sends an automatic email once the recipe is done. It will use the result of the recipe. Unfortunately, calibre's recipe system is being kind of tricked out by my recipe - it only knows of the converted epub, and the other files are downloaded behind it's back... So, there is no way the recipe system could trigger an email with files it does not know about. Even if I included an option not to download the epub version, the recipe system would still insist in building some dummy nonsense epub, which would then be sent by email...

You should however be able to send the mobi file by manual selection. Just be sure to have erased the 'epub' option and included the 'mobi' option in calibre's email configurations.

[Timbo80]

Hi everyone, as I just wrote in the german thread:
https://www.mobileread.com/forums/sho...=154764&page=2

DIE ZEIT offers a push-email-delivery of the .epub and the .mobi - a script is no longer necessary. On premium.zeit.de go to "Kontoinformationen" -> "Einstellungen für automatische Benachrichtigungen/Versand bearbeiten". Here you can enter an emailadress or a @kindle.com adress if you want the mobi.