07-29-2013, 08:16 AM | #1 |
Enthusiast
Posts: 28
Karma: 50
Join Date: Oct 2003
Location: Bavaria/Germany
Device: Palm m105, Kindle KB
|
Brand eins (DE): New recipe needed after major website relaunch
Hi everybody,
one of the most proliferated German language economy magazines is "brand eins", see http://www.brandeins.de They completely redesigned their website these days, which was long overdue. However, the relaunch affected the current calibre recipe which became non-functional immediately. Like before, all brand eins issues (with the exception of the most current issue still obtainable at the newsstand) are available as free downloads in the online archive at http://www.brandeins.de/archiv/ Unfortunately, the archived issues are no longer numbered consecutively, but now feature the issue's main topic in the permalink, for example http://www.brandeins.de/archiv/2013/motivation.html which probably makes it impossible for a script/recipe to find the relevant source link automatically. My own knowledge of calibre's scripting language is quite poor. The original authors of the recipe seem to be no loger reachable, but perhaps one of the specialists here in this forum is able to provide us with a new recipe that works? I think that a lot of calibre users would be very happy to get a decent recipe again! Thanks a lot, Ralph |
09-01-2013, 02:50 PM | #2 |
Train reader
Posts: 10
Karma: 15
Join Date: Nov 2010
Device: Kindle3
|
Thanks for asking, I am interested in an updated recipe, too.
|
09-23-2013, 05:27 PM | #3 |
Connoisseur
Posts: 89
Karma: 14894
Join Date: Oct 2007
Location: Germany
Device: KPW G2 , 10" tablet
|
Ditto.
The old version produces this error log--maybe it will help those "in the know" to come to our rescue : Code:
calibre, version 1.0.0 (win32, isfrozen: True) Conversion Error: Failed: Fetch news from brand eins Fetch news from brand eins Resolved conversion options calibre version: 1.0.0 {'asciiize': False, 'author_sort': None, 'authors': None, 'base_font_size': 0, 'book_producer': None, 'change_justification': 'original', 'chapter': None, 'chapter_mark': 'pagebreak', 'comments': None, 'cover': None, 'debug_pipeline': None, 'dehyphenate': True, 'delete_blank_paragraphs': True, 'disable_font_rescaling': False, 'dont_download_recipe': False, 'dont_split_on_page_breaks': True, 'duplicate_links_in_toc': False, 'embed_all_fonts': False, 'embed_font_family': None, 'enable_heuristics': False, 'epub_flatten': False, 'epub_inline_toc': False, 'epub_toc_at_end': False, 'extra_css': None, 'extract_to': None, 'filter_css': None, 'fix_indents': True, 'flow_size': 260, 'font_size_mapping': None, 'format_scene_breaks': True, 'html_unwrap_factor': 0.4, 'input_encoding': None, 'input_profile': <calibre.customize.profiles.InputProfile object at 0x00DD26D0>, 'insert_blank_line': False, 'insert_blank_line_size': 0.5, 'insert_metadata': False, 'isbn': None, 'italicize_common_cases': True, 'keep_ligatures': False, 'language': None, 'level1_toc': None, 'level2_toc': None, 'level3_toc': None, 'line_height': 0, 'linearize_tables': False, 'lrf': False, 'margin_bottom': 5.0, 'margin_left': 5.0, 'margin_right': 5.0, 'margin_top': 5.0, 'markup_chapter_headings': True, 'max_toc_links': 50, 'minimum_line_height': 120.0, 'no_chapters_in_toc': False, 'no_default_epub_cover': False, 'no_inline_navbars': False, 'no_svg_cover': False, 'output_profile': <calibre.customize.profiles.SonyReaderOutput object at 0x0355FAB0>, 'page_breaks_before': None, 'prefer_metadata_cover': False, 'preserve_cover_aspect_ratio': False, 'pretty_print': True, 'pubdate': None, 'publisher': None, 'rating': None, 'read_metadata_from_opf': None, 'remove_fake_margins': True, 'remove_first_image': False, 'remove_paragraph_spacing': False, 'remove_paragraph_spacing_indent_size': 1.5, 'renumber_headings': True, 'replace_scene_breaks': '', 'search_replace': None, 'series': None, 'series_index': None, 'smarten_punctuation': False, 'sr1_replace': '', 'sr1_search': '', 'sr2_replace': '', 'sr2_search': '', 'sr3_replace': '', 'sr3_search': '', 'start_reading_at': None, 'subset_embedded_fonts': False, 'tags': None, 'test': False, 'timestamp': None, 'title': None, 'title_sort': None, 'toc_filter': None, 'toc_threshold': 6, 'toc_title': None, 'unsmarten_punctuation': False, 'unwrap_lines': True, 'use_auto_toc': False, 'verbose': 2} InputFormatPlugin: Recipe Input running Using custom recipe Python function terminated unexpectedly list index out of range (Error Code: 1) Traceback (most recent call last): File "site.py", line 132, in main File "site.py", line 109, in run_entry_point File "site-packages\calibre\utils\ipc\worker.py", line 189, in main File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert File "site-packages\calibre\ebooks\conversion\plumber.py", line 1027, in run File "site-packages\calibre\customize\conversion.py", line 241, in __call__ File "site-packages\calibre\ebooks\conversion\plugins\recipe_input.py", line 114, in convert File "site-packages\calibre\web\feeds\news.py", line 980, in download File "site-packages\calibre\web\feeds\news.py", line 1145, in build_index File "<string>", line 102, in parse_index IndexError: list index out of range |
09-24-2013, 12:42 AM | #4 |
Enthusiast
Posts: 28
Karma: 50
Join Date: Oct 2003
Location: Bavaria/Germany
Device: Palm m105, Kindle KB
|
The error log probably won't be of any use due to the major redesign of the Brand eins website (see above). The new site structure is completely different, so anybody trying to fetch the contents has to start from scratch...
|
09-24-2013, 05:13 AM | #5 |
Connoisseur
Posts: 89
Karma: 14894
Join Date: Oct 2007
Location: Germany
Device: KPW G2 , 10" tablet
|
Oh, I see. Well, I for one cannot make head or tail of it, so I wouldn't have known.
I'm much better with "real" recipes though ;-) |
01-01-2014, 05:37 PM | #6 |
Enthusiast
Posts: 43
Karma: 136
Join Date: Mar 2011
Device: Kindle Paperwhite
|
I think I can make it work again. Let me know if you're still interested.
|
01-01-2014, 06:43 PM | #7 |
Connoisseur
Posts: 89
Karma: 14894
Join Date: Oct 2007
Location: Germany
Device: KPW G2 , 10" tablet
|
Yes, please, absolutely.
As far as I understand the website mechanics, it has stopped offering the full copy of the current edition. But as soon as another one is published the one before becomes available under the menu options Lesen>Archiv>[Year]>[Title]. I for one would be very grateful for a recipe which, while it doesn't work automatically, would allow me to enter the desired edition or path in a defined line in the recipe and then download that one edition. Hendrik |
01-02-2014, 02:33 AM | #8 |
Enthusiast
Posts: 28
Karma: 50
Join Date: Oct 2003
Location: Bavaria/Germany
Device: Palm m105, Kindle KB
|
I'd also be very grateful if somebody would manage to write a working brand eins recipe again! It seems, however, that with the release of a new (paper) issue, the previous one is not made available online immediately. Instead, it's articles are released one by one over a couple of days or weeks. So probably it is best to tell calibre to fetch issues that are two months old at least. This, however, should be no problem since the brand eins magazine is no daily newspaper and it's content doesn't become obsolete quickly...
Thanks, Ralph |
01-02-2014, 07:03 AM | #9 |
Enthusiast
Posts: 43
Karma: 136
Join Date: Mar 2011
Device: Kindle Paperwhite
|
Hi there,
I started working on this. I already have the index working again without the URL having the issue number included (that's the one major change you guys are referring to) and should be able to get the articles parsed hopefully later this week. Feel free to drop me a message if I should not update this thread anytime soon Since brandeins still is not uploading old issues completely, but releasing articles over time we definitely still need the "go back" option. I was thinking of changing the option. Instead of using the username field with numbers 2,3,4,5,6,etc. I thought it be nicer to just type in the issue number by year and month. E.g. password 10 (for October) and username 2012. What do you think? Cheers, -aerodynamik Last edited by aerodynamik; 01-03-2014 at 02:27 PM. Reason: Changed username and password to reflect actual implementation to avoid confusion |
01-02-2014, 08:04 AM | #10 |
Enthusiast
Posts: 28
Karma: 50
Join Date: Oct 2003
Location: Bavaria/Germany
Device: Palm m105, Kindle KB
|
Great idea! Just a hint: I'd refrain from including year and month of the issue in the name of the resulting ebook file. If the output is just called "brand eins" (regardless of which issue was processed), any Amazon Kindle will just keep the newest file sent to it on top level, transferring older issues automatically to the "old issues" folder. This is a wanted (and very much appreciated) behaviour, as far as periodicals are concerned. Individual file names (like those with years and months in it) will always cause the files to remain on top, thus needing manual sorting and handling...
Thank you very much indeed for your dedicated persistence (or is it persisting dedication?), Ralph Last edited by juco; 01-02-2014 at 08:07 AM. |
01-03-2014, 02:25 PM | #11 |
Enthusiast
Posts: 43
Karma: 136
Join Date: Mar 2011
Device: Kindle Paperwhite
|
First version of new recipe
Okay, here we go. This is a first version for the new recipe.
Could you guys give it a try and tell me what you think? What's still missing, what needs to be changed? Let me know. By default it now downloads the 3rd recent issue (the most recent has almost no articles up, the one after that has still some missing). If you want to chose your own, enter the year in the username field, and the month in the password field of calibre. Spoiler:
I have mainly 2 things open I'd still like to update (1) I have a Kindle myself, so I know and understand how it handles the periodicals. I like it that way, too. However, now in the periodical itself I cannot see anymore which issue I am reading. Any suggestions on where to add it? How did the old recipe of brandeins handle this? (2) If some of the experienced developers on here are reading this, I'd appreciate your help one thing: The website does not provide a usable cover image on the index page, but only within one of the articles. I thought I could simply use postprocess_html(), check if I am parsing that very article, extract the cover url and update the self.cover_url. However, this is not working, since get_cover_url is called way earlier. The way I 'fixed' it now is by creating another soup for just that article in get_cover_url and then getting the cover URL from there. It works, but I don't like it. Any better ideas? In addition, here is a list of issues by year, month, title and URL for your refernce to easily select the issue you want to download. Spoiler:
|
01-04-2014, 01:59 AM | #12 |
Enthusiast
Posts: 28
Karma: 50
Join Date: Oct 2003
Location: Bavaria/Germany
Device: Palm m105, Kindle KB
|
Great, the recipe works! :-) Thanks again!
BTW, I had to change the German U umlaut in the description line ("Übergang" -> "Uebergang") to make calibre accept the import of the new file. With that issue settled, calibre had no further objections... Now to your questions: (1) The issue handling is now exactly as it was before (with the old recipe still working*). You have the most current issue on top level and the older ones in the "old issues" folder. Personally, I do find this acceptable (and preferable!) since an issue number wouldn't tell you anything about the contents or major theme anyway. If you tell calibre to use the recipe monthly (for instance on every 1st day of the month), you get the idea which issue you are opening by just looking at the date. *I have to admit that I modified the old recipe by actually deleting the issue number (month/year) in the title section because titles like "brand eins 10/2012" prevented the files to be transferred to the "old issues" folder due to their unique name... My 2cents: No further action required, it probably can't get any better! (2) Can't help you here, however, since we have Kindle Keyboards here, we don't see any cover images on the index page anyway! ;-) I'm very happy with the new recipe you created, I wasn't hoping for a miracle of this kind any longer... :-) Danke, Ralph |
01-14-2014, 01:32 AM | #13 |
Junior Member
Posts: 2
Karma: 10
Join Date: Jan 2014
Device: kindle
|
How to implement the recipe?
I tried to implement the recipe with copy and paste but it is not working.
Can anyone give me an easy description how to make the recipe work? Thanks in advance |
01-14-2014, 01:48 AM | #14 |
Enthusiast
Posts: 43
Karma: 136
Join Date: Mar 2011
Device: Kindle Paperwhite
|
Just update to the latest version of Calibre. It's already included there.
|
01-30-2014, 12:07 PM | #15 |
Connoisseur
Posts: 89
Karma: 14894
Join Date: Oct 2007
Location: Germany
Device: KPW G2 , 10" tablet
|
Gee, thanks, that works just fine as far as I can tell. I'm between readers, so I checked on calibre's viewer only, but it looks great. And how nifty this idea to let me enter edition and year via the password. Chapeau!
Hendrik |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Fixed brand eins recipe | siebert | Recipes | 18 | 07-30-2013 06:56 AM |
eTextbooks for Kindle *major help needed* | JacobMiW | Conversion | 2 | 08-25-2012 10:57 PM |
Neue Zürcher Zeitung NZZ : Website relaunch | skoll1975 | Recipes | 1 | 06-09-2012 11:53 PM |
Enhanced brand eins recipe | siebert | Recipes | 12 | 04-11-2011 04:58 AM |
Recipe help needed for looping through sections of a website | Acey | Calibre | 1 | 10-16-2008 01:09 PM |