WSJ recipe not working

[mjfriedman]

Possibly WSJ changed some URLs? Error message below, BIG thank you for any help!

Michael Jay Friedman
_____

'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:wsj
Trying to get latest version of recipe: wsj
Using user agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36
Starting login process...
Sending login request...
Performing login callback...
Python function terminated unexpectedly: No articles found, aborting
Traceback (most recent call last):
File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 187, in main
return run_entry_point()
File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 121, in run_entry_point
return getattr(pmod, func)()
File "site-packages/calibre/utils/ipc/worker.py", line 209, in main
File "site-packages/calibre/gui2/convert/gui_conversion.py", line 36, in gui_convert_recipe
File "site-packages/calibre/gui2/convert/gui_conversion.py", line 28, in gui_convert
File "site-packages/calibre/ebooks/conversion/plumber.py", line 1110, in run
File "site-packages/calibre/customize/conversion.py", line 246, in __call__
File "site-packages/calibre/ebooks/conversion/plugins/recipe_input.py", line 139, in convert
File "site-packages/calibre/web/feeds/news.py", line 1022, in download
File "site-packages/calibre/web/feeds/news.py", line 1201, in build_index
ValueError: No articles found, aborting

[kovidgoyal]

I just tried the free version which is the same code as the subscription one and it downloaded the articles fine (as subscription required stubs of course). Can t try the subscription version since I am not a subscriber.

[mjfriedman]

Hi and thank you for the reply!

I'm using the recipe for the daily print edition content if that helps. I can send you credentials to log-in if it will help figure out what changed today....

Thanks again, Michael

[mjfriedman]

Hi--Also the home page for the print edition (https://www.wsj.com/itp) looks like it was reworked so possibly there were changes on the Journal's end?


Best, Michael

[kovidgoyal]

Sure PM me your login credentials and I will try to reproduce with that

[mjfriedman]

Thank **again** to Kovid for the fix!

[mjfriedman]

Hi-- today (14 May) the recipe did not download the "U.S. News" section of the paper. Will report back here if it happens again tomorrow.

[mjfriedman]

Complete paper downloaded today

[duluoz]

Originally Posted by mjfriedman

Complete paper downloaded today

Hi - is the subscription version of WSJ working for you now? Still not working for me. Thanks.

[mjfriedman]

UPDATE: About half the time recipe works first time. Other days I have to re-run it. Today it hangs regardless.

[modelstudent101]

Having some issues intermittently - either hangs or downloads sections incorrectly (i.e. only downloads Front Page and Business, not US or International, or downloads Front Page and Greater New York, but not Business). Subscription version, initially was hanging/unable to pull. Recipe is not returning any errors in GUI.

[mjfriedman]

It's getting worse for me. This is second consecutive day I've been unable to download:

Fetch news from The Wall Street Journal
Failed to initialize plugin: '/Users/michael/Library/Preferences/calibre/plugins/DeDRM.zip'
Conversion options changed from defaults:
output_profile: 'kindle_voyage'
verbose: 2
Resolved conversion options
calibre version: 5.6.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x10c7a37c0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindleVoyageOutput object at 0x10c7a3df0>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:wsj
Trying to get latest version of recipe: wsj
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Starting login process...
Sending login request...
Performing login callback...
Found section: Front page at https://www.wsj.com/print-edition/20201211/frontpage
Found article: FDA Panel Endorses Covid-19 Vaccine
A Food and Drug Administration advisory panel recommended broad distribution of the first Covid-19 vaccine in the U.S., clearing the way for the FDA to grant emergency authorization as early as Friday.
Found article: Airbnb’s Stock Price More Than Doubles in Market Debut
The IPO ended a difficult year for the home-sharing company, which has had to readjust its business amid the travel downturn because of Covid-19.
Found article: ECB Expands Stimulus Program to Prop Up Pandemic-Hit Economy
The European Central Bank scaled up its bond-buying program and unveiled a new batch of ultracheap loans for banks, a move aimed at backstopping the region’s governments and businesses.
Found article: The Office Christmas Party Is Still On—and More Awkward Than Ever
Holiday gatherings with co-workers look different this year, with ugly sweater competitions on Zoom and virtual avatars dancing the night away. ‘I’d rather braid grass’
Found article: China’s Xi Ramps Up Control of Private Sector. ‘We Have No Choice but to Follow the Party.’
The push is driven by a deepening conviction within the country’s leadership that markets and entrepreneurs are not to be fully trusted. “The market-reform camp is all but gone,” says one economist.
Failed to find Whats News section
Found section: U.S. news at https://www.wsj.com/print-edition/20201211/us
Found article: U.S. Unemployment Claims Rise to Highest Level Since September
The number of workers seeking unemployment benefits climbed to 853,000 last week, an indication layoffs remain at a high level nearly nine months into the pandemic.
Found article: GOP Leaders See Bipartisan Group’s Covid-Aid Effort Falling Short
Top Senate Republicans signaled Thursday they wouldn’t accept a bipartisan group’s efforts to craft a compromise on state and local governments and liability protections during the pandemic.
Found article: Justices Rule for Muslim Men Who Sued FBI Agents After Being Placed on No-Fly List
The men were placed on the no-fly list in alleged religious retaliation for their refusal to cooperate with counterterrorism investigations.
Found article: U.S. Carries Out Second of Six Executions Set During Biden Transition
Brandon Bernard was executed Thursday, the second of six federal inmates scheduled to die between Election Day and Jan. 20, when President-elect Joe Biden, who opposes capital punishment, takes office.
Found article: George Floyd Memorial Thrives, but Minneapolis Neighborhood Struggles
Six months after activists turned the intersection where George Floyd was killed into a memorial, Minneapolis is seeking to remember Mr. Floyd in a way that allows the streets to reopen and restores full access to businesses.
Found article: U.S. Household Net Worth Hits Record in Third Quarter
Household net worth rose 3.2% in the third quarter from the second quarter to $123.52 trillion, according to a Federal Reserve report, while household debt rose 5.6%.
Found article: Barr Worked to Keep Hunter Biden Probes From Public View During Election
Attorney General William Barr has known about a set of investigations involving Hunter Biden’s business and financial dealings since at least this spring, a person familiar with the matter said, and worked to avoid their public disclosure during the election campaign.
Found article: Joe Biden Names Susan Rice to Lead White House Domestic Policy Council
The president-elect has tasked the former ambassador to the United Nations with coordinating with federal agencies and mapping out his agenda on health care, energy and other issues.
Found article: Biden Chooses Ex-Obama Official to Lead Department of Veterans Affairs
Denis McDonough’s experience shaping domestic and foreign policy was a factor in the president-elect’s decision to select him. If confirmed, he would be the second non-veteran to hold the position.
Found article: Trump Bets on Long-Shot Texas Election Lawsuit
The president and his supporters have pinned their hopes of reversing President-elect Joe Biden’s victory on a lawsuit filed by the Republican Texas attorney general that election-law specialists say has almost no chance of success.
Found article: VA Secretary Robert Wilkie Mishandled Sexual Assault Claim, Report Finds
Veterans Affairs Secretary Robert Wilkie mishandled an investigation into sexual assault at the department and disparaged the woman making the claim, according to a report released by the VA Office of Inspector General.
Found article: House Panel Accuses Trump Administration of Concealing Evidence in CDC Coronavirus Probe
A senior CDC official told House oversight subcommittee staff that she was instructed to delete an Aug. 8 email that showed a political appointee tried to meddle in a key report, according to a letter from Rep. James Clyburn.
Found section: World news at https://www.wsj.com/print-edition/20201211/world
Found article: Covid-19 Vaccines Face Last-Mile Challenge to Reach Most Vulnerable
Nursing-home residents would benefit most from a Covid-19 vaccine, but they are proving the hardest to reach with the new shot.
Found article: Governments Will Want to Track Who Has Had a Covid-19 Vaccine
The rollout of new vaccines is highlighting gaps in countries’ monitoring of who has been inoculated.
Found article: Russia Struggles to Make Promised Doses of Its Sputnik V Vaccine
Russia is finding it hard to manufacture large quantities of its two-shot Covid-19 vaccine, Sputnik V. It expects to produce just two million doses by the end of the year, down from the 30 million promised over the summer.
Found article: Boris Johnson Warns Brexit Talks Could Fail as EU Sets No-Deal Plans
The British prime minister said there was a strong possibility that negotiations over a new trade agreement with the EU would fail as the European Commission set out contingency plans to maintain basic links between the U.K. and the bloc.
Found article: Israel and Morocco Agree to Normalize Ties, Trump and Netanyahu Say
The deal, which required a U.S. shift on Morocco policy, marks another advance in U.S. efforts to strengthen ties between once-hostile nations in the Middle East
Found article: Lebanon Leader Charged With Negligence for Beirut Explosion
The judge leading an investigation into the massive explosion at Beirut’s port charged the caretaker prime minister and three ex-ministers with negligence.
Found section: Greater new york at https://www.wsj.com/print-edition/20201211/new-york
Found article: Jersey City Says It Is Better Prepared One Year After Mass Shooting
Officials addressed the possibility of future incidents of domestic terrorism after a mass shooting targeted members of the city’s Jewish community one year ago on Thursday.
Found article: New York City Cultural Scene Increasingly Draws Immigrant Artists
New York City is home to a sizable and growing population of artists from abroad, a new report says. But they face myriad challenges, particularly during the coronavirus pandemic, that may affect their ability to continue working and living within the five boroughs.
Found article: New York City School Debates How Best to Test for Covid-19
Questions about how many students learning in person should be tested, how frequently and who should have oversight are emblematic of the struggles schools across the country are experiencing.
Found article: New York City to Gauge Students’ Learning Loss During Pandemic
Mayor Bill de Blasio says the district will assess the impact of school closures as it tries to close an “achievement gap.”
Found article: Whale Watching in New York Grows More Common
A humpback whale has been splashing around the Statue of Liberty this week, marking its third known visit to New York waters—and the first time since 2016 that a whale has been spotted north of the Verrazzano-Narrows Bridge.
Found section: Life & arts at https://www.wsj.com/print-edition/20201211/lifearts
Found article: ‘The Midnight Sky’ Review: Reaching Out in the Void
George Clooney directs and stars in this post-apocalypic saga that intertwines the stories of an Arctic researcher, returning astronauts, and a mysterious little girl.
Found article: ‘Heroes of the Fourth Turning’ Review: A Culture-War Conversation Piece
In Will Arbery’s singular, sympathetic play, young conservative intellectuals gather to discuss their post-graduate lives and political concerns in the age of Trump.
Found article: ‘Total Control’ Review: A Struggle Down Under
Conflicts over political power and indigenous rights lie at the center of Sundance Now’s new drama.
Found article: ‘Gunda’ Review: A Pig’s Life
Victor Kossakovsky’s documentary takes a sow as its subject to tell the story of agriculture and industrial farming.
Found article: Congress Makes It Official: Greg LeMond Is Golden
A rare honor is bestowed on the American cycling legend, reasserting his legacy in the sport.
Found section: Opinion at https://www.wsj.com/print-edition/20201211/opinion
Found article: Judge Sullivan’s Final ‘Verdict’
A presidential pardon prompts a norm-busting outburst from the bench.
Found article: ‘Happy Holidays’ Is Pro-Christmas
Advent and Epiphany are also holy days. Why not call them all what they are?
Found article: A Shot (Instead of Two) at Saving Lives
Waiting on the Covid booster would allow more people to be vaccinated sooner.
Found article: ‘Brazil That Never Was’ Review: His Own Private Amazon
A British neurologist explores his passionate identification with the reckless adventurer Percy Harrison Fawcett.
Found article: There’s More to Politics Than Policy
There’s no substitute for finding candidates who capture voters’ hearts. No, not even a 30-point plan.
Found article: Biden Can Make American Trade Deals Great Again
Unions might resist, but most Democrats now agree that economic isolation is a losing proposition.
Found article: Now Is as Good a Time as Any for the U.S. to Quit Afghanistan
Whether American forces stay or leave, the fighting is guaranteed to continue. Why not cut our losses?
Found article: Notable & Quotable: Senate Snitching
‘He installed a trusted former aide, Max Young, to ‘embed’ in the Judiciary Committee to make sure the hearings didn’t go off the rails.’
Found article: Breaking Up Facebook
The feds want a do-over for acquisitions they previously approved.
Found article: AOC’s Latest Achievement
Goya honors Alexandria Ocasio-Cortez as Employee of the Month.
Found article: Texas Tries an Election Long Shot
Can a state be harmed by the way other states conduct their elections?
Found article: Gov. Jindal, Health-Care Policy and the GOP
Expand the choices.
Found article: Consider Student Loans as Any Investment
The only thing that hits parents harder than college-loan debt is sacrificing for 20 years to avoid borrowing in the first place.
Found article: Emmanuel Macron’s Secular Solution Isn’t Working With Islam
A more promising path for integrating Islam into French society is the laïcité positive advocated by former president Nicolas Sarkozy.
Found article: The GOP Abandoned Fiscal Discipline Long, Long Ago
The redistributionists on the right long ago lost the ability to cry foul over spending.
Found section: Business & finance at https://www.wsj.com/print-edition/20201211/business
Found article: Mortgage Originations Are on Pace for Best Year Ever
Record-low interest rates and strong demand for homes during the Covid-19 pandemic are boosting mortgage lenders.
Found article: Brent Crude Rises Above $50 as Oil’s Turnaround Continues
Thursday’s advance continues a monthslong rebound from oil’s April lows around $19 a barrel during global economic shutdowns.
Found article: Break Up Facebook? It’s Complicated, Tech Experts Say.
Splitting off Instagram and WhatsApp from Facebook potentially would involve years of engineering and legal work, according to current and former employees.
Found article: Amazon Wants to Train 29 Million People to Work in the Cloud
The online giant’s latest effort, geared toward those who aren’t already employed by Amazon, aims to equip people around the world with the skills needed work in cloud-computing at employers seeking high-tech workers.
Found article: Disney+ Could Hit 260 Million Subscribers by 2024, CEO Bob Chapek Says
The entertainment giant’s roughly year-old, $6.99-a-month flagship streaming service has grown rapidly, to 86.8 million subscribers globally.
Found article: U.S. Air Force Gives Lift to Flying Taxis
Flying taxis, which one day may whisk passengers around town without pilots, are getting a boost from the U.S. military, with the Air Force’s safety endorsement of an electric-powered vehicle similar to a helicopter.
Found article: Ferrari CEO Louis Camilleri Resigns After Bout With Covid-19
Louis Camilleri, who took over Ferrari in 2018, stepped down unexpectedly from the Italian luxury sports-car maker after being hospitalized.
Found article: CEOs Pledge One Million Jobs for Black Americans
More than 30 CEOs are backing OneTen, a nonprofit organization that has raised more than $100 million in seed funding and aims to train Black candidates for corporate roles.
Found article: The IKEA Catalog Defined Home for Millions. Now It’s Gone.
IKEA’s decision to stop publishing its annual catalog marks the end of a tome that served as much as an aspirational lifestyle guide for millions as it did a marketing tool for the flat-pack furniture giant.
Found article: Gilead to Pay $1.4 Billion for Hepatitis Drugmaker MYR GmbH of Germany
The all-cash deal will deepen Gilead’s portfolio of hepatitis treatments and marks its latest acquisition to increase revenue growth.
Found article: Costco Keeps Drawing Homebound Shoppers
Costco reported strong quarterly sales as homebound Americans continue to spend more on food, homegoods and fitness products during the pandemic.
Found article: Lululemon Gets Another Sales Bump From Cooped-Up Shoppers
The retailer is emerging as one of the winners from the Covid-19 health crisis as pandemic-weary customers snap up its athletic apparel and other comfy gear.
Found article: Zuckerberg’s Deal Making for Facebook Is Central to Antitrust Cases
The FTC’s lawsuit against Facebook puts the regulatory spotlight on the deal-making history of CEO Mark Zuckerberg, who personally engineered the tech giant’s biggest acquisitions.
Found article: The U.S. vs. Facebook: Why Is the FTC Filing an Antitrust Lawsuit Now?
Criticism that the tech giant’s has faced over issues ranging from privacy violations to the spread of false or harmful content on its platforms in recent years has created a political climate more receptive to government action.
Found article: Facebook Case Presents Test for Antitrust Enforcement, Federal Courts
The case comes amid a national debate about whether a handful of tech companies have become too dominant—and whether antitrust enforcers should have done more to police powerful firms and concentrated industries.
Found article: Google, Amazon Fined $163 Million as France Takes Hard Line on Privacy
France’s privacy watchdog issued the fines to Google and Amazon, saying they improperly collected information about website visitors, as European regulators haggle over such sanctions.
Found article: Missed Tesla’s 12,551% Rise? Don’t Feel So Bad
Investors can’t afford regret. Every day brings new opportunities to become very rich, most of which we are bound to miss.
Found article: U.K. Allows Banks to Resume Dividend Payments
The Bank of England’s Prudential Regulation Authority says it isn’t necessary to extend precautionary action taken as the pandemic began in March.
Found article: SEC Enforcement Chief to Depart as Regulator Prepares for Turnover
The Securities and Exchange Commission’s top enforcement official will step down this month after four years in the role, a move that allows the incoming Biden administration to put its own stamp on Wall Street policing.
Found article: CFTC Chairman Heath Tarbert to Resign, Remain at the Derivatives Regulator
The head of the nation’s derivatives regulator will step down from his leadership role while remaining a commissioner, leaving control of the five-person panel in Republican hands, at least for now.
Found article: U.S. Stocks Wobble as Economic Picture Dims
The Dow and S&amp;P 500 slipped after jobless claims jumped sharply last week, though energy shares rallied.
Found article: Corporate America’s Borrowing Binge May Be Ending
Wall Street bankers and investors are preparing for a sharp drop in corporate bond sales next year.
Found article: Chinese Chip Maker in Default on $2.5 Billion of Dollar Bonds
Tsinghua Unigroup, a key player in China’s push for self-reliance in semiconductors, is in default on nearly $2.5 billion of international bonds, in the latest instance of financial stress in China spilling over into global markets.
Found article: Metals Markets Steel for Price Rises as Australia Pushes to Save Cultural Sites
Rio Tinto’s destruction of two ancient caves in Australia to expand an iron-ore mine could have ramifications for global commodity markets if local lawmakers intensify scrutiny of mining activities that threaten heritage sites.
Found article: Facebook’s Profile Will Change, Win or Lose
Facebook shares have been little affected by the most recent government lawsuits. But that doesn’t mean Facebook will skate by, and the company needs a fresh strategy to keep growing.
Found article: Health Insurers Can Ride Out the Coronavirus, Amazon Storms
Considering what has failed to sink health-insurance stocks in 2020, the shareholder outlook for next year should be rosy.
Found article: In Europe, Monetary Policy Is All About Giving Banks Free Money
With bond yields at record lows and market liquidity running free, central banks are searching for new ways to stimulate the economy. In Europe, officials are pointing in a new direction: Let banks make more money.
Found article: Goodbye Covid, Hello Inflation?
The timing of the vaccine rollout and the anniversary of the pandemic’s initial shock to the U.S. economy could make things awkward for the Federal Reserve as inflation rises above its target.
Found section: Mansion at https://www.wsj.com/print-edition/20201211/mansion
Found article: Before Covid, Golf Club Communities Were in the Rough. Now They’re Seeing Green.
Golf homes are experiencing a surge in demand, as more players hit the links during the pandemic. Will it last?
Found article: This Couple Had One Loose End to Tie Up Before Retirement: Where to Park Their RV
Rick and Susan Turner embarked on a journey to find the right Florida resort to station their 45-foot motorcoach for winter.
Found article: For One California Couple, Their New Tahoe Home Served as an ‘Antidote to L.A.’
Wes and Julie Nichols used wood and ebonized steel to help their Truckee house blend into its surroundings.
Found article: After Coronavirus Hit, Greenwich’s Real-Estate Market Staged a Major Comeback
New Yorkers looking for lower property taxes and more space amid the pandemic are giving the tony Connecticut town a big boost.
Found article: Historic Homes are Full of Charm—and Plenty of Problems, Too
Do you have a loved one who’s purchased one of these timeless (and time-sucking) properties? Here’s how to help them cope.
Found article: First He Sold Ferraris. Then Jonathan Waxman Learned to Cook
How a matchbook kicked off the celebrity chef’s culinary career.
Found article: Celebrities Aren’t the Only Buyers After Montecito Real Estate
The tony beach town has seen an influx of young families moving from big cities amid the coronavirus pandemic.
Found article: Ellen DeGeneres Is Behind $49 Million Deal for Dennis Miller’s Montecito Estate
The comedian and talk show host bought the property in an off-market deal in September.
Found article: For Some Sellers, Permanent Fixtures Remain Forever Theirs
Real-estate agents recall clients who took built-in appliances and toilet seats along with them.
Found article: In California’s Silicon Valley, an Architecturally Striking Home Asks $33 Million
The seller interviewed 200 architects before hiring a firm to design the modern, sculptural house.
Found article: L.A. Megamansion Developer Nile Niami Places Spec Home in Bankruptcy
The developer was facing the prospect of a foreclosure sale at the West Hollywood mansion.
Found article: A New York Penthouse 1,000 Feet Above Street Level Fetches $50 Million-Plus
The contract for an apartment at 111 West 57th Street is one of the biggest signed in New York since the start of the pandemic.
Found section: More at https://www.wsj.com#
Synthesizing mastheadImage
Fetching https://www.wsj.com/articles/airbnb-...p_wsj&ru=yahoo
Fetching https://www.wsj.com/articles/the-off...p_wsj&ru=yahoo
Fetching https://www.wsj.com/articles/fda-adv...p_wsj&ru=yahoo
Fetching https://www.wsj.com/articles/ecb-exp...p_wsj&ru=yahoo
Fetching https://www.wsj.com/articles/china-x...p_wsj&ru=yahoo
Fetched https://www.wsj.com/articles/china-x...p_wsj&ru=yahoo in 0.205515 seconds
Parsed https://www.wsj.com/articles/china-x...p_wsj&ru=yahoo in 0.1 seconds
Processing images...
Fetched https://www.wsj.com/articles/fda-adv...p_wsj&ru=yahoo in 0.271297 seconds
Fetching https://images.wsj.net/im-269728?width=620&size=1.5
Fetched https://www.wsj.com/articles/the-off...p_wsj&ru=yahoo in 0.367530 seconds
Parsed https://www.wsj.com/articles/fda-adv...p_wsj&ru=yahoo in 0.1 seconds
Processing images...
Skipping invalid link: mailto:Tom.Burton@wsj.com
Parsed https://www.wsj.com/articles/the-off...p_wsj&ru=yahoo in 0.1 seconds
Skipping invalid link: mailto:jared.hopkins@wsj.com
Processing images...
Skipping invalid link: mailto:tom.burton@wsj.com
Fetched https://images.wsj.net/im-269728?width=620&size=1.5 in 0.134338 seconds
Skipping invalid link: mailto:jared.hopkins@wsj.com
Fetched https://www.wsj.com/articles/ecb-exp...p_wsj&ru=yahoo in 0.428022 seconds
Fetching https://si.wsj.net/public/resources/...1210130522.jpg
Recursion limit reached. Skipping links in https://www.wsj.com/articles/fda-adv...p_wsj&ru=yahoo
Fetched https://www.wsj.com/articles/airbnb-...p_wsj&ru=yahoo in 0.457112 seconds
Parsed https://www.wsj.com/articles/ecb-exp...p_wsj&ru=yahoo in 0.1 seconds
Processing images...
https://www.wsj.com/articles/fda-adv...p_wsj&ru=yahoo saved to /private/var/folders/vv/h8jjxczd3tl94vvx0fcd5cc80000gn/C/calibre_5.6.0_tmp_ii2h11tu/22naznt6_plumber/feed_0/article_0/index.xhtml
Fetching https://images.wsj.net/im-271207?width=620&size=1.5
Fetching https://www.wsj.com/articles/weekly-...p_wsj&ru=yahoo
Fetching https://images.wsj.net/im-269730?width=620&size=1.5
Parsed https://www.wsj.com/articles/airbnb-...p_wsj&ru=yahoo in 0.2 seconds
Processing images...
Fetching https://images.wsj.net/im-271268?width=620&size=1.5
Fetched https://si.wsj.net/public/resources/...1210130522.jpg in 0.181951 seconds
Downloaded article: FDA Panel Endorses Covid-19 Vaccine from https://www.wsj.com/articles/fda-adv...p_wsj&ru=yahoo
Fetching https://images.wsj.net/im-270821?width=620&size=1.5
Fetched https://images.wsj.net/im-271207?width=620&size=1.5 in 0.085316 seconds
Skipping invalid link: mailto:tom.fairless@wsj.com
Skipping invalid link: mailto:tom.fairless@wsj.com
Recursion limit reached. Skipping links in https://www.wsj.com/articles/ecb-exp...p_wsj&ru=yahoo
https://www.wsj.com/articles/ecb-exp...p_wsj&ru=yahoo saved to /private/var/folders/vv/h8jjxczd3tl94vvx0fcd5cc80000gn/C/calibre_5.6.0_tmp_ii2h11tu/22naznt6_plumber/feed_0/article_2/index.xhtml
Fetched https://images.wsj.net/im-271268?width=620&size=1.5 in 0.079952 seconds
Fetching https://www.wsj.com/articles/covid-1...p_wsj&ru=yahoo
Fetched https://images.wsj.net/im-269730?width=620&size=1.5 in 0.109313 seconds
Fetched https://images.wsj.net/im-270821?width=620&size=1.5 in 0.078498 seconds
Downloaded article: ECB Expands Stimulus Program to Prop Up Pandemic-Hit Economy from https://www.wsj.com/articles/ecb-exp...p_wsj&ru=yahoo

[briankaemingk]

Hi! I'm having trouble with this recipe as well. Sharing the output that I get for running the non-free recipe. I tried with the free, and it does not seem to work either. Happy to help debug/share credentials if necessary. Thanks!

ebook-convert "The Wall Street Journal.recipe" "./pdfs/_wsj_.epub" --username="USERNAME" --password="PASSWORD" --output-profile="kobo" --debug-pipeline=./pdfs/
Conversion options changed from defaults:
test: None
debug_pipeline: u'./pdfs/'
output_profile: u'kobo'
1% Converting input to HTML...
InputFormatPlugin: Recipe Input running
Trying to get latest version of recipe: wsj
Using downloaded builtin recipe
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Starting login process...
Sending login request...
Performing login callback...
1% Fetching feeds...
1% Got feeds from index page
Python function terminated unexpectedly: No articles found, aborting
Traceback (most recent call last):
File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 187, in main
return run_entry_point()
File "/Applications/calibre.app/Contents/Resources/Python/lib/python2.7/site.py", line 121, in run_entry_point
return getattr(pmod, func)()
File "site-packages/calibre/ebooks/conversion/cli.py", line 401, in main
File "site-packages/calibre/ebooks/conversion/plumber.py", line 1110, in run
File "site-packages/calibre/customize/conversion.py", line 246, in __call__
File "site-packages/calibre/ebooks/conversion/plugins/recipe_input.py", line 139, in convert
File "site-packages/calibre/web/feeds/news.py", line 1028, in download
File "site-packages/calibre/web/feeds/news.py", line 1207, in build_index
ValueError: No articles found, aborting

[mjfriedman]

Some days the recipe works for me. Other days it gets stuck. If I kill the task and manually instruct Calibre to fetch WSJ Print it sometimes works on the next try; sometimes it takes a few tries. Rarely I can't get it to work at all. Typically an update to Calibre makes the recipe work reliably for a few days before the cycle begins anew...

[aritafari]

Originally Posted by mjfriedman

Some days the recipe works for me. Other days it gets stuck. If I kill the task and manually instruct Calibre to fetch WSJ Print it sometimes works on the next try; sometimes it takes a few tries. Rarely I can't get it to work at all. Typically an update to Calibre makes the recipe work reliably for a few days before the cycle begins anew...

I am having this issue also with WSJ paid edition.
It hardly ever downloads all sections (articles) on first try. I have to run the recipe a second/third time before getting the whole newspaper.
It's the only newspaper that seems to exhibit this behavior.