"HTTP Error 504" On Scheduled News Download.

[Hotrod_Linc]

Problem

I have a news recipe based of the basic one provided by Calibre that downloads four news sources a few days a week in the morning, however one of the four has recently been unable to download. Every time my scheduled download occurs it provides the following error: “Failed feed: 404 Media, HTTP Error 504: Gateway Time-out” (404 Media is the news source, not the usually associated error code). This error only occurs when the scheduled news feed job occurs, manually pulling the recipe with the “Download all scheduled news sources” option downloads the same recipe correctly every time.



Troubleshooting Steps & Info

- Calibre is running 24/7 in a docker container in a server
- The RSS feed of 404 media is a private premium feed
+ This RSS feed once in the past was unable to be pulled from as the
owners system believed Calibre to be a scraping bot, I communicated
with the owner and this error has not recurred
+ Also, this feed does work, as manually triggering the recipe does function,
in addition to validating the RSS feed via other tools.
- The recipe is programmed to initiate every Monday, Wednesday, Friday, and
Sunday after 0400 local time.
- The Error message appears in the epub itself generated by Calibre.



Code

Spoiler:

#!/usr/bin/env python
# vim:fileencoding=utf-8
from calibre.web.feeds.news import BasicNewsRecipe

class AdvancedUserRecipe1730421674(BasicNewsRecipe):
title = 'The Hotrod_Linc Daily News Brief'
description = 'A collection of news sources to provide insight into worldly happenings.'
oldest_article = 7
max_articles_per_feed = 5
auto_cleanup = True
no_stylesheets = True

# Set the custom image for the cover
# cover_url = r'file://C:\Web_Development\Assorted%20/Daily%20News%20cover-1.jpg' # Updated path

feeds = [
('404 Media', 'https://404.feed.press/private?key='), #Full RSS Removed for posting
('Canadaland', 'https://www.canadaland.com/feed/'),
('Internet Archive Blog', 'https://blog.archive.org/feed/'),
('Jacobin Blog', 'https://jacobin.com/feed/'),
]

def get_cover_url(self):
cover_url = 'file:////config/Calibre Library/Covers/Daily News cover-1.jpg'
return cover_url

Finishing Thoughts

If anyone has recommendations on how to fix this issue, I would certainly appreciate any suggestions. Also If there is a log file I can check to get more information about the failed news pulls I would love to know its location. I found a old thread on the forum that briefly mentioned this issue, but I couldn’t ascertain a solution. https://www.mobileread.com/forums/sh...HTTP+Error+504

[kovidgoyal]

That's a server error, not something that can be fixed in calibre. As for logs you can access the logs for all jobs including news fetches by clicking the jobs button in the bottom right corner of the calibre window (I am assuming you are talking of the calibre GUI).