Some unexpected Calibre behavior

[GeorgeYellow]

Today, I found that an old but stable and working Calibre installation suddenly ceased to fetch metadata.

The error was pervasive, with every metadata source:

ERROR: Download failed: Failed to download metadata. Click Show Details to see details
... 
  File "<string>", line 15, in <module>
ImportError: cannot import name _

In case I'm not the only person running an ancient Calibre, the solution is:
1. Add the single line `127.0.0.1 code.calibre-ebook.com` to `C:\Windows\System32\drivers\etc`

2. Rename or Delete the file `Calibre Settings\metadata-sources-cache.json'

The reason for this error are with the dynamic code execution that Calibre does when it checks updates. This mechanism does not check a version number so it will run python code intended for new Calibre releases on old versions.

That mechanism is in the Updates plugin at https://github.com/kovidgoyal/calibr...rces/update.py
Note that this plugin is expressly forbidden to be disabled in the Calibre UI!

The code will fetch `https://code.calibre-ebook.com/metadata-sources/hashes.json` (which, despite the file name, is `bzip2` compressed. It can be expanded with `bzcat`. If a hash in this download doesn't match the previous hashes, calibre will fetch the updated python code from `https://code.calibre-ebook.com/metadata-sources/<NAME>` (eg, google, amazon, etc)
Then, it will pass that code directly to `exec()`

I was worried when I first looked at this, because `code.calibre-ebook.com` has a strange certificate - fortunately it is self-signed so that only people with the private key can dynamically run Python code on every Calibre installation.

The initial error was because the new plugins all use the Python "throwaway" syntax in an import:

from calibre.utils.localization import _

Fortunately I'm not paranoid, and I don't believe that any actor would POSSIBLY compel a software developer to serve specific code to a specific IP range, and anyone who can think of counter examples is a conspiracy theorist.

[theducks]

localhost (127.0.0.1) is how Calibre talks to some of its pieces (worker process is a background piece).
This is a very common method.

[kovidgoyal]

Not sure what's so unexpected about that? You get the entire calibre program from the same server (group of servers), if someone compromised the developer (me) they could just as well serve you compromised calibre binaries in the first place.

And it uses a self signed certificate deliberately, precisely so no third parties other than me are in the network trust chain.

FYI various network facing bits of calibre are dynamically updated recipes, get books, metadata sources etc etc. and have been for a decade plus.

[richorlin]

How do you make the correction for Calibre running on a Mac? Thanks

[JSWolf]

Originally Posted by richorlin

How do you make the correction for Calibre running on a Mac? Thanks

What correction is needed?

[richorlin]

Originally Posted by JSWolf

What correction is needed?

The solution in the first post.

[DNSB]

MacOS uses the same directory name as Windows and Linux and multiple other OS. Look for the etc folder in the private folder (goto folder: /private/etc). Inside that folder, look for the hosts file.

Personally, I can't see how that line was added to etc by the original poster since etc is a directory/folder and not an editable file. You can add that information to the hosts file inside the etc folder (under MacOS, open Terminal and enter sudo nano /etc/hosts which should do the job).
.
Not to mention that all the OP was doing was redirecting to the local machine (127.0.0.1 is the local computer) so fetching the new information would always fail.