Python function terminated unexpectedly...

[Kindle3w_Pdfs]

I think Kovad is right... the HIPS event log of my antivirus (Comodo) shows that it blocks Calibre from modifying two "HKLM/.../Tcpip/Parameters" values in the Windows Registry. (And maybe it would block more, if I unblock those to allow Calibre to proceed further.) Why does Calibre need to modify those values?

[kovidgoyal]

calibre does not modify those registry values. calibre does not modify the windows registry at all, with two exceptions:

1) The msi installer use the windows registry
2) On first run calibre uses the windows registry to register itself with windows default programs.

[BetterRed]

FWIW - I don't have a HKLM\SOFTWARE\Microsoft\Tcpip\Parameters key in the Win10 registry. All I have is HKLM\...\Tcpip\Current Version, with (Default value) unset.

Nor do I have Comodo of course.

BR

[Kindle3w_Pdfs]

@Kovad: You said above that Calibre never modifies the Registry's .../Tcpip/Parameters, and I assume you know Calibre's code well enough that that's surely true. But what about Python (or some other third party library, if any, that the Python function might call)? Do you also know Python well enough to be certain it doesn't modify those Registry values? Is there another explanation for why the Comodo event log says Calibre attempted to modify those values? My firewall reports that Calibre-debug attempts to access the internet soon after Calibre-debug is launched, and that Calibre does too (if I set Comodo to allow Calibre to run freely). If Calibre calls a Python function for net access, that could be where the .../Tcpip/Parameters values are modified.

[kovidgoyal]

You are asking me to prove a negative, that is impossible. I'll just say that since calibre-debug and calibre run the exact same code, the only difference being that calibre-debug adds some extra print statements, that your antivirus program claims that one modifies the registry and the other doesn't, strains credulity, to put it mildly.

Antivirus programs are giant steaming piles of ...

I have no idea why comodo says what it says. I wouldn't be surprised if the idiotic antivirus program tries to modify the registry itself and gets confused.

In any case, you have three options

1) Get a better antivirus program, or even better, stop using antivirus at all, since the entire category of software is just a giant snake oil bottle.

2) Exclude calibre.exe from your antivirus

3) Keep using calibre-debug.exe and hope and pray that your antivirus does not suddently decide to start quarantining it.

I am not going to spend any more of my time trying to debug problems caused by antivirus software

[DoctorOhh]

Originally Posted by Kindle3w_Pdfs

My firewall reports that Calibre-debug attempts to access the internet soon after Calibre-debug is launched, and that Calibre does too (if I set Comodo to allow Calibre to run freely).

Calibre does access the internet as explained on their site to collect usage information. This connection does not require any change in the Registry.

"Usage statistics are collected whenever a user starts calibre. Every calibre installation has a unique ID, this ID remains unchanged by upgrades and even an uninstall/re-install. This ID is used to collect usage statistics. Only this ID is stored, no other identifying information is collected."

[Kindle3w_Pdfs]

@Kovad: I don't believe I asked you to prove a negative that's impossible to prove. I merely asked whether you know the internals of the Python functions called (directly or indirectly) by Calibre well enough to be certain that Python doesn't modify those Registry values.

Regarding the different Comodo behavior for Calibre and Calibre-debug, it should not be attributed to a bug in Comodo. I had a custom Comodo HIPS rule set up for Calibre but not for Calibre-debug. Note: The custom rule didn't prevent an earlier version of Calibre from working properly, and when I removed the custom rule Calibre 3.13 ran okay.

It seems reasonably likely, from my perspective, that an update of a third party library (probably Python) is behaving in a way you don't fully understand, and that you're blaming users' antivirus software in such cases. Perhaps you're more familiar with the internals of an older version of Python that didn't behave this way; I wouldn't expect you to spend time studying the details of every update made to Python. I don't mean to give offense, and if I'm wrong I hope you can appreciate that this theory is plausible from my perspective.

Question: In which Calibre source code module(s) is the code involved in accessing the internet during Calibre startup? Since Calibre is open source, maybe I can find the problem, if I know where to start looking.

@DoctorOhh: Whether or not changes to the Registry are logically 'required' (to collect usage statistics) is irrelevant. What is relevant (to the antivirus defense) are: (1) Does the app makes changes to the Registry? (2) Does the app ask the operating system to grant a privilege that would allow the app to make Registry changes later? The Python function that issued the error message when blocked by Comodo may have requested more access to the Registry than is really needed to send usage statistics. See for example: https://stackoverflow.com/questions/...-access-denied (where the programmer may have inadvertently used a Registry access parameter that asks for an unnecessarily high privilege.)

[BetterRed]

Originally Posted by DoctorOhh

Calibre does access the internet as explained on their site to collect usage information. This connection does not require any change in the Registry.

"Usage statistics are collected whenever a user starts calibre. Every calibre installation has a unique ID, this ID remains unchanged by upgrades and even an uninstall/re-install. This ID is used to collect usage statistics. Only this ID is stored, no other identifying information is collected."

- and at startup calibre also looks for any updates, to itself and installed optional plugins.

@Kindle3w_Pdfs - if you think your problem is specific to version 3.13, why don't you revert to 3.12 and see if the 'problem' persists. Previous versions are here. To install a previous version you will first need to uninstall what you have, doing that won't affect your libraries or configuration settings etc.

Curious - why do you suspect Python and not say Qt or libmtp?

Do you have any suggestions as why I don't see the "HKLM/.../Tcipip/Parameters" registry key on my two Windows computers on which calibre is installed? One is Win 10 Pro 1703, the other is Win 7 Pro SP1 - both are up to date with latest patches etc. The Win 10 system has latest calibre, the Win 7 system has calibre 3.7.

BR

[PeterT]

Windows 7 --> HKLM/SYSTEM/CurrentControlSet/services/Tcpip/Parameters

Windows 10 --> Big changes;not used

[Kindle3w_Pdfs]

@BetterRed: I don't just suspect version 3.13 of Calibre, because my previous version was pretty old. The reasons I've been focusing on Python rather than other third party libraries such as Qt or libmtp are (1) the error message when Comodo blocked Calibre from launching referred to a Python function terminating unexpectedly, and (2) I didn't know what other third party libraries besides Python were used by Calibre. The Comodo event log's references to attempts to access ".../Tcpip/Parameters" in the Windows Registry is a hint that the function that triggered Comodo was net-related, so hopefully we can eliminate from consideration any libraries that never access the net. (Does Qt ever access the net?) By the way, the version of Windows on my pc is Vista, and where I wrote "..." in the ".../Tcpip/Parameters" string I wasn't being literal; I was eliding part of a much longer string.

[BetterRed]

Re Win 7, I searched for Tcpip, didn't look at where it was found

I didn't see a 'Parameters' subkey, but that machine is at the end of a satellite link (3 sec latency delay) so I may have mis-keyed/clicked when I opened the key.

Added - aaah Vista, a different can of worms entirely - I skipped it, also skipped 8. And Vista is past its use by date - April 2017.

BR

[AnotherCat]

I haven't followed all your arguments for your case (we no longer have any pre-Windows 10 machines here) but to me, on first flush, I would have thought the following.

If one has confidence that the Calibre package contains no threats (there is no world experience of it doing so and I think it a safe assumption that it doesn't as long as obtained from the official site) and if Calibre starts if you remove Comodo (and all other Comodo products if you are using others), then it would seem to me that it is a Comodo issue and if you wish to remain a user of Comodo products then the matter should be addressed to them.

[Kindle3w_Pdfs]

@AnotherCat: I see no reason to suspect Comodo of misbehaving, because Calibre runs if I remove the custom Comodo rule I created that worked okay for an older version of Calibre. Removing the custom rule didn't involve uninstalling Comodo. Regarding confidence that Calibre will never contain malware or spyware because of people's experience that it has never contained malware or spyware in the past, I prefer not to trust unnecessarily. I never rely on antivirus' and firewall's whitelists or default settings, for example. Remember, developers who've been good for many years could be subject to bribery or blackmail at any time. Evil developers could cleverly behave well for many years if they're playing a long con, to develop a large user base before beginning any attacks.

[BetterRed]

@Kindle3w_Pdfs - there's a list of libraries that calibre uses here

I don't know which 3rd party libraries are involved in calibre accessing the 'net. I've seen mention of a sockets library, and afaik python's socket library makes use of the underlying platform sockets library - on Windows I would guess that's winsock.

Curious - why do you have multiple versions of python installed? I have several packages that I know use Python and/or Qt etc, but they each bundle the versions against which they've been tested, so I have no installed versions of python, Qt, etc as such.

BR

[BetterRed]

Originally Posted by Kindle3w_Pdfs

@AnotherCat: I see no reason to suspect Comodo of misbehaving, because Calibre runs if I remove the custom Comodo rule I created that worked okay for an older version of Calibre. Removing the custom rule didn't involve uninstalling Comodo. Regarding confidence that Calibre will never contain malware or spyware because of people's experience that it has never contained malware or spyware in the past, I prefer not to trust unnecessarily. I never rely on antivirus' and firewall's whitelists or default settings, for example. Remember, developers who've been good for many years could be subject to bribery or blackmail at any time. Evil developers could cleverly behave well for many years if they're playing a long con, to develop a large user base before beginning any attacks.

That's pretty close to being libellous - I'm done with this thread.

BR