calibre news fails on ubuntu 10.10 - why?

[butitsmyname]

Hi guys

I have two PCs on a D-Link router, one windows XP (for my wife) and Ubuntu 10.10. They're linked via Samba for local file exchange and so I can print on my wife's laser printer over SMB.

On Windows caibre performs perfectly - specifically the news downloads which is the bit I want (I'm learning spanish).

I've run KDE for years as my preferred desktop (first Red hat, then Suse and now ubuntu) but after installing calibre from the repository I got constant Knotify errors (weirdly "Knotify has crashed"?) as well as a lack of news. Re-installed Kubuntu *to a clean, formatted disk*, added calibre and the errors came back but no news.

So I formatted the disk again and installed Gnome 10.10 clean and added calibre from the repository.

News fails and if I start it from a terminal I can see the connection timeout errors and a mass of traceback from Python. I get the same empty EPub or Mobi files with just error messages about failed connections in them.

I looked at the recipes and tried some that tackle Print pages, HTML pages and RSS feeds. I get the same result with all of them.

My browsers, email, torrent, etc all work perfectly - no networking problems. I have a clean, stable install where everything works except calibre.

When I came here first Kovid advised me to try the binary version. I wish I hadn't. I checked the dependencies, checked the python version and then followed the instructions. The installation crashed spectacularly - locked my desktop, couldn't even switch to a terminal - and now, even after doing my best to clean up manually - the binary version won't even try to install.

It's not as if I'm a newbie. I've been running Linux since kernel 2.0 was released, but after three days of rebuilds and failures calibre appears to have defeated me.

Interestingly, when I installed calibre I also got problems with other Python based apps - which disappeared when I uninstalled it.

I stopped programming before Python existed so I'm definitely not up to figuring out the problem but if anyone can think of where to go from here I'd be delighted to hear it.

Typical traceback follows - thanks

Traceback (most recent call last):
File "/usr/lib/calibre/calibre/web/feeds/recipes/model.py", line 147, in get_builtin_recipe
return download_builtin_recipe(urn)
File "/usr/lib/calibre/calibre/web/feeds/recipes/collection.py", line 97, in download_builtin_recipe
return br.open_novisit('http://status.calibre-ebook.com/recipe/'+urn).read()
File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 205, in open_novisit
return self._mech_open(url, data, visit=False, timeout=timeout)
File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
response = UserAgentBase.open(self, request, data)
File "/usr/lib/python2.6/dist-packages/mechanize/_opener.py", line 191, in open
response = urlopen(self, req, data)
File "/usr/lib/python2.6/urllib2.py", line 409, in _open
'_open', req)
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
File "/usr/lib/python2.6/dist-packages/mechanize/_http.py", line 729, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.6/dist-packages/mechanize/_http.py", line 706, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 110] Connection timed out>
Fetch news from El Pais


Traceback (most recent call last):
File "/usr/lib/calibre/calibre/web/feeds/recipes/model.py", line 147, in get_builtin_recipe
return download_builtin_recipe(urn)
File "/usr/lib/calibre/calibre/web/feeds/recipes/collection.py", line 97, in download_builtin_recipe
return br.open_novisit('http://status.calibre-ebook.com/recipe/'+urn).read()
File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 205, in open_novisit
return self._mech_open(url, data, visit=False, timeout=timeout)
File "/usr/lib/python2.6/dist-packages/mechanize/_mechanize.py", line 236, in _mech_open
response = UserAgentBase.open(self, request, data)
File "/usr/lib/python2.6/dist-packages/mechanize/_opener.py", line 191, in open
response = urlopen(self, req, data)
File "/usr/lib/python2.6/urllib2.py", line 409, in _open
'_open', req)
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
File "/usr/lib/python2.6/dist-packages/mechanize/_http.py", line 729, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.6/dist-packages/mechanize/_http.py", line 706, in do_open
raise URLError(err)
URLError: <urlopen error [Errno 110] Connection timed out>
^CException in thread Thread-4 (most likely raised during interpreter shutdown):

[DoctorOhh]

Originally Posted by butitsmyname

So I formatted the disk again and installed Gnome 10.10 clean and added calibre from the repository.

I have no idea what the problem is, but I do know that it is recommended that you do a binary install outlined here and not a repository install.

[butitsmyname]

I did a binary install and it locked my PC - as well as failing

[janvanmaar]

so which calibre version are you using? I guess it may be pretty old if it is from repositories...

[butitsmyname]

0.7.18

[DoctorOhh]

Originally Posted by butitsmyname

I did a binary install and it locked my PC - as well as failing

The install locked your machine? Or did fetching the news lock your machine and fail? Have you tried this since rebuilding your machine?

0.7.18 is 16 versions back.

I wish you well.

[janvanmaar]

OK, it is older but not that old, I think I started using Calibre from around that version and news downloads always worked well... so that should not be the problem.

A am afraid I do not know what could be the problem though... I did both binary and from source install on Linux (Fedora KDE) and never got this problem - it must be something specific - but it is hard to say what. Clearly the error message says connection timeout but you state no problems with connection - so I am afraid I do not know...

[DoctorOhh]

Hopefully someone in the know will stop by with some insight into this problem.

[JSWolf]

Try the current Calibre. Nobody will help you get an old version like that going.

[butitsmyname]

OK, here is more data to chew on. I managed to run the binary script without the system lockup - this is the result:

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.6/urllib2.py", line 391, in open
response = self._open(req, data)
File "/usr/lib/python2.6/urllib2.py", line 409, in _open
'_open', req)
File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
result = func(*args)
File "/usr/lib/python2.6/urllib2.py", line 1170, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.6/urllib2.py", line 1145, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>


If I try to run the source install I get this:

--2010-12-21 14:05:59-- http://status.calibre-ebook.com/dist/src
Resolving status.calibre-ebook.com... 1.0.0.0
Connecting to status.calibre-ebook.com|1.0.0.0|:80... failed: Connection timed out.
Retrying.

--2010-12-21 14:06:21-- (try: 2) http://status.calibre-ebook.com/dist/src
Connecting to status.calibre-ebook.com|1.0.0.0|:80... failed: Connection timed out.
Retrying.

--2010-12-21 14:06:44-- (try: 3) http://status.calibre-ebook.com/dist/src
Connecting to status.calibre-ebook.com|1.0.0.0|:80... failed: Connection timed out.
Retrying.

And so it goes on until the end of time...In fact, basically, the installation scripts are failing in the same way as the recipes - is this interesting or informative at all? Is it something to do with Python? Surely it's using the same method of accessing the net as any other app?

[janvanmaar]

hm, a blind shot, check what do you have in your /etc/hosts... but it more looks like your DNS lookup does not work, so check your /etc/resolve.conf

[chaley]

The attempt to connect over the internet is failing. That means:
- you have an outgoing firewall that is blocking connections
- you use a proxy and have not configured it
- (edit) calibre thinks that there is a proxy when there isn't
- your dns isn't working

If you are connecting to the internet through a proxy, have you done what is needed to tell calibre? I know that calibre looks for proxy information in some 'standard' places, but I don't know where they are. If all else fails, you could use the http_proxy environment variable.

[butitsmyname]

Hi Javanmaar

/etc/hosts/
192.168.1.4 pondlife # Added by NetworkManager
127.0.0.1 localhost.localdomain localhost
::1 pondlife localhost6.localdomain6 localhost6
127.0.1.1 pondlife

# The following lines are desirable for IPv6 capable hosts
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

But if dns lookup wasn't working I wouldn't be able to reach any websites either, for example, or ping the bbc - which I can. It does make me wonder if it's a ipv6/ipv4 issue. I have ipv6 off because my router doesn't support it. Leaving it on encourages some apps to fail because if it's there they expect it to work.

[janvanmaar]

You can try commenting those ip6 lines out but I doubt that would be the problem.
It is still possible that some applications can connect and others not depending on their settings and settings of your firewall (as chaley mentioned). I somehow assumed that you turned firewall off as the first thing and do not use a proxy but just to make sure - is this really the case?
If so, then I would simply try pinging status.calibre-ebook.com from command line as the next step (ping status.calibre-ebook.com)

[butitsmyname]

It's odd. I can't download using Wget, it just times out, but it downloads perfectly well from a browser.

PING status.calibre-ebook.com (209.20.65.15) 56(84) bytes of data.
64 bytes from status.calibre (209.20.65.15): icmp_req=1 ttl=53 time=150 ms
64 bytes from status.calibre (209.20.65.15): icmp_req=2 ttl=53 time=139 ms
64 bytes from status.calibre (209.20.65.15): icmp_req=3 ttl=53 time=147 ms
64 bytes from status.calibre (209.20.65.15): icmp_req=4 ttl=53 time=147 ms
64 bytes from status.calibre (209.20.65.15): icmp_req=5 ttl=53 time=141 ms
64 bytes from status.calibre (209.20.65.15): icmp_req=6 ttl=53 time=146 ms

And I can ping the google code repositories as well.

The recipe problem is clearly a wget problem at heart. I don't know why but wget just doesn't connect. Mind you, the /etc/wgetrc file doesn't look too friendly either