Beta testers for calibre 3.0 needed

[PandathePanda]

Another one I got:

Fetching metadata source hashes...
Traceback (most recent call last):
  File "site-packages/calibre/gui2/notify.py", line 76, in __call__
  File "site-packages/dbus/proxies.py", line 145, in __call__
  File "site-packages/dbus/connection.py", line 651, in call_blocking
DBusException: org.freedesktop.DBus.Error.ServiceUnknown: The name :1.164 was not provided by any .service files

[kovidgoyal]

@PandaThePanda: This first error looks like a mixing up of beta binary with normal source. The second error is harmless and just means that notifications will not work since no notifier daemon was found.

[davidfor]

Originally Posted by kovidgoyal

Well, it was only a guess. The other thing you could try is adding some print statements to see if the client is actually receiving all the data or aborting in between. You can change line 310 to disable the use of sendfile and then add the prints to the write() method to see what is being written to the socket.

That shows me something. But, I can't see what is wrong with it.

I also added a print in the send() in loop.py to see if there was an exception. I got:

send - exception - e: [Errno 10053] An established connection was aborted by the software in your host machine

That seems to happen on the last packet of the file sent.

Wireshark tells me roughly the same thing. The device is sending packets back with the reset bit set. I assume that's why the above error happens. I'll have to hunt through it and see what else I can find.

[Martina Schein]

Today I tried to connect my Kobo Aura H2O to the Caliber server. With my smartphone that worked well.

The simple page link is displayed after the call, but after a tip on this link nothing happens. No login is displayed, only a blank white page. The screenshots illustrate this.
I think, however, that this is due to Kobo. Even after years the browser still has the beta status.

I do not even need this function on my reader, because I copy the eBooks by means of Calibre via USB.
Now and then I like to play with my gadgets and try what is possible.

I have tested it with Calibre 2.99.9.

[Saioko]

I had Calibre for several years, it worked ok. But today it simply refused to start up. There were no changes - I didn't update Calibre or install any updates before the failure, then I tried updating Calibre and installing latest updates for my Linux, but nothing changed.
Here is what I get:

calibre, version 2.99.9
Traceback (most recent call last):
  File "/home/katya/calibre/src/calibre/gui2/main.py", line 292, in initialize_db_stage2
    self.start_gui(db)
  File "/home/katya/calibre/src/calibre/gui2/main.py", line 227, in start_gui
    main = self.main = Main(self.opts, gui_debug=self.gui_debug)
  File "/home/katya/calibre/src/calibre/gui2/ui.py", line 160, in __init__
    ac = self.init_iaction(action)
  File "/home/katya/calibre/src/calibre/gui2/ui.py", line 174, in init_iaction
    ac = action.load_actual_plugin(self)
  File "/home/katya/calibre/src/calibre/customize/__init__.py", line 614, in load_actual_plugin
    ac = getattr(importlib.import_module(mod), cls)(gui,
  File "importlib/__init__.py", line 37, in import_module
  File "/home/katya/calibre/src/calibre/gui2/actions/catalog.py", line 13, in <module>
    from calibre.gui2.tools import generate_catalog
  File "/home/katya/calibre/src/calibre/gui2/tools.py", line 16, in <module>
    from calibre.gui2.convert.single import NoSupportedInputFormats
  File "/home/katya/calibre/src/calibre/gui2/convert/single.py", line 20, in <module>
    from calibre.gui2.convert.search_and_replace import SearchAndReplaceWidget
  File "/home/katya/calibre/src/calibre/gui2/convert/search_and_replace.py", line 11, in <module>
    from calibre.gui2.convert.search_and_replace_ui import Ui_Form
  File "/home/katya/calibre/src/calibre/gui2/convert/search_and_replace_ui.py", line 155, in <module>
    from regex_builder import RegexEdit
  File "/home/katya/calibre/src/calibre/gui2/convert/regex_builder.py", line 18, in <module>
    from calibre.ebooks.conversion.search_replace import compile_regular_expression
  File "/home/katya/calibre/src/calibre/ebooks/conversion/search_replace.py", line 7, in <module>
    import regex
  File "site-packages/regex/__init__.py", line 383, in <module>
  File "site-packages/regex/_regex_core.py", line 21, in <module>
  File "/home/katya/calibre/src/calibre/constants.py", line 209, in __getitem__
    raise KeyError('No plugin named %r'%name)
KeyError: "No plugin named '_regex'"

[DoctorOhh]

Originally Posted by Saioko

I had Calibre for several years, it worked ok. But today it simply refused to start up. There were no changes - I didn't update Calibre or install any updates before the failure, then I tried updating Calibre and installing latest updates for my Linux, but nothing changed.
Here is what I get:

You upgraded to the Calibre 3 Beta version, was this your intent? If so then this needs to be moved to the beta thread.

Moderator Notice
Went ahead and moved post to the Beta 3 thread.

[Saioko]

Originally Posted by DoctorOhh

You upgraded to the Calibre 3 Beta version, was this your intent? If so then this needs to be moved to the beta thread.

The problem appeared before updating, and before that I updated over a month ago, so I am not sure if it is an issue with Calibre or if something crashed in Linux.

[kovidgoyal]

@Saioko: You appear to be mixing the binary install of the beta with source code that refers to the non-beta. Delete /home/katya/calibre/

ans you should be fine.

[kovidgoyal]

@Martina: Yes the Kobo browser has lots of problems with the new server, see previous discussion with davidfor and m. s.

[kovidgoyal]

@davidfor: If it is happening on the last packet, it sounds like the Kobo browser is not doing a proper TCP shutdown. i.e. it is not sending FIN followed by waiting for ACK, instead it is just closing the connection causing the kernel to send RST.

The question is why is it happening with the new server and not the old? Does removing the Content-Length header make a difference?

[kovidgoyal]

On the theory that the Kobo browser is probably using some ancient version of WebKit, I tried out the mobile page with the version of WebKit bundled with calibre, which is also pretty ancient, but downloading works fine there. See attached script which can be run with calibre-debug and will save the downloaded file as raw.epub in the current directory

from __future__ import absolute_import, division, print_function, unicode_literals

from PyQt5.Qt import *


class Page(QWebPage):

    def __init__(self, p):
        QWebPage.__init__(self, p)
        self.setForwardUnsupportedContent(True)
        self.downloadRequested.connect(self.download_requested)
        self.unsupportedContent.connect(self.unsupported_content)

    def javaScriptConsoleMessage(self, msg, lineno, msgid):
        print(msg, lineno, msgid)

    def unsupported_content(self, reply):
        self.reply = r = reply
        r.error.connect(self.on_error)
        r.finished.connect(self.on_finish)

    def download_requested(self, request):
        self.unsupported_content(self.networkAccessManager().get(request))

    def on_finish(self):
        with open('raw.epub', 'wb') as f:
            f.write(self.reply.readAll())

    def on_error(self, error):
        print('error:', error)


app = QApplication([])
w = QWebView()
page = Page(w)
w.setPage(page)
w.setUrl(QUrl('http://localhost:8080/mobile'))
w.show()
app.exec_()

[davidfor]

Originally Posted by kovidgoyal

@Martina: Yes the Kobo browser has lots of problems with the new server, see previous discussion with davidfor and m. s.

The UI seems to be fine here. Last night I saw the javascript-free prompt and pressing "here" took me to the mobile version. The list displayed fine with the books and details. I did do one search and reordered the list without any problems. The only problem I see is downloading books.

[davidfor]

Originally Posted by kovidgoyal

@davidfor: If it is happening on the last packet, it sounds like the Kobo browser is not doing a proper TCP shutdown. i.e. it is not sending FIN followed by waiting for ACK, instead it is just closing the connection causing the kernel to send RST.

From the packet capture, it might have terminated connection after the second last packet. And the last produced the error because of that. But, I'll have to look at that again.

The question is why is it happening with the new server and not the old?

I ran a packet capture with the 1.48 server last night, and couldn't see any differences, other than the fact that it works. But, watching it did tell me something. When I tap the link, the browser requests the file and displays prompt about whether to download or not. If I hit yes, it seems to start the download again.

Does removing the Content-Length header make a difference?

I'll try this tonight. I'll also try your script and do a packet capture just to see what is happening.

One other thing I tried last night was older firmware. Kobo broke and fixed downloading files larger than 2MB recently. I tested with a firmware version from before the breakage with the same result - download from the server didn't work.

[gbm]

After updating calibre to a new version on first run crashes firefox, have to close and restart to recover. After restart works normally.

Test conditions:
Server running on a Linux Mint 18.1 P.C using auto authentication, FireFox for Linux. The box for user name and password opens, enter, then press "OK" box for " The caibre web application has been updated. Do you want reload the site?" appears at this point Firefox will not accept and input on any tabs.

Works on Firefox for andoird and Firefox for windows.

bernie

[kovidgoyal]

@gbm: Doesn't happen on my linux Firefox. When the relaod question popup appears, I can still open pother tabs in firefox and interact with them.

As far as I can tell, if it does happen, it's a bug in firefox. The updated notification is created using the standard window.confirm() javascript API.

Most likely, you still have a single process firefox on your system so the dialog is application modal -- it will be fine when you update to multiprocess firefox.