Wireless Connection very buggy recently

[mobamoba]

I use Calibre Companion to connect wirelessly via the Android app and, with recent updates, I'm now rarely able to connect. I have no idea whether the problem is with Calibre or with CC and I'm not entirely sure how to tell where the problem lies.

The problem manifests like this: I'll connect via Wireless Connection from CC to Calibre; it works fine; I disconnect; next time I try to reconnect, it doesn't work. Generally the only solution is to restart Calibre and even that only works some of the time. Here's what I get in the Debug Device Detection log:

calibre 2.54 [64bit] embedded-python: True is64bit: True
Windows-8-6.2.9200 Windows ('64bit', 'WindowsPE')
('Windows', '8', '6.2.9200')
Python 2.7.9
Windows: ('8', '6.2.9200', '', 'Multiprocessor Free')
Successfully initialized third party plugins: DeDRM && Kindle and Mobipocket DeDRM && Inept Epub DeDRM && KePub Metadata Reader && Import List && Reading List && Kobo Utilities && EpubSplit && KePub Output && KoboTouchExtended && Inept PDF DeDRM && KePub Metadata Writer
USB devices on system:
(USBDevice(vendor_id=None product_id=None bcd=None devid=usb\root_hub&vid8086&pid2934&rev0003 devinst=1),
USBDevice(vendor_id=0x408 product_id=0x3ba bcd=0x109 devid=usb\vid_0408&pid_03ba&rev_0109&mi_00 devinst=2),
USBDevice(vendor_id=None product_id=None bcd=None devid=usb\root_hub&vid8086&pid2939&rev0003 devinst=3),
USBDevice(vendor_id=None product_id=None bcd=None devid=usb\root_hub20&vid8086&pid293a&rev0003 devinst=4),
USBDevice(vendor_id=None product_id=None bcd=None devid=usb\root_hub20&vid8086&pid293c&rev0003 devinst=5),
USBDevice(vendor_id=None product_id=None bcd=None devid=usb\root_hub&vid8086&pid2936&rev0003 devinst=6),
USBDevice(vendor_id=None product_id=None bcd=None devid=usb\root_hub&vid8086&pid2935&rev0003 devinst=7),
USBDevice(vendor_id=0x408 product_id=0x3ba bcd=0x109 devid=usb\vid_0408&pid_03ba&rev_0109 devinst=8),
USBDevice(vendor_id=None product_id=None bcd=None devid=usb\root_hub&vid8086&pid2938&rev0003 devinst=9),
USBDevice(vendor_id=0x138a product_id=0x1 bcd=0x372 devid=usb\vid_138a&pid_0001&rev_0372 devinst=10),
USBDevice(vendor_id=None product_id=None bcd=None devid=usb\root_hub&vid8086&pid2937&rev0003 devinst=11))

No disabled plugins
Looking for devices of type: MTP_DEVICE
The Windows WPD service says there are no portable devices connected

Looking for devices of type: SMART_DEVICE_APP
All IP addresses {'{F67136C6-8B02-4F6A-962E-A7230C75BA2B}': [{'broadcast': '192.168.206.255', 'netmask': '255.255.255.0', 'addr': '192.168.206.1'}], '{1F6D51A5-D1DA-11E5-9699-806E6F6E6963}': [{'broadcast': '127.255.255.255', 'netmask': '255.0.0.0', 'addr': '127.0.0.1'}], '{3446AE24-84BF-4EE8-8FFE-D023EF75AD21}': [{'addr': '169.254.86.62'}], '{14E9656C-9578-4069-BB27-F35B8DF122C9}': [{'broadcast': '192.168.238.255', 'netmask': '255.255.255.0', 'addr': '192.168.238.1'}], '{0CD68D9F-D359-42C4-B5C8-00518DCC8456}': [{'broadcast': '192.168.1.255', 'netmask': '255.255.255.0', 'addr': '192.168.1.123'}]}
No device is connected

Looking for devices...
USBDevice(vendor_id=0x408 product_id=0x3ba bcd=0x109 devid=usb\vid_0408&pid_03ba&rev_0109&mi_00 devinst=2)
Detected possible device ANDROID

Devices possibly connected: Android driver,
Trying to open Android driver ... Storage number map:
{(7L, 0L): [(2L, 'C')]}
Iterating over all devices of composite device: usb\vid_0408&pid_03ba&rev_0109
Drive letters for: usb\vid_0408&pid_03ba&rev_0109&mi_00 : []
Drive letters for: usb\vid_0408&pid_03ba&rev_0109 : []
Storage number map:
{(7L, 0L): [(2L, 'C')]}
Iterating over all devices of composite device: usb\vid_0408&pid_03ba&rev_0109
Drive letters for: usb\vid_0408&pid_03ba&rev_0109&mi_00 : []
Drive letters for: usb\vid_0408&pid_03ba&rev_0109 : []
Drive letters for USBDevice(vendor_id=0x408 product_id=0x3ba bcd=0x109 devid=usb\vid_0408&pid_03ba&rev_0109&mi_00 devinst=2)
{u'drive_letters': [],
u'pnp_id_map': {},
u'readonly_drives': set([]),
u'sort_map': {}}
failed
Opening of the following devices failed
<calibre.devices.android.driver.ANDROID object at 0x0000000007592940>
Traceback (most recent call last):
File "site-packages\calibre\devices\__init__.py", line 154, in debug
File "site-packages\calibre\devices\usbms\device.py", line 854, in open
File "site-packages\calibre\devices\usbms\device.py", line 268, in open_windows
DeviceError: Unable to detect any disk drives for the device: Android phone. Try rebooting

Like I said, this has been happening recently, like over the past few months and happens with all of my Android devices.

Sorry if I'm posting in the wrong forum but I really can't figure out the source of the problem and, since I could at least post the device debug log from Calibre, I figured I'd start there. Calibre is running 64-bit on fully updated Win10 if that matters and my ebook library is on my internal storage on Android.

BTW it's only with Wireless Device connection; Content Server works fine.

[theducks]

I will assume
that you are accessing from your LOCAL network


change the port (it is on the WDC startup screen).

I believe 'random' (not fixed assignment) is the default

Tick the box and use a different number in the 909x range

I will also assume that you have white-listed calibre.exe (or the fixed port number) in your firewall

[mobamoba]

It's not a network or firewall problem as those would be obvious culprits and that's where I started. It's an intermittent problem that goes away randomly only by restarting Calibre and even that only works sometimes.

It's a problem with recent updates to either CC or Calibre that's limited solely to the Wireless Connection.

[chaley]

Originally Posted by mobamoba

It's not a network or firewall problem as those would be obvious culprits and that's where I started. It's an intermittent problem that goes away randomly only by restarting Calibre and even that only works sometimes.

It's a problem with recent updates to either CC or Calibre that's limited solely to the Wireless Connection.

You are jumping to conclusions here. You have supplied no evidence that proves it cannot be a network, firewall, or AV problem. So on we go ...

The wireless connection is very different technically from the content server connection. The wireless device connects and stays connected for the life of the session. The content server connects every time you do something, so transient connection problems aren't as much of a problem. So we don't know that the content server connection isn't suffering from the same problem.

I received your email. Unfortunately there wasn't a debug log attached to your message. There was a .eml attachment but it is empty. If you send it again, could you also send a calibre debug log? Instructions on how to get one are here. It would be best if you send a calibre log for a session where it failed. My guess is that both calibre and CC will think that the other program closed the connection.

Given that it fails with several android devices, the common points are the home network (probably a single router) and the machine running calibre. Have you upgraded or changed the home router? It might be timing out (not likely). Another possibility is that some very close neighbor is using the same wifi channel as you are, interfering with the connection enough to force a disconnect.

I note from the log you posted that you are using Windows. Given that the problem tends to be "sticky" once it happens, I strongly suspect an antivirus (AV) or other security software. That SW could be deciding that CC is evil based on some traffic analysis. Once it came to that conclusion then it would block connections until either you reboot or it "forgets" about the problem.

You might be able to test if some AV by changing the wireless device port in calibre after a disconnect and when it fails to reconnect. If the AV is blocking a port then giving CC a new one will circumvent it. Another test: disable the AV and see if things fail. This second test is harder because it is sometimes very hard to truly shut off the AV. The one sure way is to reboot into safe mode so the AV never starts.

[chaley]

Moderator Notice
This is really a CC thread. I am moving it to the CC forum

[mobamoba]

Hmm I don't know what happened with that log. I'll resend.

Here's how I know it's not a network or firewall problem: this morning for example, I connected via Wireless Connection, it all went fine; I hit Disconnect in CC; disconnected fine; I hit Connect a few seconds later; nothing connects. Given that absolutely nothing changed in those few seconds, it would seem to rule out network/firewall.

Also, network/firewall was my first stop when I noticed this a few months ago so I did some of my own debugging, like turning off my firewall and AV then trying to connect, installing Calibre on a different machine (including a Linux VM I was running), trying to connect from LTE rather than my LAN, trying on the 5Ghz band rather than the 2.4, etc. I mercifully don't experience neighbor Wifi interference and the computer Calibre is on, FWIW, is wired. The Android devices I use are all different versions - Gingerbread, Kitkat, and Marshamallow, which would seem to rule out something Android-version-specific.

Finally, since the only sometimes-solution is to restart Calibre, that would also seem to rule out network/firewall since as Calibre restart doesn't change either of those.

And I guess the final thing that would seem to rule out network/firewall is that it was all running great up until a few months ago and nothing has changed in my network or firewall, so same machine, same network, same firewall settings (and yes double-checked those) and now intermittent Wireless Connection is what made me conclude that problem lies with either CC or Calibre rather than the setup in my home.

I'll email you the logs you asked for and hopefully they'll point in a direction. Thanks.

[chaley]

@mobamoba: things update all the time all by themselves. For example your AV probably updates daily. Windows updates monthly. You have almost certainly updated other software on the machine.

Calibre restart does in fact change the firewall. It notices that the program is no longer running and changes its internal rule set. The same thing is true for the AV.

If you want to say that nothing changed so the problem must be elsewhere, then we can say the same thing between connections. Nothing at all changed in CC or calibre between those connections, so whatever is blocking the connection must be external to them. Now, I don't really believe that, but it is the same argument you are making.

I hope that the debug logs say something interesting.

[mobamoba]

Understood, but not really from one second to the next like I had this morning right?

[theducks]

Some AV (usually paid) check and update 4 times a day

Windows (<10) updates Weekly, with the malicious software tool being in the Monthly (10 obscures how often. I had earlier versions TELL ME when updates were available, which I usually did most) bundle.

OP
The point is, Your PC is in a state of flux if you allow fully automatic updates. Unless you dig, you don't know what-when.

This is not to say, don't do updates.
Most often 'Do them', just in a way you know that there WAS a change.
That way when something misbehaves, you look at ALL the changes.

BTW is the PC (hosting Calibre) on a Wired portion of the LAN? It just might be interference . There are places in my house where I see 12 usable signal (secured ) networks. A can't imagin what Appartment or Condo residents deal with

[chaley]

Got the debug logs. One strange thing is that calibre is continuously attempting to connect to some device connected by a cable. Calibre thinks that it is an android device, but that might be wrong.

Do you have something plugged into the laptop? Does the problem go away if you unplug it?

[mobamoba]

Literally the only thing plugged into the laptop is a Win10 system recovery USB stick - Calibre wouldn't mistake that for an Android device would it? If so I guess I can pull it out and try again.

[chaley]

It would if it shares a USB ID with some android device. If you look at the device detection log in your first post, you see the item and its IDs. Clearly something is there. Its info is vid_0408&pid_03ba&rev_0109. Calibre thinks that this is a T-Mobile phone.

If you do not use the cabled connection with Android devices then try disabling calibre's Android device plugin. Go to Preferences / Plugins, open the "Device Interface Plugins" section, click on the Android plugin then click Enable/Disable plugin. Restart calibre and (dreaming ...) perhaps the problem will go away.

[mobamoba]

That's the Device ID of my built-in Webcam! I'm not even entirely sure if I can disconnect that since it's embedded in my laptop - and why would Calibre think that's an Android device?!?

I'll disable Android devices in Calibre however and give it a go.

[mobamoba]

I think that solved it! Or at least after getting rid of the Android driver, I did the connect/disconnect/connect thing successfully and am now wirelessly syncing a different Android device just fine as well.

So maybe it was a buggy Android driver in Calibre and not a CC thing after all?

[chaley]

Originally Posted by mobamoba

That's the Device ID of my built-in Webcam! I'm not even entirely sure if I can disconnect that since it's embedded in my laptop - and why would Calibre think that's an Android device?!?

Calibre thinks it is an android device because someone at some point gave that info to Kovid. It sounds like some manufacturer is "pirating" USB ids. I have no idea if it is the webcam vendor or the phone vendor.

Originally Posted by mobamoba

I think that solved it! Or at least after getting rid of the Android driver, I did the connect/disconnect/connect thing successfully and am now wirelessly syncing a different Android device just fine as well.

So maybe it was a buggy Android driver in Calibre and not a CC thing after all?

The android driver isn't buggy. It is doing what users said it needs to do.

My suspicion is that the connection problem is caused by timeouts. The CC connection sequence assumes that if calibre answers the "are you there" message (which yours is doing) then it really is there. However, if in the meantime calibre decides to ask (yet again) the putative Android webcam if this time it has become a phone then a lot of time can pass before calibre will get back to the wireless device. CC will wait 5 seconds for calibre to answer. If the putative phone takes longer than that to say it isn't really a phone then CC will fail to connect.

In any event, I hope that things will be more stable for you now!