Error communicating with device

[WS64]

I see this message all the time.
No matter which ebook reader, even when connecting to a folder (no ebook reader involved).

Is there anything I can do to prevent this error?
It seems to help to stop memory-hungry applications (Firefox), but on the other hand this Win7 computer has 4GB and absolutely no problems with other applications.

calibre, version 1.43.0
ERROR: Error: Error communicating with device



Traceback (most recent call last):
  File "site-packages\calibre\gui2\device.py", line 86, in run
  File "site-packages\calibre\gui2\device.py", line 520, in _sync_booklists
  File "site-packages\calibre\devices\usbms\driver.py", line 395, in sync_booklists
  File "site-packages\calibre\devices\usbms\driver.py", line 393, in write_prefix
  File "site-packages\calibre\ebooks\metadata\book\json_codec.py", line 123, in encode_to_file
  File "json\__init__.py", line 250, in dumps
  File "json\encoder.py", line 210, in encode
MemoryError

[kovidgoyal]

How large is the .metadata.calibre file in the folder you are connecting to?

[WS64]

Right now 127.456KB

[kovidgoyal]

Then I dont see how it could possibly be causing a memory error, short of a bug in the python stdlib json module or unless your computer is under extreme memory pressure. The next time it happens check to see how much free memory you have and what is using the memory.

[WS64]

Hope it's not too big...
Calibre with 800000K, next is Firefox with 500000K.

Spoiler:

[kovidgoyal]

That screenshot implies that the json module is trying to create a string of over 1.2 GB in size to run out of memory with that base memory usage. I dont see how that's possible. Unless you have something liek the full text of all the book in your library int he comments column (I know one or two people that do that). And even then...

[WS64]

Just curious, where do you see that info in the screenshot?

The funny thing is that I actually do not need any metadata at all to connect to the folder since I use this folder to send books to Koreader 8which does not use metadata at all).
I just can't use "send to folder" because I still want to know which books actually are there, so calibre needs to know which books are "installed on that device" (the folder).

And even more, I don't need the comments at all... Would it be worth to try to delete all comments and try again?

[WS64]

Well, I can answer that, I removed all comments (Set the comments for all selected books), no change

[kovidgoyal]

calibre.exe is using 800MB a 32 bit process on windows can allocate a max of 2GB if it is running out of memory that implies it is trying to allocate a further 1.2GB, I dont see how that's possible while writing metadata.calibre, since all that file contains is a cache of metadata. How many books in your library, how many are you sending to the device?

Try creating a new empty library add one or two books and send those to the folder and see what happens.

[WS64]

Originally Posted by kovidgoyal

calibre.exe is using 800MB a 32 bit process on windows can allocate a max of 2GB if it is running out of memory that implies it is trying to allocate a further 1.2GB, I dont see how that's possible while writing metadata.calibre, since all that file contains is a cache of metadata. How many books in your library, how many are you sending to the device?

Try creating a new empty library add one or two books and send those to the folder and see what happens.

10000 books, and none sent. This error happens already when connecting to the folder. (which is another thing I don't understand, why is it sending metadata at that point?)

I created a new library and added 3 books, then connected to the folder. Same error happened.
I watched the task manager, when the error happened memory was a bit over 1GB, after the message popped up it fell down to 800000.

I should mention btw that I restore the .metadata.calibre file from an actual backup after every error since Calibre always set the file to zero length, and usually I can forget the whole folder after that since Calibre does not know anymore which book belongs to which book in the library.

[kovidgoyal]

I'm at a loss.

[WS64]

Just for the record, I edited my .metadata.calibre file and replaced there

    "comments": .*?$

with

    "comments": null,

(means I also deleted all comments from this file), after that the file was 121000 KB, but unfortunately no change with the error.

What I did not mention so far, there is one thing that nearly always helps: Kill Firefox.
After that everything works, even with the new file without the comments.

I just hoped I could connect to my folder without having to kill Firefox first...

Is there another way to connect my library to a folder in a way that Calibre remembers which book in the folder belongs to which book in the library and can update the folder by writing books there in "{author_sort[0]}/{authors}/{title}" form?
What I don't need with this folder are any kind of metadata info...

[BetterRed]

@WS64 Try running Firefox in safe mode.

IIRC I saw a similar problem with a file manager product, the problem was not firefox itself but an extension or plug-in. It was some sort of 'close embrace', firefox (via the ext/pi) it would 'lock' a resource that the file manager had allocated, hence it never got de-allocated.

Dunno whether that makes any sense in the context of calibre - but maybe worth a try

BR

[WS64]

It is not Firefox fault.
Today it happened a few times without Firefox running.
Actually nearly all other applications were shutdown before .
I really have no idea anymore.
Is there any other way I can sync the library to a folder?

[BetterRed]

Originally Posted by WS64

It is not Firefox fault.
Today it happened a few times without Firefox running.
Actually nearly all other applications were shutdown before .
I really have no idea anymore.
Is there any other way I can sync the library to a folder?

Yes via Connect/Share->Connect to folder then use Send to Device. Suggest try with a few books/small library to get templates etc as you want them.

Did you try running Windows in Safe Mode?

BR