pickle/cPickle problems

davidfor · 07-14-2019, 12:38 AM

While trying to debug a problem with loading the configuration in the KoboTouchExtended driver, I had to find an older version of the configuration file. This is the .py type rather than the .py.json. file. With this, when I opened the driver configuration using 3.45.2, it reverted to the default configuration.

Running from source, I found an error message:

Code:

Failed to parse old style options string with error: loads() argument 1 must be string, not unicode

I did some checking, and the problem is in calibre\utils\config_base.py where it interprets the old config file. I added a "traceback.print_exc()" where the message was printed and got:

Code:

Traceback (most recent call last):
  File "E:\Development\GitHub\calibre\src\calibre\utils\config_base.py", line 291, in parse_old_style
  File "<string>", line 7, in <module>
TypeError: loads() argument 1 must be string, not unicode
Failed to parse old style options string with error: loads() argument 1 must be string, not unicode

Looking further, the problem is caused by these lines in the config file:

Code:

format_map = cPickle.loads('\x80\x02]q\x01(X\x05\x00\x00\x00kepubq\x02X\x04\x00\x00\x00epubq\x03X\x03\x00\x00\x00cbrq\x04X\x03\x00\x00\x00cbzq\x05X\x03\x00\x00\x00pdfq\x06X\x03\x00\x00\x00txtq\x07e.')

extra_customization = cPickle.loads('\x80\x02]q\x01(X\x12\x00\x00\x00#myshelves, seriesq\x02\x88\x89\x89\x89\x89\x89\x89\x89\x88\x89\x89X\x00\x00\x00\x00\x88\x89\x89\x89\x89\x89\x89X\x00\x00\x00\x00\x89e.')

The first one is going to be in the config files for all the different drivers. And testing opening the configuration for some of the other drivers that I have at some time or another configured, produced the same errors.

I did some testing with older versions, and this doesn't happen with any of the older versions. So, if they driver configurations have already been migrated to the JSON versions, they are migrated correctly. But, anyone who skipped straight to 3.45.x will lose their driver configuration.

kovidgoyal · 07-14-2019, 05:55 AM

This should take care of it

https://github.com/kovidgoyal/calibr...18df8f0da317ed

davidfor · 07-14-2019, 06:12 AM

That works. Thanks.

07-14-2019, 12:38 AM	#1
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	pickle/cPickle problems While trying to debug a problem with loading the configuration in the KoboTouchExtended driver, I had to find an older version of the configuration file. This is the .py type rather than the .py.json. file. With this, when I opened the driver configuration using 3.45.2, it reverted to the default configuration. Running from source, I found an error message: Code: Failed to parse old style options string with error: loads() argument 1 must be string, not unicode I did some checking, and the problem is in calibre\utils\config_base.py where it interprets the old config file. I added a "traceback.print_exc()" where the message was printed and got: Code: Traceback (most recent call last): File "E:\Development\GitHub\calibre\src\calibre\utils\config_base.py", line 291, in parse_old_style File "<string>", line 7, in <module> TypeError: loads() argument 1 must be string, not unicode Failed to parse old style options string with error: loads() argument 1 must be string, not unicode Looking further, the problem is caused by these lines in the config file: Code: format_map = cPickle.loads('\x80\x02]q\x01(X\x05\x00\x00\x00kepubq\x02X\x04\x00\x00\x00epubq\x03X\x03\x00\x00\x00cbrq\x04X\x03\x00\x00\x00cbzq\x05X\x03\x00\x00\x00pdfq\x06X\x03\x00\x00\x00txtq\x07e.') extra_customization = cPickle.loads('\x80\x02]q\x01(X\x12\x00\x00\x00#myshelves, seriesq\x02\x88\x89\x89\x89\x89\x89\x89\x89\x88\x89\x89X\x00\x00\x00\x00\x88\x89\x89\x89\x89\x89\x89X\x00\x00\x00\x00\x89e.') The first one is going to be in the config files for all the different drivers. And testing opening the configuration for some of the other drivers that I have at some time or another configured, produced the same errors. I did some testing with older versions, and this doesn't happen with any of the older versions. So, if they driver configurations have already been migrated to the JSON versions, they are migrated correctly. But, anyone who skipped straight to 3.45.x will lose their driver configuration.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Branston Pickle lorry crashes near Cheddar	PeterT	Lounge	0	10-05-2013 10:10 AM
A Pickle	jackastor	Calibre	11	07-14-2011 09:15 AM
Free (Kindle) Frankie Pickle and the Mathematical Menace	arcadata	Deals and Resources (No Self-Promotion or Affiliate Links)	0	10-12-2010 12:01 AM
Other Fiction Smollett, Tobias: The Adventures of Peregrine Pickle. v1. 23 May 07	Dr. Drib	BBeB/LRF Books	2	06-10-2007 06:54 AM

07-14-2019, 05:55 AM	#2
kovidgoyal creator of calibre Posts: 43,842 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	This should take care of it https://github.com/kovidgoyal/calibr...18df8f0da317ed

07-14-2019, 06:12 AM	#3
davidfor Grand Sorcerer Posts: 24,907 Karma: 47303748 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	That works. Thanks.

Advert