Plugin not saving lists in C2.20

teresaapp · 02-20-2015, 06:37 PM

I'm using Calibre 2.20, and just installed the Reading List plugin today. Everything works fine--I can add devices and lists, and add items to lists--until I close Calibre. The next time I start Calibre, the lists are gone, (both the contents and the lists themselves) and sometimes the devices are gone. Any ideas as to why this info is lost? Where is it normally saved?

Update--This may be a more general problem. I also installed the Count Pages plugin today, and I keep having to reconfigure it every time I start Calibre.

Update the second: Rolled back to 2.19, and plugins are now working fine.

I'll see if I can move this post to the general Calibre category

Moderator Notice
Done - BR

Jon4Fun · 02-21-2015, 12:34 AM

I rolled-back Calibre to 2.19 and Reading List (RL)v1.65 of plugin (v1.66 had same issues with v2.19 of Calibre - for me).
The reading list started working again.

Not loosing ALL the info after the update.
I have backups but restoring RL.json does not do the trick.

I doubt if there is a separate trigger for 'remembering' what books were on WHICH LISTS??? Is there?

Going to update to Calibre v2.20 and keep v1.65 of RL and see if it keeps working. I have lost all my info (except for some crap on the [default] list).

UPDATE: RL v1.65 works with Calibre v2.20

BTW, I LOVE the RL plug-in, this is NOT a dig at the plug-in. This is the only time since I have started using it that it has gone this wacko.

Cheers

Quote:

Originally Posted by teresaapp

I'm using Calibre 2.20, and just installed the Reading List plugin today. Everything works fine--I can add devices and lists, and add items to lists--until I close Calibre. The next time I start Calibre, the lists are gone, (both the contents and the lists themselves) and sometimes the devices are gone. Any ideas as to why this info is lost? Where is it normally saved?

Update--This may be a more general problem. I also installed the Count Pages plugin today, and I keep having to reconfigure it every time I start Calibre.

Update the second: Rolled back to 2.19, and plugins are now working fine.

I'll see if I can move this post to the general Calibre category

Done - BR

davidfor · 02-21-2015, 01:06 AM

I have just done some testing. It appears that any configuration in plugins that is specific to a library is not being written to the database. The changes are going into the metadata_db_prefs_backup.json file.

kovidgoyal · 02-21-2015, 02:13 AM

@davidfor: Which API call is this? add_custom_book_data()?

davidfor · 02-21-2015, 02:21 AM

Kovid: Not, its the saving of the library configuration for the plugin. It looks to be due to https://github.com/kovidgoyal/calibr...486444441b2a3c and some not that good code in the plugins.

From a quick look, the plugins that save configuration in the library all have code that something looks like:

Code:

def get_library_config(db):
    library_config = None

    if library_config is None:
        library_config = db.prefs.get_namespaced(PREFS_NAMESPACE, PREFS_KEY_SETTINGS,
                                                 copy.deepcopy(DEFAULT_LIBRARY_VALUES))
    migrate_library_config_if_required(db, library_config)
    debug_print("get_library_config - library_config:", library_config)
    return library_config

def set_library_config(db, library_config):
    debug_print("set_library_config - library_config:", library_config)
    db.prefs.set_namespaced(PREFS_NAMESPACE, PREFS_KEY_SETTINGS, library_config)

And tracking through the code, "library_config" is changed directly and means the comparison added in the __set__ method will always return that there are no changes. Like I said, some "not that good code".

The fix is easy. The plugins should be taking a deep copy of the config and changing that. But, it needs to be done for all the plugins. I seem to have 10 plugins that do this, and there might be more.

kovidgoyal · 02-21-2015, 02:38 AM

Hmm, I suppose one workaround would be to have __getitem__ do a deepcopy of the value it returns if that value is a mutable type. That will be a performance penatly though.

kovidgoyal · 02-21-2015, 02:55 AM

Or have load_from_db() do the deep copying, once, and store the vals separately for the lookup in __setitem__

davidfor · 02-21-2015, 03:07 AM

I'm not sure what you meant, but it feels like the __setitem__ would take a big hit.

What if get_namespaced did the deep copy? As far as I see, this and set_namespaced are only used by the plugins to set their configuration. That should minimise the impact.

kovidgoyal · 02-21-2015, 03:38 AM

This should do the trick:

https://github.com/kovidgoyal/calibr...8c7b0b0eb2afc6

At the cost of one extra db query when setting a preference.

kovidgoyal · 02-21-2015, 03:39 AM

Quote:

Originally Posted by davidfor

What if get_namespaced did the deep copy? As far as I see, this and set_namespaced are only used by the plugins to set their configuration. That should minimise the impact.

I'd rather fix it for __setitem__ just in case there are any places in calibre that do the same thing (or someone adds such code to calibre in the future).

davidfor · 02-21-2015, 04:19 AM

Quote:

Originally Posted by kovidgoyal

I'd rather fix it for __setitem__ just in case there are any places in calibre that do the same thing (or someone adds such code to calibre in the future).

There is that.

You fix looks good. Or at least it would if I wasn't getting the following with it...

Code:

calibre, version 2.20.0
ERROR: Startup error: There was an error during calibre startup. Parts of calibre may not function. Click Show details to learn more.

Traceback (most recent call last):
  File "D:\Development\GitHub\calibre\src\calibre\gui2\main.py", line 245, in initialize_db_stage2
  File "D:\Development\GitHub\calibre\src\calibre\gui2\main.py", line 182, in start_gui
  File "D:\Development\GitHub\calibre\src\calibre\gui2\ui.py", line 361, in initialize
  File "D:\Development\GitHub\calibre\src\calibre\gui2\tag_browser\ui.py", line 33, in init_tag_browser_mixin
  File "D:\Development\GitHub\calibre\src\calibre\gui2\tag_browser\view.py", line 181, in set_database
  File "D:\Development\GitHub\calibre\src\calibre\gui2\tag_browser\model.py", line 282, in set_database
  File "D:\Development\GitHub\calibre\src\calibre\db\backend.py", line 118, in set
  File "D:\Development\GitHub\calibre\src\calibre\db\backend.py", line 112, in __setitem__
AttributeError: 'apsw.Cursor' object has no attribute 'fetchone'

kovidgoyal · 02-21-2015, 04:43 AM

apsw version differences on my dev machine and the windows build. Tis should fix that: https://github.com/kovidgoyal/calibr...3c2c557f375aad

davidfor · 02-21-2015, 04:49 AM

Yes, that fixed it.

HarryT · 02-21-2015, 05:29 AM

Will there be a re-release of 2.20 to resolve this issue, Kovid, or would you advise people affected by it to revert to 2.19?

kovidgoyal · 02-21-2015, 07:14 AM

I have uploaded a new build of 2.20 with the fix

02-20-2015, 06:37 PM	#1
teresaapp Junior Member Posts: 4 Karma: 10 Join Date: Feb 2015 Device: none	Plugin not saving lists in C2.20 I'm using Calibre 2.20, and just installed the Reading List plugin today. Everything works fine--I can add devices and lists, and add items to lists--until I close Calibre. The next time I start Calibre, the lists are gone, (both the contents and the lists themselves) and sometimes the devices are gone. Any ideas as to why this info is lost? Where is it normally saved? Update--This may be a more general problem. I also installed the Count Pages plugin today, and I keep having to reconfigure it every time I start Calibre. Update the second: Rolled back to 2.19, and plugins are now working fine. I'll see if I can move this post to the general Calibre category Moderator Notice Done - BR Last edited by BetterRed; 02-20-2015 at 11:56 PM. Reason: New info

02-21-2015, 02:21 AM	#5
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Kovid: Not, its the saving of the library configuration for the plugin. It looks to be due to https://github.com/kovidgoyal/calibr...486444441b2a3c and some not that good code in the plugins. From a quick look, the plugins that save configuration in the library all have code that something looks like: Code: def get_library_config(db): library_config = None if library_config is None: library_config = db.prefs.get_namespaced(PREFS_NAMESPACE, PREFS_KEY_SETTINGS, copy.deepcopy(DEFAULT_LIBRARY_VALUES)) migrate_library_config_if_required(db, library_config) debug_print("get_library_config - library_config:", library_config) return library_config def set_library_config(db, library_config): debug_print("set_library_config - library_config:", library_config) db.prefs.set_namespaced(PREFS_NAMESPACE, PREFS_KEY_SETTINGS, library_config) And tracking through the code, "library_config" is changed directly and means the comparison added in the __set__ method will always return that there are no changes. Like I said, some "not that good code". The fix is easy. The plugins should be taking a deep copy of the config and changing that. But, it needs to be done for all the plugins. I seem to have 10 plugins that do this, and there might be more.

02-21-2015, 02:38 AM	#6
kovidgoyal creator of calibre Posts: 45,255 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Hmm, I suppose one workaround would be to have __getitem__ do a deepcopy of the value it returns if that value is a mutable type. That will be a performance penatly though. Last edited by kovidgoyal; 02-21-2015 at 02:49 AM.

02-21-2015, 02:55 AM	#7
kovidgoyal creator of calibre Posts: 45,255 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Or have load_from_db() do the deep copying, once, and store the vals separately for the lookup in __setitem__ Last edited by kovidgoyal; 02-21-2015 at 03:03 AM.

02-21-2015, 05:29 AM	#14
HarryT eBook Enthusiast Posts: 85,544 Karma: 93383099 Join Date: Nov 2006 Location: UK Device: Kindle Oasis 2, iPad Pro 10.5", iPhone 6	Will there be a re-release of 2.20 to resolve this issue, Kovid, or would you advise people affected by it to revert to 2.19? Last edited by HarryT; 02-21-2015 at 05:48 AM.

02-21-2015, 01:06 AM	#3
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	I have just done some testing. It appears that any configuration in plugins that is specific to a library is not being written to the database. The changes are going into the metadata_db_prefs_backup.json file.

02-21-2015, 02:13 AM	#4
kovidgoyal creator of calibre Posts: 45,255 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	@davidfor: Which API call is this? add_custom_book_data()?

02-21-2015, 03:07 AM	#8
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	I'm not sure what you meant, but it feels like the __setitem__ would take a big hit. What if get_namespaced did the deep copy? As far as I see, this and set_namespaced are only used by the plugins to set their configuration. That should minimise the impact.

02-21-2015, 03:38 AM	#9
kovidgoyal creator of calibre Posts: 45,255 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	This should do the trick: https://github.com/kovidgoyal/calibr...8c7b0b0eb2afc6 At the cost of one extra db query when setting a preference.

02-21-2015, 04:43 AM	#12
kovidgoyal creator of calibre Posts: 45,255 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	apsw version differences on my dev machine and the windows build. Tis should fix that: https://github.com/kovidgoyal/calibr...3c2c557f375aad

02-21-2015, 04:49 AM	#13
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	Yes, that fixed it.

02-21-2015, 07:14 AM	#15
kovidgoyal creator of calibre Posts: 45,255 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I have uploaded a new build of 2.20 with the fix

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[FileType Plugin] YVES Bible Plugin	ClashTheBunny	Plugins	27	01-16-2023 01:25 AM
Reading Lists plugin and Virtual Libraries	dustyp	Calibre	3	11-22-2014 05:21 AM
Plugin not customizable: Plugin: HTML Output does not need customization	flyingfoxlee	Conversion	2	02-24-2012 02:24 AM
Losing files when saving saving to disk	theaccountant	Library Management	4	03-10-2011 02:38 PM
Journal Not Saving Writing When Moving to a New Page or Saving it	eberhardt333	enTourage Archive	5	11-24-2010 12:47 AM

Advert

Advert