[Library Closed Plugin] Backup configuration folder(s)

[BetterRed]

Snag: The release version (1.0.3 and 1.0.4) takes over a minute to shutdown, this wasn't happening on the pre-release version. It happens on close and restart but not library switch. The backup is created on close.

Here's a debug output

Spoiler:

ApplicationPaletteChange event ignored
Using calibre Qt style: True
ApplicationPaletteChange event received
Traceback (most recent call last):
File "calibre\gui2\ui.py", line 148, in __init__
File "calibre\gui2\ui.py", line 166, in init_iaction
File "calibre\customize\__init__.py", line 659, in load_actual_plugin
File "importlib\__init__.py", line 126, in import_module
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "calibre\customize\zipplugin.py", line 205, in exec_module
File "calibre_plugins.JS_faves.action", line 17, in <module>
File "calibre\customize\zipplugin.py", line 205, in exec_module
File "calibre_plugins.JS_faves.config", line 20, in <module>
ModuleNotFoundError: No module named 'calibre_plugins.BRMenu_menu'
calibre Debug log
calibre 7.24 embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.11.5
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Successfully initialized third party plugins: Backup Configuration Folder (1, 0, 4) && Count Pages (1, 13, 6) && Drop Search Results (1, 0, 14) && Favourites Menu (1, 3, 1) && Job Spy (1, 0, 239) && JobSpy Favourites (1, 2, 0) && Modify ePub (1, 8, 5) && Scroll Book Details (1, 0, 2) && View Manager (1, 10, 3)
calibre 7.24 embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.11.5
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Successfully initialized third party plugins: Backup Configuration Folder (1, 0, 4) && Count Pages (1, 13, 6) && Drop Search Results (1, 0, 14) && Favourites Menu (1, 3, 1) && Job Spy (1, 0, 239) && JobSpy Favourites (1, 2, 0) && Modify ePub (1, 8, 5) && Scroll Book Details (1, 0, 2) && View Manager (1, 10, 3)
QPA platform: windows
devicePixelRatio: 1.5
logicalDpi: 96.0 x 96.0
physicalDpi: 104.87741935483871 x 104.50285714285712
[0.00] Starting up...
[0.00] Initializing db...
[0.22] db initialized
[0.22] Constructing main UI...
[1.09] GUI main window shown
Job Spy has begun initialization...
orig_option = prefs.get[option_key]: True
Tweak Library Name: Books
'read_file_metadata' global.py file prefs.set FROM: True TO: True
'read_file_metadata' global prefs[option_key] value changed FROM: True TO: True
author_sort_copy_method >>> default tweaks.py method is: invert
current tweaks['title_series_sorting']: library_order
--->>>apply_title_series_sorting_by_library: libname IS tweaked; new value for: Books is: library_order
title_series_sorting changed FROM: library_order TO: library_order
Calibre, and hence Job Spy, was gracefully shut down last time? True
Last time daemon started: never
Last time daemon failed: never
Total daemon starts inception_to_date: 0
Total daemon failures inception-to-date: 0
JS: ===>>> change_gui_alternating_row_colors()
current library not specified in Job Spy last-viewed tool active library names... Books
Job Spy has finished initialization...
[1.36] main UI initialized...
[1.36] Started up in 1.36 seconds with 4 books

I'll try it in portable with no other optional plugins installed. And I will try Windows Safe mode in case it's a startup task getting in the way.

NB: my config folder is at C:\_AppData\Calibre\Config_64bit, the backup is C:\_AppData\Calibre\backup-Config_64bit.

BR

[Comfy.n]

noticed that too

[BetterRed]

Originally Posted by Comfy.n

noticed that too

Where is your config, in default location or where CALIBRE_CONFIG_DIRECTORY says it is?

[Comfy.n]

default

[BetterRed]

Originally Posted by BetterRed

I'll try it in portable with no other optional plugins installed. And I will try Windows Safe mode in case it's a startup task getting in the way.

NB: my config folder is at C:\_AppData\Calibre\Config_64bit, the backup is C:\_AppData\Calibre\backup-Config_64bit.

BR

OK in 7.24 portable with factory settings (i.e. no other plugins) also OK in 7.24 portable with a copy of my install settings (i.e. my normal set of optional plugins).

Not OK in minimal Windows Safe mode, same long delay which means its not one of my start up tasks.

Going to try copying content of C:\_AppData\Calibre\Config_64bit to the default location C:\Users\<yours truly>\AppData\Roaming\calibre

BR

[BetterRed]

Originally Posted by Comfy.n

default

Bang goes that theory, I'll try it anyway just in case, my envar editor makes it easy.

[BetterRed]

Originally Posted by BetterRed

Bang goes that theory, I'll try it anyway just in case, my envar editor makes it easy.

Nope - I'm out of ideas.

⸘So, how come its OK in Portable, with the same config‽

BR

[Comfy.n]

Well, I probably was running FTS re-indexing in fast mode when I noticed that. Now I changed it to slow mode and restart time was similar to 1.0.3

[BetterRed]

I only have FTS on my Test library and I never reindex it. I'll take it out of the equation.

Why are you reindexing anyway - I run two indexers across my 2x2TB data drives (which includes my calibre libraries) - Windows Search and X1 - I haven't reindexed either in years.

BR

[Comfy.n]

Well one of the reasons is I had stupidly added the scariest epub in the world to my main library. It's a 905 MB EPUB created by "Jubjub3000". Can't remember where I got it. It's got more than 900 authors for a fanfiction anthology.

I never could open it or split it, or even run Count Pages on it. Most I got was opening it in Editor, but if I tried the report tool, it would hang indefinitely.

Tried converting to TXTZ, no good. TXT... no good.

When I started FTS reindexing it would estimate 3 days. lol

Deleted it, then I'm back to like 12 hours indexing estimate. Phew.

But my main reason to reindex is that I have converted pretty much all of my MOBI and AZW3 formats to EPUB, and even after deleting the mobis and azw3 permanently, I saw an increase in FTS db size. So I thought it would be a good time to reindex.

[BetterRed]

Now I'm really stumped - I had version 1.0.1 squirreled away, so I added it into my installed calibre. Same problem… 1 minute plus to shutdown.

I did my initial evaluation on portable.

BR

[chaley]

I noticed that it is sometimes very slow if the content server needs to be shut down.

I added timing statements. Here is a shutdown on my machine (win11):

[8.61] [Backup Configuration Folder]: starting backup process
[11.72] [Backup Configuration Folder]: wrote config backup to [filename]
[11.72] [Backup Configuration Folder]: finished

You can see that it ran in 3.11 seconds, most of which was spent creating the zip file.

Note that the time to create the archive depends directly on the amount of stuff in the config folder. Mine contains 374 files, 48 folders, with a total size of 23.9MB (reported by Windows properties).

Attached is a version with timed prints. It has a new version number, but I haven't released it.

EDIT: If you don't see the debug output then you might need to use calibre-debug -g. I'll try it.
EDIT 2: No, the plugin's debug prints don't show in the "restart in debug mode" log.
EDIT 3: I made a release, putting the new zip in the first post. I deleted the version in this post.
EDIT 4: A fix for the problem where the plugin's log messages don't appear when calibre is restarted in debug mode is in calibre source.

[BetterRed]

No server here

calibre-debug -g

Spoiler:

Microsoft Windows [Version 10.0.19045.5371]
(c) Microsoft Corporation. All rights reserved.

c:\>calibre-debug -g
calibre 7.24 embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.11.5
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Successfully initialized third party plugins: Backup Configuration Folder (1, 0, 5) && Count Pages (1, 13, 6) && Drop Search Results (1, 0, 14) && Favourites Menu (1, 3, 1) && Job Spy (1, 0, 239) && JobSpy Favourites (1, 2, 0) && Modify ePub (1, 8, 5) && Scroll Book Details (1, 0, 2) && View Manager (1, 10, 3)
QPA platform: windows
devicePixelRatio: 1.5
logicalDpi: 96.0 x 96.0
physicalDpi: 104.87741935483871 x 104.50285714285712
ApplicationPaletteChange event ignored
Using calibre Qt style: True
ApplicationPaletteChange event received
[0.00] Starting up...
[0.00] Initializing db...
[0.08] db initialized
[0.08] Constructing main UI...
Traceback (most recent call last):
File "calibre\gui2\ui.py", line 148, in __init__
File "calibre\gui2\ui.py", line 166, in init_iaction
File "calibre\customize\__init__.py", line 659, in load_actual_plugin
File "importlib\__init__.py", line 126, in import_module
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "calibre\customize\zipplugin.py", line 205, in exec_module
File "calibre_plugins.JS_faves.action", line 17, in <module>
File "calibre\customize\zipplugin.py", line 205, in exec_module
File "calibre_plugins.JS_faves.config", line 20, in <module>
ModuleNotFoundError: No module named 'calibre_plugins.BRMenu_menu'
qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile
qt.gui.imageio: libpng warning: iCCP: cHRM chunk does not match sRGB
[0.98] GUI main window shown
Job Spy has begun initialization...
orig_option = prefs.get[option_key]: True
Tweak Library Name: Test
'read_file_metadata' global.py file prefs.set FROM: True TO: True
'read_file_metadata' global prefs[option_key] value changed FROM: True TO: True
author_sort_copy_method >>> default tweaks.py method is: invert
current tweaks['title_series_sorting']: library_order
--->>>apply_title_series_sorting_by_library: libname is NOT tweaked; default value used for: Test is: library_order
title_series_sorting changed FROM: library_order TO: library_order
Calibre, and hence Job Spy, was gracefully shut down last time? True
Last time daemon started: never
Last time daemon failed: never
Total daemon starts inception_to_date: 0
Total daemon failures inception-to-date: 0
qt.gui.imageio: libpng warning: iCCP: known incorrect sRGB profile
JS: ===>>> change_gui_alternating_row_colors()
current library not specified in Job Spy last-viewed tool active library names... Test
Job Spy has finished initialization...
[1.36] main UI initialized...
[1.36] Started up in 1.36 seconds with 97 books
[9.97] [Backup Configuration Folder]: starting backup process
[85.30] [Backup Configuration Folder]: wrote config backup to C:\_AppData\Calibre\backup-Config_64bit\2025-02-06 at 09-46-39.zip
[85.31] [Backup Configuration Folder]: finished

c:\>

Something I haven't tried, but will try, is relocating my config from the SDD system drive, to an HDD data drive, which is where my calibre portables live.

BR

[BetterRed]

Solved - something huge (3.2GB) had snuck into the C:\_AppData\Calibre\Config_64bit\plugins folder. It should have been in C:\_Sandpit

BR

[chaley]

Originally Posted by BetterRed

Solved - something huge (3.2GB) had snuck into the C:\_AppData\Calibre\Config_64bit\plugins folder. It should have been in C:\_Sandpit

BR

I was just getting ready to ask you how big the backup zip was. All the time was in making the archive, which implied exactly what you found.

I wonder if I should report the size of the written archive?