Getting custom profile/configurations into new Install

[MontyJ]

First, apologies for the length, but better more detail than not enough, me thinks!

Using Mint Linux 17.3, AMD x64

When I first used the Calibre binary install for Linux, I did not uninstall the existing Calibre install from the package/repository, which was version 1.28 or so. It had been setup ok, however, and I had all my custom setup stuff working great except one setting. I did back up all "file system" and Home directory folders first, then did the binary install, which worked and installed w/o error.

So, V2.45 is not installed. However, it did not keep all my custom settings. Also, when I went into the "Edit Metadata" tab, and clicked on the "Custom Metadata" tab, the page is blank; totally blank grey, absolutely no menus, no fields, nada!

Ok, per a previous post on getting in my custom stuff from a Windows install, I copied in the .Config/Calibre folder in total over (merge/replace) the contents of that folder.

When I do that, calibre simply won't launch; it does not like one or more of the files from my previous working 1.28 install I am using, apparently. Note: sometimes, but not always, it will launch from command line but not the GUI icon. However, regardless of that, none of the customizations ever show up.

Next I tried a complete removal of all Calibre folders and files; first I just ran the uninstall file provided with the binary install script. Same result. Next I did the uninstall and then manually searched out every file in system (except those .PNG image files) and deleted them. Reinstalled the binary install and everything comes up normal with, of course, total default setup configuration.

Again, I copy over the previously working .config/calibre folder contents. Again, it will not launch after that.

So, my question is, is there a specific file(s) I can copy from my previous install that will bring back my original:

- Plugins & plugin setups
- Custom Scripts
- Main Menu Layout customizations
- Custom column setup
- Conversion customizations

My libraries are all intact, being on external ext4 hard drive partitions, so I have no issue there getting them back.

In other words, everything I did to "customize" my setup? I had like 5 years of customizations and I would like to bring all that back and not have to revert to the unsupported V1.28 package install to do it!

Thanks!

[eschwartz]

I don't know why that would happen. Assuming mysterious corruption, a/k/a "gremlins".


There is no one single file, all those files actually do things.

You can, however, test with bringing back pieces at a time.


Custom columns are stored in the per-library database. Inapplicable.

plugins and plugin configurations (should be safe):

~/.config/calibre/plugins/${plugin-name}.zip    # this is the downloaded plugin
~/.config/calibre/plugins/${plugin-name}.json    # this is the configuration for the plugin

Conversion customizations are in:

~/,config/calibre/conversion/

Each file is a python dictionary of the settings. Should be safe.


Main menu: stored in gui.json under the key "action-layout-toolbar*" -- this file stores a lot of stuff, including things like window geometries. Possibly not very safe at all.

[MontyJ]

Thanks for the tips!

i went through a little process of backup/copy/paste to see which of the folders/files in the folder would work and do anything. Here is what, so far, I have found:

These from my backup cause Calibre to NOT launch:
gui.json
gui.py
customize.py
history.plist
save_to_disk.py **

These folders/files from my backup allow Calibre to launch:

Folder--metadata-resources
Folder--conversion
Folder--plugins
edit-book-dynamic.pickle
tweaks.py
iterator.pickle
mtp_devices.json

However, when I got to the save_to_disk.py file, Calibre threw up an error msg on launch, but would not launch. After closing the error message, Calibre then failed to launch from ghe GUI. When I ran "calibre" from a su command prompt, it will launch, but starts at the wizard level again.

Also, I am not sure anything changed even when I was able to launch with the old files. I got all the plugin folder in and it worked, I did not see any changes. Example: There were no user plugins showing at all after restarting Calibre, and I have none showing. The plugin "Find Duplicates.zip" and "Find Duplicates.json" are there, but do not show up as installed.

I suppose I might be able to 'import them from file' to reload them, but now I have to start over as the last thing I tried (** above) has locked me out again and removing it did not let me back in like the ones previous had.

I am beginning to think there are simply too many dependencies in the various installation folders scattered all over for his process to ever work and I am doomed to either start all over with my customizations or go back to V1.28. :-(

Monty

[eschwartz]

su would run calibre as root, and start a new profile for the root user.

This isn't Windows, running things as an administrator isn't required and is only useful for things that are specifically supposed to be run as an administrator -- like user and service management, and installing/updating/removing programs.

Good point on the plugins. calibre lists known installed plugins in customize.py:

plugins = cPickle.loads('stuff')

Just reinstall each plugin. The configuration is the important part, and it is already there.

Or, try copying that file over, and deleting every other line. It might work, depending one what is corrupted and how?

[PeterT]

Guess what.... Running things as root or with Administrator rights is not needed on Windows either. It tends to imply that the user has messed up their system, the same as they could on Linux.

[eschwartz]

The fix (or inadvisable workaround) on linux doesn't usually involve running the application as root.

And some programs mess up when not run as root on Windows. Blame it on bad developers and an unfortunate insecure OS history.

[kovidgoyal]

It would help if you posted an actual error message. Run it in debug mode like this:

calibre-debug -g

[MontyJ]

Thanks for the additional help!

I decided to press on with a new install, along with copying some of the config files into the new .config/calibre folder. Trial and error takes a while, but found out what works and what didn't, LoL.

Anyway, I cannot find where my "Search and Replace" regex and character S&R's were stored. I had probably 40 custom items in my list.

Any possibility those might still be in my backup somewhere? File name?

Thanks!

[Update]

Well, I found the file, and by good fortune, all my regex's were there. But, during the process of copying that file over, I discovered what was probably causing many of my problems previously:

The backup files somehow got changed to 'root' ownership. So while I was copying the files over as 'root', I did not think to look at the file permissions. Once I changed ownership and group ownership, at least this file was picked up by Calibre just fine!

So, now have all plugins, etc loaded. There are some minor catalog generation stuff I need to tweak to get it back to what I had, but now it is pretty much the way I want it.

Thanks everyone for the tips, pointers, and help!

[eschwartz]

Of course.

Copied as root, huh. Well, why did you restore them using sudo???
`cp` by default does not preserve ownership, in order to preserve ownership you should use `cp -rp` or `man cp`.

Honestly, I am surprised *anything* worked.

I *cannot* predict everything you might do. This ^^ was important infomation. *
I assumed if you were restoring files from a backup you were recreating them as the current user. Why would you *want* to use the previous owner, it might not even be the same as the current owner... just a bad practice to get into.


Well, now you know for next time.



* -- Next time I help someone, I may just have to suggest a preliminary `sudo chown -R $(whoami): ~/.config/calibre && chmod -R u+rw ~/.config/calibre` on general principle. Thank you

[MontyJ]

Originally Posted by eschwartz

Of course.

Copied as root, huh. Well, why did you restore them using sudo???
`cp` by default does not preserve ownership, in order to preserve ownership you should use `cp -rp` or `man cp`.

Honestly, I am surprised *anything* worked.

I *cannot* predict everything you might do. This ^^ was important infomation. *
I assumed if you were restoring files from a backup you were recreating them as the current user. Why would you *want* to use the previous owner, it might not even be the same as the current owner... just a bad practice to get into.


Well, now you know for next time.



* -- Next time I help someone, I may just have to suggest a preliminary `sudo chown -R $(whoami): ~/.config/calibre && chmod -R u+rw ~/.config/calibre` on general principle. Thank you

Well, ya wanna lambaste a noob, I am as good as any when it comes to linux.

-There was an attempt to restore config files from a Windows install.
- I had problems with permissions on my Netgear NAS writing and reading
- My first Linux Calibre was the package install, which I had to overwrite with the binary install
- I tend to use GUI whenever I can and if I can't do something with a file, the shortest distance to get it done is usually 'open as root'.
- GRSYNC seemed to also at times force me to use root permissions

Yes, I do learn by doing, and often I 'do' and things get screwed up. You can ask 'why' all day long and the best answer is: "It seemed like good idea at the time!"

Again, thanks for the help.

Monty

[eschwartz]

Originally Posted by MontyJ

Well, ya wanna lambaste a noob, I am as good as any when it comes to linux.

Eh, we all do embarrassing things at first (and we don't necessarily stop later).

-There was an attempt to restore config files from a Windows install.
- I had problems with permissions on my Netgear NAS writing and reading
- My first Linux Calibre was the package install, which I had to overwrite with the binary install
- I tend to use GUI whenever I can and if I can't do something with a file, the shortest distance to get it done is usually 'open as root'.
- GRSYNC seemed to also at times force me to use root permissions

rsync is usually used for archiving, in which case root is often what you want -- especially since it then defaults to preserving ownership. Of course, it depends on what your source was. If you were migrating from Windows and already had previous permissions issues... you want to make sure your user retrieves all the files, Can't go wrong there, if anything messes up you will know because you never got the file to begin with.

permissions when migrating OSes are NOT fun.

Yes, I do learn by doing, and often I 'do' and things get screwed up. You can ask 'why' all day long and the best answer is: "It seemed like good idea at the time!"

Again, thanks for the help.

Monty

Anyway, glad to see you took my comments in the good-natured way they were intended.
The important thing is you learned something from the experience.