Adding titles to Server running as Service on Ubuntu

[ckronengold]

Struggling with the Server/GUI balance.

I have Calibre-Server set to run as a service on startup. I understand I can't run both the server and the GUI at the same time.

Went to add titles. So I "sudo systemctl stop calibre-server" first.
When I opened the GUI, the library was empty.
When I checked the folder location, there was no problem. All titles were there.

Are there issues with using a mounted folder, even if its on the same machine? I have my library on a logical volume (/mnt/media).
No problem with the server reading it, or with the GUI reading it.

Any suggestions?

[kovidgoyal]

Check the library location in the GUI

[ckronengold]

Originally Posted by kovidgoyal

Check the library location in the GUI

Whenever I do this, Calibre tells me that the database is corrupted and asks to rebuild, even after shutting down calibre-server first.

After rebuilding, I get errors when trying to update / edit the metadata. All of the files are locked.

For example...when trying to add the publishing date to this title....(it is not open in any other program.)

calibre, version 3.15.0
ERROR: Permission denied: Could not change the on disk location of this book. Is it open in another program?

Locked file: /home/calibre/Calibre-Library/Jonathan Safran Foer/Extremely Loud and Incredibly Close (3240)/cover.jpg

Traceback (most recent call last):
File "site-packages/calibre/gui2/metadata/single.py", line 579, in apply_changes
File "site-packages/calibre/gui2/metadata/basic_widgets.py", line 1210, in commit
File "site-packages/calibre/db/legacy.py", line 713, in set_cover
File "site-packages/calibre/db/cache.py", line 63, in call_func_with_lock
File "site-packages/calibre/db/cache.py", line 1230, in set_cover
File "site-packages/calibre/db/backend.py", line 1426, in set_cover
File "site-packages/calibre/utils/img.py", line 221, in save_cover_data_to
File "site-packages/calibre/startup.py", line 139, in local_open
IOError: [Errno 13] Permission denied: u'/home/calibre/Calibre-Library/Jonathan Safran Foer/Extremely Loud and Incredibly Close (3240)/cover.jpg'

When I chmod 755 the library folder, the database became unreadable, I couldn't open titles, and had to restore the db. Restoring the database failed.

Traceback (most recent call last):
File "site-packages/calibre/db/restore.py", line 127, in run
File "site-packages/calibre/db/restore.py", line 273, in replace_db
OSError: [Errno 13] Permission denied: '/home/calibre/Calibre-Library/metadata_pre_restore.db'

calibre, version 3.15.0
ERROR: Unhandled exception: <b>IOError</b>:[Errno 13] Permission denied: u'/home/calibre/Calibre-Library/calibre_test_case_sensitivity.txt'

calibre 3.15 embedded-python: True is64bit: True
Linux-4.13.0-25-generic-x86_64-with-debian-stretch-sid Linux ('64bit', 'ELF')
('Linux', '4.13.0-25-generic', '#29-Ubuntu SMP Mon Jan 8 21:14:41 UTC 2018')
Python 2.7.12
Linux: ('debian', 'stretch/sid', '')
Interface language: None
Traceback (most recent call last):
File "site-packages/calibre/gui2/actions/choose_library.py", line 489, in restore_database
File "site-packages/calibre/gui2/ui.py", line 695, in library_moved
File "site-packages/calibre/db/legacy.py", line 81, in __init__
File "site-packages/calibre/db/legacy.py", line 48, in create_backend
File "site-packages/calibre/db/backend.py", line 382, in __init__
File "site-packages/calibre/utils/filenames.py", line 120, in is_case_sensitive
IOError: [Errno 13] Permission denied: u'/home/calibre/Calibre-Library/calibre_test_case_sensitivity.txt'

And I can't open any titles.
calibre, version 3.15.0
ERROR: Failed to set data: Could not set data, click Show Details to see why.

Traceback (most recent call last):
File "site-packages/calibre/gui2/library/models.py", line 1091, in setData
File "site-packages/calibre/gui2/library/models.py", line 1166, in _set_data
File "site-packages/calibre/db/legacy.py", line 441, in set
File "site-packages/calibre/db/cache.py", line 63, in call_func_with_lock
File "site-packages/calibre/db/cache.py", line 1092, in set_field
File "site-packages/calibre/db/write.py", line 544, in set_books
File "site-packages/calibre/db/write.py", line 208, in set_title
File "site-packages/calibre/db/write.py", line 197, in one_one_in_books
File "site-packages/calibre/db/backend.py", line 855, in executemany
File "site-packages/calibre/db/backend.py", line 834, in conn
AttributeError: 'DB' object has no attribute '_conn'

[kovidgoyal]

You have permissions problems. You need to fix the permissions on the containing folder and all files inside recursively, and then make sure that systemctl is running the server as the same user as your desktop user. IIRC there is a User directive you can use in the unit file.

[ckronengold]

Originally Posted by kovidgoyal

You have permissions problems. You need to fix the permissions on the containing folder and all files inside recursively, and then make sure that systemctl is running the server as the same user as your desktop user. IIRC there is a User directive you can use in the unit file.

I thought chmod 775 -R would have handled all of that. No?

At this point, it honestly seems easier just to do a clean import and rebuild the library where I want it rather than moving an old library from an old computer to a server with different users and permissions.

I've set up a user 'calibre' and am putting the library in the /home/calibre folder to minimize the number of issues. I'll run the GUI to import and build the library and then leave the calibre-server.service to handle the rest once I shut the GUI.

[Divingduck]

There is a permissions problem. Your log is telling this to you.
It seems, you have different file and process owners. 775 is maybe not enough, check for chmod 777.

[sjfan]

Originally Posted by ckronengold

I thought chmod 775 -R would have handled all of that. No?

At this point, it honestly seems easier just to do a clean import and rebuild the library where I want it rather than moving an old library from an old computer to a server with different users and permissions.

I've set up a user 'calibre' and am putting the library in the /home/calibre folder to minimize the number of issues. I'll run the GUI to import and build the library and then leave the calibre-server.service to handle the rest once I shut the GUI.

If calibre is running as a user “calibre”, you'll need to make sure you “chown -R calibre” the library directory (or at least chown it to a group that user is in) as well as your chmod 775.

chmod 777 (as mentioned) would allow calibre access, but it'd also allow anything to write/overwrite/delete from your library. Giving non-user/group write privileges is generally a bad idea, except in very limited circumstances combined with the sticky bit (e.g. /tmp, and even there it's been responsible for a lot of security problems via races and the like).

[ckronengold]

I think I'm still struggling with permissions.
I got the server up and running, but can't seem to add books from a monitored folder.
I created a crontab entry, but it doesn't seem to be doing anything.

>*/10 * * * * xvfb-run calibredb add /home/calibre/CalibreAdd/ -r --with-library /home/calibre/Calibre-Library && rm /home/calibre/CalibreAdd/*

When I run it outside of cron, I get "ReadOnlyError:attempt to write a readonly database"

I also created a script, but that is giving me errors as well.
>if [ "$(ls -A /[autoaddfolder])" ]; then
> calibredb add -r "/[autoaddfolder]" --library-path="/[calibrelibraryfolder]"
> rm /[autoaddfolder]/*
>fi
Traceback (most recent call last):
File "site.py", line 77, in main
File "site-packages/calibre/db/cli/main.py", line 253, in main
File "site-packages/calibre/db/cli/main.py", line 55, in run_cmd
File "site-packages/calibre/db/cli/cmd_add.py", line 374, in main
File "site-packages/calibre/db/cli/cmd_add.py", line 169, in do_add
File "site-packages/calibre/db/adding.py", line 190, in recursive_import
File "site-packages/calibre/db/adding.py", line 149, in import_book_directory_ multiple
File "site-packages/calibre/db/cli/cmd_add.py", line 79, in add_books
File "site-packages/calibre/db/cli/main.py", line 190, in run
File "site-packages/calibre/db/cli/main.py", line 177, in db
File "site-packages/calibre/db/legacy.py", line 81, in __init__
File "site-packages/calibre/db/legacy.py", line 48, in create_backend
File "site-packages/calibre/db/backend.py", line 384, in __init__
File "site-packages/calibre/db/schema_upgrades.py", line 19, in __init__
File "site-packages/calibre/db/backend.py", line 844, in execute
File "src/cursor.c", line 236, in resetcursor
ReadOnlyError: ReadOnlyError: attempt to write a readonly database

So both ways are giving me the same errors.

Changing the /home/calibre/ directory to chmod -R 755 generated a different error. "Permission denied: /home/calibre/Calibre-Library/calibre_test_casesensitivity.txt"

When I chmod to 777, it runs, shows the same errors, but then tries to add the books. It tells me which books are already in the library (although I can't see them on localsite:8080).

It also says that it added book IDs and then lists about a dozen (8096-8113), but I don't see them in the localsite:8080 either.

[theducks]

localsite? do you mean localhost?

[ckronengold]

Originally Posted by theducks

localsite? do you mean localhost?

yes, localhost.

[Divingduck]

Guess, your server is running?

Don't you need to add a calibre server, a user with password to your call?

[kovidgoyal]

1) Dont use xvfb-run
2) You need to tell calibredb to conenct to the running server, not try to access the library directly. --with-library has to specify the server URL. See https://manual.calibre-ebook.com/gen...calibredb.html

[Divingduck]

Here is an example call that works for a single document:

calibredb add --with-library "http://localhost:8080/#Your-Library" --username YourUsername --password YourPassword "c:\Path\to\your\documents\dokument.pdf"
or
calibredb add --with-library "http://Your-Server:8080/#Your-Library" --username YourUsername --password YourPassword "c:\Path\to\your\documents\dokument.pdf"

You need to change Your-Library, YourUsername, YourPassword and "c:\Path\to\your\documents\dokument.pdf" to your needs. If you connect with a additional device to your server, you need a server name instead of localhost (second sample) and/or if you have assign a different port as 8080 you need to adust this too.

[ckronengold]

Originally Posted by Divingduck

Guess, your server is running?

Don't you need to add a calibre server, a user with password to your call?

No. It is the server. Why would I need the server to log in to the server just to read a different folder?

Also, I have the server set up so no password or login is required.

I just need it to read the folder where I dump books to get added.

[ckronengold]

Is this for crontab or for the .sh script?

Originally Posted by Divingduck

Here is an example call that works for a single document:

calibredb add --with-library "http://localhost:8080/#Your-Library" --username YourUsername --password YourPassword "c:\Path\to\your\documents\dokument.pdf"
or
calibredb add --with-library "http://Your-Server:8080/#Your-Library" --username YourUsername --password YourPassword "c:\Path\to\your\documents\dokument.pdf"

You need to change Your-Library, YourUsername, YourPassword and "c:\Path\to\your\documents\dokument.pdf" to your needs. If you connect with a additional device to your server, you need a server name instead of localhost (second sample) and/or if you have assign a different port as 8080 you need to adust this too.

Maybe I'm not being clear about what I'm trying to do.

I have a Calibre Server. It is working and is viewable on localhost:8080 or online at xxx.xxx.xxx.xxx:8080. It maintains its library at /myserver/calibre/calibrelibrary.

I want to monitor a folder for new books to be imported into the library /myserver/calibre/calibreadds. I was following the instructions posted on DigitalOcean's tutorials. Everything worked up until the point of running the monitoring the folder and adding to the main library.

The calibre server and library are both running under the user 'calibre' with full access (or so I thought).

When I ran the script and the crontab, it seemed to be adding books. It confirmed it added titles (as mentioned above). But those titles do not appear in the list of books or cover tiles when I check localhost:8080.