Calibre 3.0 Content Server standalone

[kovidgoyal]

Oh and just by the way, if you are storing hashed passwords for digest auth, that means you have to hash them with md5, without a salt, which means they can be pretty trivially brute-forced.

[loviedovie]

The problem is this, gui apps might crash, killed etc due to memory etc and the user might not know about it or might not have a way to fix the issue remotely. But the services can be restarted automatically. That is why I personally like the separate server app.

I liked having the gui for manipulating "when needed" and let the server running at all times.

Originally Posted by kovidgoyal

I dont see why you cannot leave the GUI running, its not like leaving it running somehow consumes extra resources. Your RAM does not care how full it is.

.

[snoopy_1978]

Originally Posted by kovidgoyal

You are syncing default passwords, sow hat earthly difference does it make?

I can't follow you.. what default passwords do u mean?
You proposed to sync LDAP Users to calibre DB for what I had to know the users passwords. I said I don't know them, so I can't do such a sync.
Because of that and the fact, that the passwords really are saved in cleartext(!!) in the sqlite DB (just verified by looking at the calibre userdb right now) I only would use a few technical users with passwords I nowhere else use just for this one and only purpose.
Using critical passwords in a system which stores them in cleartext is very risky and insecure!

[snoopy_1978]

Originally Posted by loviedovie

The problem is this, gui apps might crash, killed etc due to memory etc and the user might not know about it or might not have a way to fix the issue remotely. But the services can be restarted automatically. That is why I personally like the separate server app.

I liked having the gui for manipulating "when needed" and let the server running at all times.

I'm absolutly with you.. Programs/Processes with the function of offering services to the outside should never rely on a GUI or even an always OS - logged-in User.... what is more, serious server processes mostly run on servers without any graphical UI or on VMs. For this case, calibre-server is generally the right way IMHO

What I'm wondering, too, is, why a seperate server-process (which seems nothing more than a WebServer) is neccessary at all.
Wouldn't it be enogh to write some sort of WebApp (in PHP or whatever language) that deals with the calibre content, offers an OPDS Feed in a completely read-only way and let Apache or NGinx serve it?
Then one could use all the features of a "real" WebServer with all its modules (Basic Auth in many different ways is the first that comes to my mind), so concentrate on the plain "Business Logic" and don't have to deal with how to implement all the HTTP(S) - Protocol / Auth / Security stuff...
But I think, there will be some good reasons, why calibre-server exists..

[snoopy_1978]

Originally Posted by kovidgoyal

Oh and just by the way, if you are storing hashed passwords for digest auth, that means you have to hash them with md5, without a salt, which means they can be pretty trivially brute-forced.

I never ever would come to the idea to store hashed passwords without a salt no matter which hashing algorithm is used. And if I have the choice, I would never use MD5 for password hashing at all because of its brute force weakness that u too mentioned
(see also:
https://security.stackexchange.com/q...dered-insecure
or this:
"As of 2010, the CMU Software Engineering Institute considers MD5 "cryptographically broken and unsuitable for further use",[29] and most U.S. government applications now require the SHA-2 family of hash functions.[30] In 2012, the Flame malware exploited the weaknesses in MD5 to fake a Microsoft digital signature." (from: https://en.wikipedia.org/wiki/MD5))

[BetterRed]

Originally Posted by snoopy_1978

[. . .] What I'm wondering, too, is, why a seperate server-process (which seems nothing more than a WebServer) is neccessary at all.

[. . .]

But I think, there will be some good reasons, why calibre-server exists.

FWIW -- My understanding is that Kovid wants to provide a subset of the update features of the Library Manager GUI within the server UI, metadata updates, and conversions have been mentioned. He has also hinted at something like plugins.

He recently posted a link to his TODO list somewhere, in which some these things were mentioned

[. . .]

Ah-ha, here 'tis ==>> https://www.mobileread.com/forums/sh...53#post3541853.

BR

[kovidgoyal]

Sigh, I am going to try this one last time, just because I am eternal optimist.

1) You sync user accounts from LDAP to calibre with default passwords. i.e. the same dummy password for all user accounts.

2) You get Apache to re-write the Authorization header so it contains the username and this default password instead of the actual password.

3) The web server uses digest authentication. Digest authentication requires unhashed passwords (or at best passwords hashed without a salt and with MD5). You dont like it, take it up with the IETF.

4) You cannot run the server and the GUI at the same time because they both can make changes to the calibre library. Pick one.

I am done with this thread. It seems to be filled with people asking the same questions over and over again, despite my having them answered already.

[loviedovie]

kovid

Please take it easy. We all love Calibre but we are trying to understand why the certain decisions were made such that they changed the previous behaviors drastically.

Sure, I understand that multiple sources modifying the database is a no no. I get that. But what I do not understand is that why the new tangent was a choice of design compared to the previous behaviour? I already tried to give you an example where the gui or the server approach just make it sound just like highway or my way thing instead of user friendliness.


Please do not get me wrong. I am pretty sure that you have a much better grasp of these things compared to most of us but from the user perspective, certain changes are just "too much" to adapt right away if you will, when it comes to the server/client thing. I am personally not upgrading( not mad or anything) because I need to figure out first how I can fit this new model into my user space.


thanks

[snoopy_1978]

Originally Posted by BetterRed

FWIW -- My understanding is that Kovid wants to provide a subset of the update features of the Library Manager GUI within the server UI, metadata updates, and conversions have been mentioned. He has also hinted at something like plugins.

He recently posted a link to his TODO list somewhere, in which some these things were mentioned

[. . .]

Ah-ha, here 'tis ==>> https://www.mobileread.com/forums/sh...53#post3541853.

BR

The idea to extend the calibre-server (or the WebFrontend it provides) with more and more functions and features is very good I think, I like WebApps at all.
But I don't understand the need of wrapping the WebFrontend with a self implemented webserver and not just take standard technologies (Nginx/Apache + PHP/OtherLanguage + some DB like in thousands of other Desktop like WebApps nowadays) and just concentrate on the WebApps core functionalities...

[snoopy_1978]

Originally Posted by kovidgoyal

Sigh, I am going to try this one last time, just because I am eternal optimist.

1) You sync user accounts from LDAP to calibre with default passwords. i.e. the same dummy password for all user accounts.

2) You get Apache to re-write the Authorization header so it contains the username and this default password instead of the actual password.

Then we came to a similar solution, as I wrote, this is almost what I do now. Me too rewrite the AuthHeaders in Apache, not just the Password but the users to default ones too. This would be easier, cause in the other case one had to base64-encode the dynamic usernames with the help of the apache config (The AuthString for BasicAuth is bas64 encoded form of "username: password"); don't figured out if and how this is possible. This is my solution:

   #NonAdult                                                                                                                                                
    RequestHeader set Authorization "Basic <NonAdultUserPassString>" expr=(%{REMOTE_USER}in{"Sarah"})                                                                                                    
    #All                                                                                                                          
    RequestHeader set Authorization "Basic <AllUserPassString>" expr=(%{REMOTE_USER}in{"John","Jane"})

This means I have just one technical user with an unimportant password for each virt lib and just "map" the LDAP Users to these technical calibre users.

This complicates things a little compaired to V2 but it generally woks.

Originally Posted by kovidgoyal

4) You cannot run the server and the GUI at the same time because they both can make changes to the calibre library. Pick one.

I know, I don't do this even in V2. I have one little "Work" library where I import new Books edit them to my needs with the calibre Windows GUI and then - also with the calibre GUI - move the finished Book to the "Online" library which calibre-server on the linnux-vm serves. Works quite good.

Originally Posted by kovidgoyal

I am done with this thread. It seems to be filled with people asking the same questions over and over again, despite my having them answered already.

Why so excited? Just relax, and perhaps think a little, WHY many people ask similar questions, there must be a reason for it...
We just all want to help make this good software even better.. a single developer can't think of (and less test) each and every use case that could came up in a larger community like this one of calibre, just quite normal. So shouldn't you be a little more thankful of the communities contribution?

[snoopy_1978]

Originally Posted by loviedovie

kovid

Please take it easy. We all love Calibre but we are trying to understand why the certain decisions were made such that they changed the previous behaviors drastically.

This is exactly what I meant in my last post.. the community (esp. the people also working in the IT for many many years) just want to understand such drastic design changes like between V2 and V3 and have to learn, how to deal with it. Mainly if those changes affects the actual implementations in such a way, that it has to get extremly be re-designed or even brakes some functionalities that where successfully used in former versions.

Originally Posted by loviedovie

Sure, I understand that multiple sources modifying the database is a no no. I get that. But what I do not understand is that why the new tangent was a choice of design compared to the previous behaviour? I already tried to give you an example where the gui or the server approach just make it sound just like highway or my way thing instead of user friendliness.

I also think it is always not a good idea to develop "next to" the users needs only because me as developer personally think a new decision is a good thought but perhaps 90% of the users don't need or don't like it or the way, it will be implemented.

Originally Posted by loviedovie

Please do not get me wrong. I am pretty sure that you have a much better grasp of these things compared to most of us but from the user perspective, certain changes are just "too much" to adapt right away if you will, when it comes to the server/client thing. I am personally not upgrading( not mad or anything) because I need to figure out first how I can fit this new model into my user space.

Exactly... such drastic design changes can make an update very complicated or even impossible so that users could look for alternatives

[BetterRed]

Originally Posted by snoopy_1978

[. . .] Why so excited? Just relax, and perhaps think a little, WHY many people ask similar questions, there must be a reason for it...
We just all want to help make this good software even better.. a single developer can't think of (and less test) each and every use case that could came up in a larger community like this one of calibre, just quite normal. So shouldn't you be a little more thankful of the communities contribution?

See ==>> https://calibre-ebook.com/new-in/twelve - under the horizontal line.

I don't know how many users took advantage of the beta test period, but it didn't seem to be very many, and few of them appeared to use the old server in a very sophisticated manner.

Perhaps Kovid needs a better way of calling for beta testers. A once a week notice on calibre startup - "Hey, there's a new version in beta testing - please help" with a link to a thread here.

BR

[Ravensknight]

Or just release a "new" version and watch the people who have never showed up before or commented before come out of the woodwork and act like they own the place...

[eschwartz]

Originally Posted by snoopy_1978

I'm absolutly with you.. Programs/Processes with the function of offering services to the outside should never rely on a GUI or even an always OS - logged-in User.... what is more, serious server processes mostly run on servers without any graphical UI or on VMs. For this case, calibre-server is generally the right way IMHO

What I'm wondering, too, is, why a seperate server-process (which seems nothing more than a WebServer) is neccessary at all.
Wouldn't it be enogh to write some sort of WebApp (in PHP or whatever language) that deals with the calibre content, offers an OPDS Feed in a completely read-only way and let Apache or NGinx serve it?
Then one could use all the features of a "real" WebServer with all its modules (Basic Auth in many different ways is the first that comes to my mind), so concentrate on the plain "Business Logic" and don't have to deal with how to implement all the HTTP(S) - Protocol / Auth / Security stuff...
But I think, there will be some good reasons, why calibre-server exists..

Minimal PHP webapps that look at your calibre library and serve files, already exist. calibre-server does much more than that, however, and I don't really understand why you care which language it uses to implement complex read/write operations that query a database and can modify files as needed in-transit (cf. plugboards) etc.

And it also needs to implement auth, protocol, etc. because not everyone wants to set up an apache instance in order to use calibre's Content Server.

Originally Posted by snoopy_1978

The idea to extend the calibre-server (or the WebFrontend it provides) with more and more functions and features is very good I think, I like WebApps at all.
But I don't understand the need of wrapping the WebFrontend with a self implemented webserver and not just take standard technologies (Nginx/Apache + PHP/OtherLanguage + some DB like in thousands of other Desktop like WebApps nowadays) and just concentrate on the WebApps core functionalities...

This is complete and utter nonsense. There are many server applications powering important things on the internet, which are self-implemented in, for example, python rather than being written in PHP.

[eschwartz]

FWIW I quite want calibre to be able to connect to a running server instance, and I hope Kovid eventually gets that working.

In the meantime, I have updated my calibre wrapper script to accommodate the server:

#!/bin/bash

eval $(systemctl --user show calibre-server -q -p ActiveState)

toggle-server() {
    local action=$1

    if [[ $ActiveState = active ]]; then
        systemctl --user $action calibre-server
    fi
}

toggle-server stop

if [ $# -eq 0 ] || [[ "$1" = "--detach" ]]; then
    echo -e "\n\n\nStarting calibre at $(date).\n\n\n" >> "/tmp/calibre-debug-$USER.log" 2>&1
    # FIXME: Once calibre supports running the GUI with a server database, we can stop doing this.
    if [[ $ActiveState = active ]]; then
        echo -e "Pausing the systemd-managed server due to calibre v3 restraints.\n" >> "/tmp/calibre-debug-$USER.log" 2>&1
        echo -e "FIXME: Once calibre supports running the GUI with a server database, we can stop doing this.\n\n\n" >> "/tmp/calibre-debug-$USER.log" 2>&1
    fi
    calibre-debug -g 2>&1 | tee -a "/tmp/calibre-debug-$USER.log"
else
    /usr/bin/calibre "$@"
fi

toggle-server start