COPS : Calibre OPDS PHP Server

[mariosipad]

@MontyJ

About larger cover images.
I have just tried again with my "solution" for larger cover images (in a browser) and it still works (for me).

To make it work in a browser (firefox) you need to empty the browser cache first. Then the browser loads a new version of the COPS CSS and …. larger covers.

If you want larger covers in a OPDS view you need to change a value in config_local.php.

    /*
     * Height of thumbnail image for OPDS
     */
    $config['cops_opds_thumbnail_height'] = "164";

However, I have not been able to make it work.

Do you use COPS in a browser or through OPDS?

[MontyJ]

Originally Posted by mariosipad

@MontyJ

About larger cover images.
I have just tried again with my "solution" for larger cover images (in a browser) and it still works (for me).

To make it work in a browser (firefox) you need to empty the browser cache first. Then the browser loads a new version of the COPS CSS and …. larger covers.

If you want larger covers in a OPDS view you need to change a value in config_local.php.

    /*
     * Height of thumbnail image for OPDS
     */
    $config['cops_opds_thumbnail_height'] = "164";

However, I have not been able to make it work.

Do you use COPS in a browser or through OPDS?

Thanks. I use Firefox.

I will try it again with your cache suggestion and report back.

[edit: These tips worked perfect for me!]

[pven]

I am a bit stuck.

I have a Synology DS410 and a DS214+. Cops is working perfectly on the DS410, but it is very slow. So I copied everything (library and cops-directory) to exactly the same location on DS214+, installed WEB station, enabled the PHP-modules and added the Calibre-library to open_base-dir. But it does not work completely. When I check http://diskstation2/cops I see all the overviews, but when I want to download a epub I get ''404 The page you are looking for cannot be found.'. To be very sure it is not a privilege-problem I did a 'chmod -R 777' for my library.

What can I do to fix this?

[mariosipad]

@pven

DSM 5 or DSM 6?

(I can test with DSM 5, but not with DSM 6 because my Synology DS-110j doesn't run DSM 6.)

And what version of COPS?

Edit:
I just copied the latest version of COPS from an Apache Debian Server to my Synology to take the place of an existing COPS installation. I had to edit the config_local.php (actually copied it over from the old install) and it worked with my Firefox browser and Epub plugin.

[pven]

Originally Posted by mariosipad

@pven

DSM 5 or DSM 6?

(I can test with DSM 5, but not with DSM 6 because my Synology DS-110j doesn't run DSM 6.)

And what version of COPS?

Edit:
I just copied the latest version of COPS from an Apache Debian Server to my Synology to take the place of an existing COPS installation. I had to edit the config_local.php (actually copied it over from the old install) and it worked with my Firefox browser and Epub plugin.

DS410 runs DSM5, DS214+ runs DSM6.

Version: uh, the one that is on SymCommunity

What exactly did you edit?

Update:

I changed $config['cops_use_url_rewriting'] = "1"; to 0, now it works!

[MontyJ]

QNAP TS-251

First, I do have COPS setup and working. It is a pretty nice setup, as I backup my entire Calibre library to the NAS, then simply point COPS to that backup folder.

But I want to let other people have access via the Internet. On my Linux PC, I do that directly from Calibre Server daemons that point to specific ports. Then I use the PageKite DNS service (my ISP changes my dynamic IP often) to provide the domain names for each of those Calibre libraries. They are very large libraries so need to break them down into smaller categories that make sense to the users.

I tried to install Pagekite on QNAP, but the file/folder organization does not seem to be normal Linux, so I was unable to get Pagekite properly installed. I am not anywhere near enough of a guru to get their source code and modify it.

I do not need a full blown host website. The COPS interface is fine for my users. So how can I inexpensively set up something like Pagekite for COPS?

I know there is a Web Server function in QTS, and I do have it turned on with ports 80 and 8081 selected. But how do I connect that to COPS and then how do I get it connected for outside access using a domain name.

Oh yea, I do need to provide a secure login. With Pagekite, it is rudimentary, but I can set individual user usernames and passwords for each of the 'kite' connections I setup.

The reason I want to use the Qnap NAS is simply because I don't want to leave my PC on 24x7, while the NAS is a perfect platform for that!

Thanks!

[mariosipad]

@MontyJ

The QNAP TS-251 looks like a perfect platform for a COPS setup. so let's try to make it work!

For my own sake (so that I can correctly understand the problem) I rephrased your questions in a few points:

Q1 - Connect to server from the internet
- your external IP changes regularly (daily/monthly/???)
- you need to set up a Dynamic DNS and you choose PageKite
- you need to setup PageKite on your QNAP (normal install did not work)
- how to install PageKite on your QNAP

Q2 - "Secure" login to COPS
- multiple (2 or more) different users
- different content per user? (Not all users need to see all the the libraries, each have their own selection)

My answers:

First Q2 - Yes this is possible. I used this wiki page to make it work https://github.com/seblucas/cops/wiki/User-based-config
If you need further assistance in setting this up, just ask!

Then Q1 - You said that the installation did not work. [Disclaimer: I do not have a QNAP not have I ever installed PageKite]
- have you installed Python version 2.x on your QNAP?
- have you tried (http://pagekite.net/downloads)

If you prefer, you can download the program yourself and install wherever you like. Just remember to make the program executable by running: chmod +x pagekite.py

Link location: https://pagekite.net/pk/pagekite.pyhttps://pagekite.net/pk/pagekite.py
- put it in a nice directory (for example ./cops/pagekite) and run it (first SSH into your QNAP, but you knew that ) from that directory.

./pagekite.py &

And let us know if it works this way?

[MontyJ]

@mariosipad

Thanks for the comments/suggestions!

I went with this NAS because of the many helpful features, including music file transcoding. I have a quite large music library, tho not Calibre based of course, that I wanted to use that feature with...haven't got to much testing there yet.

On to the Q's

Q1. I live in Mexico and use Megacable for my ISP. The dynamic IP they provide changes 'often' due to a whole bunch of things:

- Power outtages are common, both locally for my residence and for their cable distribution system as well. When there is any glitch in the power that causes anything to reboot or reset, then my IP usually changes.

- My tendency to mess around with network devices, new network connections, etc, tend to require me to reboot routers and PC's often, usually, but not always, causing a IP change.

- Megacable seems to have a desire to prevent home webservers w/o a business plan and if their sniffer programs detect those kinds of efforts, they change up the iP often. I use VPN's and while they allow that, I get the feeling they do all they can to circumvent their use on home based connections as well.

Pagekite. This lets me setup fixed URLs for access. Currently on my Linux PC I simply run a Calibre Server daemon for each of the libraries, on different ports of course. But users have to juggle a different URL for each library if they want to use more than one library and most do; again on a sporadic basis. Pagekite does provide a way in the config file to set a username/password for each user. This way the 'security' happens before it gets to COPS. It's not a super secure system, but is enough for this simple setup.

I want to use the NAS in combination with COPS and Pagekite to now provide a SINGLE URL into the COPS menu, thanks to COPS serving multiple libraries in a single interface.

Q2. Perhaps 2 dozen users will sporadically need to access COPS. But they are all over the world, so I need to provide 24x7 access for them. The 'multi-library' feature of COPS is fantastic; a single connection to the COPS server port then lets uses select the library of choice. They are not 'assigned' to any specific library. Once they see the COPS user interface, they can, and usually do, use more than one library. It is rare, but I do need to accommodate multiple simultaneous users. The bandwidth requirements are low, and the ebook files are, mostly, less than 1Mb in size. But some PDF files in the libraries are quite large, say 30-50Mb in size. But again, simultaneous users downloading large files is very rare.

You seem to suggest I can indeed use the QTS Linux default OS to install COPS and Pagekite in. Yes, I have COPS installed and working for local access. But Pagekite is the problem.

Sorry for the babbling on, but wanted to clarify just what I am trying to do.

The QNAP Linux Station loads a mostly(?) standard Ubuntu 16.04 install. Pagekite does install there nicely, as does COPS of course. The problem is those large Calibre library databases and how to access them through that Linux install...

I will post later today the details of what I attempt and the errors that result when installing Pagekite in QTS. Since QTS is not 'standard' Linux, the fixed requirements of Pagekite seem to present a problem for installation. Pagekite IS Open Source, but like I said, modifying any of that is well beyond my capability. And so far, it seems to be more than a simple tweak of a config file, given the dependencies. I did query the Pagekite developer about tips/hints for QNAP install, but they did not bother answering.

[MontyJ]

First break!

Thanks to @mariosipad's suggestions, I have taken a step forward.

Somewhere along the line I had done a

$ python -V

and received a positive reply on v2.7.12. I kept on believing that was a valid install.

But after the prompt to check again, I actually looked for python files on the QTS fs. None found. But there were a bunch in the Ubuntu Linux Station install. Apparently the python version check did not limit itself to python in QTS, but checked in the Ubuntu install as well. Grrrrrr!

So, went into the app center on QNAP and sure enough, neither of the 2 python vers shown were installed. So installed v2.7.3. However, when I do a version check, it still shows only the v2.7.12, not v2.7.3. Well, pagekite likes whatever it is, so I am not going to argue or try to remove v2.7.12 unless it somehow gives me problems later.

I found a pagekite.py file in my linux pc backups and put it in

/usr/local/bin

Now when I enter pagekite.py, I get the normal intro menu with a blank setup; just as it should be.

I have all the COP files in the

Web/cops

folder. So now I need to get the pagekite.rc file up to snuff with my configuration so it can connect to the COPS server.

I will report back with those results.

Thanks again @mariosipad!

[MontyJ]

Update on QNAP, Pagekite, and COPS install.

I was able to get Pagekite at least running and reconfigured my original pagekite.rc config file for the single COPS server on port 80.

That whole rigamarole is working! Pagekite likes all the stuff and registers a valid domain name which then points to my setup properly.

However, when I go to that url with my browser, I get an Apache 403 error:

Forbidden

You don't have permission to access / on this server.

Apache Server at mydomainname.pagekite.me Port 80

I have no clue why I am getting this error. I did not have any problems with Apache on my Linux install on my pc using the Calibre Server. And pagekite does not require any port forwarding either.

Q1) So is it just a matter of telling Apache to allow access to "/" on QNAP? If so, how do I go about that? More importantly, what are the security ramifications doing this? I mean it sounds like 'root access to everything...'

Q2) Is port 80 the best place to have COPS on QNAP? Using the Calibre Server on Linux, I used ports 8182-8186 because of conflicts with the 'normal' ports being in use. Changing pots in pagekite is easy, but how to change in COPS if I need to?

Thanks!

[jillmess]

Originally Posted by MontyJ

Update on QNAP, Pagekite, and COPS install.

I was able to get Pagekite at least running and reconfigured my original pagekite.rc config file for the single COPS server on port 80.

That whole rigamarole is working! Pagekite likes all the stuff and registers a valid domain name which then points to my setup properly.

However, when I go to that url with my browser, I get an Apache 403 error:

Forbidden

You don't have permission to access / on this server.

Apache Server at mydomainname.pagekite.me Port 80

I have no clue why I am getting this error. I did not have any problems with Apache on my Linux install on my pc using the Calibre Server. And pagekite does not require any port forwarding either.

Q1) So is it just a matter of telling Apache to allow access to "/" on QNAP? If so, how do I go about that? More importantly, what are the security ramifications doing this? I mean it sounds like 'root access to everything...'

Q2) Is port 80 the best place to have COPS on QNAP? Using the Calibre Server on Linux, I used ports 8182-8186 because of conflicts with the 'normal' ports being in use. Changing pots in pagekite is easy, but how to change in COPS if I need to?

Thanks!

I don't know if any of the following applies with your pagekit setup but it my help.

Q1 On my qnap I have my COPS install in a folder Web/books. To access it from the internet I use 'http://mydomain.dyndns.org/books' If I want to access it without having to add the folder (in your case /cops) then I need to set up a Virtual Host - you can find this in the webserver setup area on the QNAP.

Q2 I am just using port 80 and a number of virtual hosts to access my different web pages, including Cops.

[mariosipad]

@MontyJ

After re-rereading some of your previous post I finally understand (hopefully) something about your "complicated" setup.

You use the QNAP Operating System "QTS" with Ubuntu "on top".

Where is COPS? In the QNAP webserver (/web/cops) or in a webserver (apache) in Ubuntu (/var/www/html/cops) and where are the Calibre libraries located?

If you access COPS locally from your PC (http://<internal-IP-QNAP>/cops) does it work?
If your QNAP has a screen attached to it and can run a browser can you access COPS (http://localhost/cops)? Or use remote access

From where is PageKite.py running and started? From QTS or from Ubuntu and with what user (root, admin, local-QTSuser, local Ubuntu-user)?
Then without fiddling to much with pagekite.rc (located in what home directory, QTS or Ubuntu /home/user?) where does PageKite "land"?
If you use port 80 it should "land" on http://localhost .
With http://mydomainname.pagekite.me/cops you should connect to http://localhost/cops

This does not answer your Q1, but ..........

With regards to Q2: You cannot change ports in COPS, but you can change ports in the webserver. But with PageKite there is no need. There are no ports open in your router for port forwarding. And that is quite secure!
Setting basic authentication (always a good idea) ca be done in COPS and or the webserver (.htaccess). Virtual hosts are also possible but not needed for your setup.

I hope this helps you think about a solution to your problem(s). First fix Q1 then Q2 is easy.

[mariosipad]

@jillmess

I hope you have secured your webserver access (on port 80) from the WWW with at least a username/password? (I'm 99.9% sure of this, but better safe than sorry)

[MontyJ]

Originally Posted by mariosipad

@MontyJ

After re-rereading some of your previous post I finally understand (hopefully) something about your "complicated" setup.

You use the QNAP Operating System "QTS" with Ubuntu "on top".

Where is COPS? In the QNAP webserver (/web/cops) or in a webserver (apache) in Ubuntu (/var/www/html/cops) and where are the Calibre libraries located?

If you access COPS locally from your PC (http://<internal-IP-QNAP>/cops) does it work?
If your QNAP has a screen attached to it and can run a browser can you access COPS (http://localhost/cops)? Or use remote access

From where is PageKite.py running and started? From QTS or from Ubuntu and with what user (root, admin, local-QTSuser, local Ubuntu-user)?
Then without fiddling to much with pagekite.rc (located in what home directory, QTS or Ubuntu /home/user?) where does PageKite "land"?
If you use port 80 it should "land" on http://localhost .
With http://mydomainname.pagekite.me/cops you should connect to http://localhost/cops

This does not answer your Q1, but ..........

With regards to Q2: You cannot change ports in COPS, but you can change ports in the webserver. But with PageKite there is no need. There are no ports open in your router for port forwarding. And that is quite secure!
Setting basic authentication (always a good idea) ca be done in COPS and or the webserver (.htaccess). Virtual hosts are also possible but not needed for your setup.

I hope this helps you think about a solution to your problem(s). First fix Q1 then Q2 is easy.

Mario,

Thanks for the detailed reply!

But I see I have possibly confused things, probably by posting unnecessary details, LoL.

1. Yes, my original question was using Ubuntu under QTS. But Ubuntu really isn't "on top" as you put it. It installs in its own "parallel" install. The Ubuntu fs is 'normal' linux with normal linux structures. The QTS fs is not 'normal' linux due to their unique tweaks to provide some pretty nice NAS features. The two fs's run simultaneously.

I did this because at that time I could not make Pagekite work in QTS. I installed what QNAP calls the Linux Station, which loads more or less 'normal' Ubuntu in its own fs space. I was then able to get Pagekite to work in Ubuntu, but could not figure out how to get COPS to see the Calibre libraries, which were located over in the QTS fs. That was my original starting questions scenario.

2. Then you gave me the idea to recheck if python was installed in QTS. It wasn't. So I went back to the QTS fs and installed python v2.7.3. That let me proceed with my original quest of getting COPS+Pagekite to work under QTS, not Ubuntu! We can leave Ubuntu out of the current setup question! The Apache 403 error I asked about is using COPS+Pagekite in the original QTS fs.

Ok, now on to location details.

When I SSH into the QTS fs, here are the paths/contents I get.

Initial entry to QNAP NAS at the "root" folder. I login as "admin":

[~] # ls -a
./                     .bashrc                .ssh@
../                    .buzzer_warnning.conf  .vimrc
.BitTornado@           .docker/               index_default.html
.bash_history          .pagekite.org          pagekite.log
.bash_logout           .pagekite.rc
.bash_profile          .profile
[~] #

NOTE: When I ran pagekite.py, this is the default location where it placed the (hidden) pagekite.rc config file and pagekite.log file as default. I guess I could move these files, but since Pagekite is working and seeing COPS, I want to leave it alone unless it is causing the Apache 403 error.

If I then go to "/" here is where the normal QTS fs shows up:

[~] # cd /
[/] # ls -la
drwxr-xr-x   22 admin    administ       540 Jan 15 09:26 ./
drwxr-xr-x   22 admin    administ       540 Jan 15 09:26 ../
lrwxrwxrwx    1 admin    administ        32 Jan 15 09:26 KVM -> /share/CACHEDEV1_DATA/.qpkg/QKVM/
drwxr-xr-x    2 admin    administ      1680 Jan 15 09:26 bin/
drwxr-xr-x   15 admin    administ     20160 Jan 16 03:00 dev/
drwxr-xr-x   28 admin    administ      3100 Jan 16 14:28 etc/
drwxr-xr-x    2 admin    administ        40 Jan 15 09:23 flashfs_tmp/
lrwxrwxrwx    1 admin    administ        13 Jan 15 09:23 hd_root_tmp -> /mnt/HDA_ROOT/
drwxr-xr-x    4 admin    administ        80 Dec 13 12:17 home/
drwxr-xr-x    6 admin    administ      2540 Jan 15 09:26 lib/
drwxr-xr-x    2 admin    administ       160 May  3  2015 lib64/
lrwxrwxrwx    1 admin    administ        11 Jan 15 02:22 linuxrc -> bin/busybox*
drwx------    2 admin    administ        40 Dec 13 12:18 lost+found/
drwxr-xr-x    9 admin    administ       200 Jan 16 14:28 mnt/
drwxr-xr-x    2 admin    administ        40 Jan 15 02:22 new_root/
lrwxrwxrwx    1 admin    administ        38 Jan 15 09:26 opt -> /share/CACHEDEV1_DATA/.qpkg/Entware-ng/
lrwxrwxrwx    1 admin    administ        19 Jan 15 02:22 php.ini -> /etc/config/php.ini
dr-xr-xr-x  428 admin    administ         0 Jan 15 02:22 proc/
drwxr-xr-x    3 admin    administ       320 Jan 17 10:16 root/
drwxr-xr-x    2 admin    administ        40 Jan 15 09:25 rpc/
drwxr-xr-x    5 admin    administ       100 Jan 15 09:27 run/
drwxr-xr-x    2 admin    administ      8620 Jan 15 09:26 sbin/
drwxrwxrwt   31 admin    administ       740 Jan 15 09:25 share/
dr-xr-xr-x   12 admin    administ         0 Jan 15 02:22 sys/
drwxrwxrwx   19 admin    administ      3060 Jan 17 10:40 tmp/
drwxr-xr-x   10 admin    administ       200 Jan 16 14:29 usr/
drwxrwxrwx   10 admin    administ       660 Jan 17 10:39 var/
[/] #

[/] # cd home
[/home] # ls -l
drwxr-xr-x    3 admin    administ       180 Jan 15 09:25 Qhttpd/
drwxr-xr-x   12 admin    administ       660 Jan 15 09:26 httpd/
[/home] #

NOTE: The question about a "home" install for pagekite came up. This is the contents of "home" and there are no other subfolders here.


Ok, the actual pwd paths to various stuff.

COPS location. Note the Ubuntu location shows up as well. It is not used currently!:

[/] # find / -name COPS -type d
/share/CACHEDEV1_DATA/.qpkg/COPS
/share/CACHEDEV1_DATA/.qpkg/COPS/COPS
/share/CACHEDEV1_DATA/.qpkg/ubuntu-hd/lxc/ubuntu_1604/rootfs/home/admin/Web/COPS
find: ‘/proc/12211’: No such file or directory
[/] # 
[/] # cd /share/CACHEDEV1_DATA/Web/COPS
[/share/CACHEDEV1_DATA/Web/COPS] # pwd
/share/CACHEDEV1_DATA/Web/COPS

Ok, here is where pagekite is located:

[/] # find / -name pagekite.py -type f
/share/CACHEDEV1_DATA/.qpkg/ubuntu-hd/lxc/ubuntu_1604/rootfs/usr/local/bin/pagekite.py
/usr/local/bin/pagekite.py
[/] #

Again, pagekite.py shows up in the Ubuntu area, but ASFAIK, it is not being used. I put it in a matching directory only for conformity. I guess it could be put anywhere as long as it is in the admins run path. The Ubuntu install will be removed if I can get it to work here in the QTS fs, and it looks like I am very close.

So, there are the details. Hopefully they help. I tend to overdue the details as I am pretty new at this and want to eliminate assumptions by filling in all the gaps I can.

Here is what I see when I run pagekite.py:

[/] # pagekite.py
>>> Hello! This is pagekite.py v0.5.8e.                         [CTRL+C = Stop]
 Connecting to front-end 173.230.xxx.xxx:xxx ...                            
     - Protocols: http http2 http3 https websocket irc finger httpfinger raw   
     - Protocols: minecraft                                                    
     - Ports: 79 80 443 843 2222 3000 4545 5222 5223 5269 5670 6667 8000 8080  
     - Ports: 8081 9292 25565                                                  
     - Raw ports: 22 virtual                                                   
    Quota: (my personal acct data)              
    Connecting to front-end 35.160.xxx.xxx:xxx ...                             
~<> Flying localhost:80 as https://mydomain.pagekite.me/                 
 << pagekite.py [dyndns]   Updating DNS for mydomain.pagekite.me...

Since it appears to be working at this point, I point my browser to:

mydomain.pagekite.me

Then I get a Pagekite login window. The users and passwords are setup in in the pagekite config file. The login windows shows:

"Authentication Required"
http://mydomain.pagekite.me is requesting your username and password. The site says: “PageKite”

Then I simply enter my username and password in the fields provided.

This is where I get the Apache 403 error:

Forbidden 

You don't have permission to access / on this server. 

Apache Server at mydomain.pagekite.me Port 80

Whew! Too many details I know, Hopefully enough to point to what I did wrong, or haven't done at all that I should!

[mariosipad]

@MontyJ

Thank you for your very elaborate post.

I do not see anything obviously wrong, but I must admit that I do not understand everything I read.

Can you:
1 - post the outcome of a "ls -la" of the ./web directory.
2 - type in a browser: https://mydomain.pagekite.me/cops/index.php

Maybe it's just a permissions problem, if not .........