Help with Calibre Portable and Linux - Page 6

kacir · 03-30-2015, 03:23 PM

Quote:

Originally Posted by bambuko

Further update:
Tried it with PClinuxOS LXDE and it fails if I run as user, but is perfectly happy if I run it as root.
No biggie - I am happy to run it as root

Be careful.
You run Calibre as a root once and then you always have to run it as root, because any changes you make will result in files where owner is root.
Then you will want to use the library as an ordinary user and Bad Things (TM) start to happen. Your Calibre will start to complain that it can't update some books, and other weird things.
Then you will spend a week investigating the issue, suspecting that your filesystem got damaged, or that your library integrity has been compromised.

Fortunately you can change owner for all files and directories from a command line, or using a good file manager (such as mc).
Do not change the file permissions with chmod 777 as some posters suggested.

It is not a good idea to run things as root. Believe me. Been there, done that, lived to tell the tale ;-)

Changing the owner - besides using sudo mc
sudo chown -R me /home/me/Calibre\ Library
or
find /home/me/Calibre\ Library/ -exec chown me {}
or something like that. Be careful, the above examples have to be tested that they do what you think they do.

eschwartz · 03-30-2015, 03:39 PM

Why would you not want to use chmod 777??? What exactly are you warning against?

chaley · 03-30-2015, 04:14 PM

Quote:

Originally Posted by eschwartz

Why would you not want to use chmod 777??? What exactly are you warning against?

Is this a trick question?

There are many reasons one should not use 777, which is equivalent to giving every user and the web server all access to those files. If there is a hole in some web server application then all those 777 files are exposed. If someone manages to hijack an account then all those 777 files are exposed. I am sure that any machine/VPS you manage are like mine, being probed hundreds, sometimes thousands, of times per day by bots doing dictionary attacks and probing web-visible vulnerabilities. Why make it easier for them?

As a side note, many versions of tar will by default restore uid/gids from the archive if run as root. This could account for the strange numbers being seen.

theducks · 03-30-2015, 04:15 PM

Quote:

Originally Posted by eschwartz

Why would you not want to use chmod 777??? What exactly are you warning against?

777 is using a sledgehammer. I see no pressing reason to give 'World' any more than Read rights

eschwartz · 03-30-2015, 06:51 PM

Then you are bothall missing the most basic and fundamental point of this whole entire thread.

This is a portable version of calibre. It is meant to be run from a flashdrive, designed for use across computers with different user accounts and G/UIDs.

In such a case, the only solution is world read/write. The alternative is using vfat... which is also world read/write!

And yes, when you unplug and remount the drive, it easily becomes owned by the attacker, that is chmod 777 as far as I am concerned.
Are you freaking kidding me, talking about security on a home computer when using external hard drives?

In fact, I committed changes to the calibre-portable.sh launcher, that are designed to ensure all new files created under the scope of that launcher are created 777, in order that you can actually use the darn thing in the first place! Without making fstab rules or patching udisks-daemon to stop mounting vfat as noexec...

In this case, running chmod 777 is merely playing catch-up to my change.

bambuko · 03-30-2015, 06:52 PM

Just got back from the pub and...
all the copying complete and everything runs fine in Puppy Linux (where I did copying etc), but when I get to a "grown up Linux" like PCLinuxOS all goes to pot and doesn't run unless I run it as root

I am sure it is because I have now copied a library with owner as set by backup copy (owner=1024) and other bits as installed on the stick (owner=root)

Library owner is 1024 because that's what is set up by backup OS, so that's how it gets copied.

Puppy Linux is OK because it runs as root by default, PCLinuxOS is causing grief because it is "proper" Linux and gets confused as to the owners bit.

I am too pissed to think straight right now.
Back tomorrow...

bambuko · 03-30-2015, 06:55 PM

Quote:

Originally Posted by eschwartz

Then you are both missing the most basic and fundamental point of this whole entire thread.

This is a porrtable version of calibre. It is meant to be run from a flashdrive, designed for use across computers with different user accounts and G/UIDs.

In such a case, the only solution is world read/write. The alternative is using vfat... which is also world read/write!

And yes, when you unplug and remount the drive, it easily becomes owned by the attacker, that is chmod 777 as far as I am concerned.

In fact, I committed changes to the calibre-portable.sh launcher, that are designed to ensure all new files created under the scope of that launcher are created 777, in order that you can actually use the darn thing in the first place! Without making fstab rules or patching udisks-daemon to stop mounting vfat as noexec...

In this case, running chmod 777 is merely playing catch-up to my change.

Don't worry - you got the idea

It is meant to be portable

There is always some guy in Linux forum that will spread doom and gloom.
We are grown up adults having fun.
Go away doom mongers

chaley · 03-31-2015, 03:40 AM

Quote:

Originally Posted by eschwartz

Are you freaking kidding me, talking about security on a home computer when using external hard drives?

Actually, I don't care if your machine gets penetrated. I do care that it joins the army of bots sending spam, running DDS attacks, and generally making life unpleasant.

Quote:

In fact, I committed changes to the calibre-portable.sh launcher, that are designed to ensure all new files created under the scope of that launcher are created 777, in order that you can actually use the darn thing in the first place! Without making fstab rules or patching udisks-daemon to stop mounting vfat as noexec...

Without looking at the patch, I hope that the executables are 755 or better 555 to prevent drive-by infection.

Quote:

Originally Posted by bambuko

Don't worry - you got the idea

It is meant to be portable

There is always some guy in Linux forum that will spread doom and gloom.
We are grown up adults having fun.
Go away doom mongers

Well, this discussion got personal and insulting faster than most. Hope you have lots of fun.

eschwartz · 03-31-2015, 04:08 AM

Quote:

Originally Posted by chaley

Actually, I don't care if your machine gets penetrated. I do care that it joins the army of bots sending spam, running DDS attacks, and generally making like unpleasant.
Without looking at the patch, I hope that the executables are 755 or better 555 to prevent drive-by infection.

In what way is an ext3 external hard drive/flashdrive partition with ebook files that can be written to by world, a security risk that will lead to my computer becoming part of a botnet?

In what way is this any more dangerous than saving the files to a vfat partition? The sum total of differences is that it will not be mounted noexec, thus aiding portability... because the alternative was saving to vfat.
I desire the files to be vulnerable to drive-by-infection... it requires physical access to the hardware, and the user who uses the drive is drive-by-infecting it with new ebooks and updated metadata -- because said user sure as hell isn't the original owner.

Likewise, the executables must be 777 to allow the user to update calibre.
How is it dangerous for the calibre executables to be editable by world, when to edit them, an attacker must have already pwned your system?
Or stolen the physical drive, which means your security has instantly become a joke.

What precisely is your recommendation for portableizing calibre in a "safer" manner? Keep in mind that data on removable drives is not very secure anyway. Keep in mind that by design, any user who plugs the flashdrive in must be able to use it.

chaley · 03-31-2015, 08:21 AM

Quote:

Originally Posted by eschwartz

In what way is an ext3 external hard drive/flashdrive partition with ebook files that can be written to by world, a security risk that will lead to my computer becoming part of a botnet?

Many penetrations come through lesser-privilege vectors, such as SSH/SMTP/POP/telnet via dictionary attacks, the web server, web apps, a user running infected downloads, and infected portable drives. Many of these penetrations do at least two things once they get in. First, they phone home with a summary of the executables on the machine to see if there are any unpatched local escalation vulnerabilities. Second, they scan the machine for writable executables and infect them, inserting themselves into them. The infection attempts to be persistent, remaining in memory and restarting after a boot, becoming a less-privileged bot. While it is running the scan happens whenever something new is mounted. If an executable is infected and then some other user ID runs one the infected executables then the process is repeated with the new permissions. If the infected programs are run on a different machine then the process starts anew. If root ever runs an infected exe then the machine is toast.

My assumption is that because we are talking about "calibre portable", the "drive" will be used in various machines and the drive contains the calibre executables. If any one of the machines is infected with something like described above then mode 777 calibre exes on the drive will be infected. The infection will transfer to any subsequent machine where the newly-infected program is run.

The basic way to plug this sort of lesser-privilege vulnerability beyond blocking the entry vectors is to ensure that executables (or other important system files such as crontabs or password files) cannot be modified except under controlled conditions. How you do this, or whether you do it at all, is up to you.

eschwartz · 03-31-2015, 03:27 PM

That is indeed the risk you run when using a PortableApp. Nothing new here.

Note that if the executables are not discovered on the drive itself then the launcher will fallback on the system search path. I assume anyone storing program binaries on a flashdrive is aware that there is a level of risk.
However, it is necessary in order to be fully portable (the goal of the launcher is to give choice) that the user is permitted to choose where to store the program binaries.

The way I see it, there is only a net gain over storing it on a vfat drive.

bambuko · 03-31-2015, 07:19 PM

Back to portable business

I have sorted out the owner business (as you have said it looks like it was something to do with where I kept the backup copy of the library, where I created the usb portable calibre stick etc) - it is fine now and all the files, and directories have the same owner.

So no longer complaints about corrupted database.
It runs fine in Puppy Linux (because Puppy always runs as root)
It runs fine in PCLinuxOS if I run it as root:

Code:

CONFIG FILES: /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreConfig
--------------------------------------------------
LIBRARY FILES: /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreLibrary
--------------------------------------------------
SOURCE FILES: *** Not being Used ***
--------------------------------------------------
PROGRAM FILES: /media/995de026-70b2-4991-a641-eb0067f25a5a/calibre
--------------------------------------------------
TEMPORARY FILES: /tmp/CALIBRE_TEMP
--------------------------------------------------

Press CTRL-C if you do not want to continue
Press ENTER to continue and start Calibre

Starting up Calibre from portable directory "/media/995de026-70b2-4991-a641-eb0067f25a5a"
Using library at /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreLibrary
libpng warning: iCCP: Not recognizing known sRGB profile that has been edited

but when run as user, it starts kind of OK? but with this messages:

Code:

Press CTRL-C if you do not want to continue
Press ENTER to continue and start Calibre

Starting up Calibre from portable directory "/media/995de026-70b2-4991-a641-eb0067f25a5a"
Using library at /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreLibrary
Failed to initialize plugin: ISBNDB (1, 0, 0)
Failed to initialize plugin: <class 'calibre.ebooks.metadata.sources.isbndb.ISBNDB'>
Exception in thread Thread-1:
Traceback (most recent call last):
  File "threading.py", line 810, in __bootstrap_inner
  File "site-packages/calibre/utils/fonts/scanner.py", line 231, in run
  File "site-packages/calibre/utils/fonts/scanner.py", line 280, in do_scan
  File "site-packages/calibre/utils/fonts/scanner.py", line 350, in write_cache
  File "site-packages/calibre/utils/config.py", line 363, in __exit__
  File "site-packages/calibre/utils/config.py", line 352, in commit
  File "site-packages/calibre/utils/lock.py", line 139, in __enter__
  File "site-packages/calibre/utils/lock.py", line 118, in unix_open
OSError: [Errno 13] Permission denied: '/media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreConfig/fonts/scanner_cache.json'

libpng warning: iCCP: Not recognizing known sRGB profile that has been edited

eschwartz · 04-01-2015, 01:33 AM

I don't know, but since CALIBRE_TEMP_DIR was never exported

(holdover I didn't notice) that wouldn't affect anything.
bash scripts don't default to exiting on the first error either.

I fixed the brokem tempdir problem, in case you want to try my latest anyway.

It also now writes a configuration file containing the settings. Do you think it is easier to see what settings there are now?

bambuko · 04-01-2015, 04:16 AM

Thank you!
I have run it as user (it started fine) but these were the messages in terminal:

Code:

/media/995de026-70b2-4991-a641-eb0067f25a5a/calibre-portable.sh: line 113: $(pwd)/calibre-portable.conf: Permission denied
CONFIG FILES:       /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreConfig
--------------------------------------------------
LIBRARY FILES:      /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreLibrary
--------------------------------------------------
SOURCE FILES:       *** Not being Used ***
--------------------------------------------------
PROGRAM FILES:      /media/995de026-70b2-4991-a641-eb0067f25a5a/calibre
--------------------------------------------------
TEMPORARY FILES:    /tmp/CALIBRE_TEMP_yI74c0m
--------------------------------------------------

Press CTRL-C if you do not want to continue
Press ENTER to continue and start Calibre

Starting up Calibre from portable directory "/media/995de026-70b2-4991-a641-eb0067f25a5a"
Using library at /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreLibrary
Failed to initialize plugin: ISBNDB (1, 0, 0)
Failed to initialize plugin: <class 'calibre.ebooks.metadata.sources.isbndb.ISBNDB'>
Exception in thread Thread-1:
Traceback (most recent call last):
  File "threading.py", line 810, in __bootstrap_inner
  File "site-packages/calibre/utils/fonts/scanner.py", line 231, in run
  File "site-packages/calibre/utils/fonts/scanner.py", line 280, in do_scan
  File "site-packages/calibre/utils/fonts/scanner.py", line 350, in write_cache
  File "site-packages/calibre/utils/config.py", line 363, in __exit__
  File "site-packages/calibre/utils/config.py", line 352, in commit
  File "site-packages/calibre/utils/lock.py", line 139, in __enter__
  File "site-packages/calibre/utils/lock.py", line 118, in unix_open
OSError: [Errno 13] Permission denied: '/media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreConfig/fonts/scanner_cache.json'

libpng warning: iCCP: Not recognizing known sRGB profile that has been edited

and... running as root:

Code:

CONFIG FILES:       /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreConfig
--------------------------------------------------
LIBRARY FILES:      /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreLibrary
--------------------------------------------------
SOURCE FILES:       *** Not being Used ***
--------------------------------------------------
PROGRAM FILES:      /media/995de026-70b2-4991-a641-eb0067f25a5a/calibre
--------------------------------------------------
TEMPORARY FILES:    /tmp/CALIBRE_TEMP_48ekKrq
--------------------------------------------------

Press CTRL-C if you do not want to continue
Press ENTER to continue and start Calibre

Starting up Calibre from portable directory "/media/995de026-70b2-4991-a641-eb0067f25a5a"
Using library at /media/995de026-70b2-4991-a641-eb0067f25a5a/CalibreLibrary
libpng warning: iCCP: Not recognizing known sRGB profile that has been edited

Quote:

...Do you think it is easier to see what settings there are now? ...

Yes, I think this is an improvement!

eschwartz · 04-01-2015, 10:07 AM

The umask appears to have done nothing at all, calibre ignored it.

I set a trap to explicitly chmod everything, it is a bit safer now since even CTRL+C in the terminal will perform cleanup.

Try again, remembering to chmod everything that already exists.

03-30-2015, 06:51 PM	#80
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Then you are bothall missing the most basic and fundamental point of this whole entire thread. This is a portable version of calibre. It is meant to be run from a flashdrive, designed for use across computers with different user accounts and G/UIDs. In such a case, the only solution is world read/write. The alternative is using vfat... which is also world read/write! And yes, when you unplug and remount the drive, it easily becomes owned by the attacker, that is chmod 777 as far as I am concerned. Are you freaking kidding me, talking about security on a home computer when using external hard drives? In fact, I committed changes to the calibre-portable.sh launcher, that are designed to ensure all new files created under the scope of that launcher are created 777, in order that you can actually use the darn thing in the first place! Without making fstab rules or patching udisks-daemon to stop mounting vfat as noexec... In this case, running chmod 777 is merely playing catch-up to my change. Last edited by eschwartz; 03-30-2015 at 06:56 PM.

04-01-2015, 01:33 AM	#88
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	I don't know, but since CALIBRE_TEMP_DIR was never exported (holdover I didn't notice) that wouldn't affect anything. bash scripts don't default to exiting on the first error either. I fixed the brokem tempdir problem, in case you want to try my latest anyway. It also now writes a configuration file containing the settings. Do you think it is easier to see what settings there are now? Last edited by eschwartz; 04-01-2015 at 01:39 AM.

04-01-2015, 10:07 AM	#90
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	The umask appears to have done nothing at all, calibre ignored it. I set a trap to explicitly chmod everything, it is a bit safer now since even CTRL+C in the terminal will perform cleanup. Try again, remembering to chmod everything that already exists. Last edited by eschwartz; 04-01-2015 at 10:20 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre Portable	cstandifird	Devices	3	01-24-2012 02:29 PM
Portable Calibre?	hermes	Related Tools	2	06-02-2011 02:50 PM
Calibre Portable	beckywc	Calibre	4	12-21-2010 11:13 AM
Portable Linux version of Sigil	readx	Sigil	9	09-03-2010 02:59 AM
Portable Linux-Mobile-Guide for iSilo and Plucker	Colin Dunstan	Lounge	0	08-31-2005 05:48 AM

03-30-2015, 03:39 PM	#77
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Why would you not want to use chmod 777??? What exactly are you warning against?

03-30-2015, 06:52 PM	#81
bambuko Connoisseur Posts: 62 Karma: 10 Join Date: Feb 2014 Device: Kobo mini, Kobo Clara HD	Just got back from the pub and... all the copying complete and everything runs fine in Puppy Linux (where I did copying etc), but when I get to a "grown up Linux" like PCLinuxOS all goes to pot and doesn't run unless I run it as root I am sure it is because I have now copied a library with owner as set by backup copy (owner=1024) and other bits as installed on the stick (owner=root) Library owner is 1024 because that's what is set up by backup OS, so that's how it gets copied. Puppy Linux is OK because it runs as root by default, PCLinuxOS is causing grief because it is "proper" Linux and gets confused as to the owners bit. I am too pissed to think straight right now. Back tomorrow...

03-31-2015, 03:27 PM	#86
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	That is indeed the risk you run when using a PortableApp. Nothing new here. Note that if the executables are not discovered on the drive itself then the launcher will fallback on the system search path. I assume anyone storing program binaries on a flashdrive is aware that there is a level of risk. However, it is necessary in order to be fully portable (the goal of the launcher is to give choice) that the user is permitted to choose where to store the program binaries. The way I see it, there is only a net gain over storing it on a vfat drive.

Advert

Advert