How to set CALIBRE_TEMP_DIR on Linux?

excaliber · 06-27-2016, 12:48 PM

Hi all,

I want to change the temporary directory used by calibre. According to the manual:
https://manual.calibre-ebook.com/customize.html
this is controlled by the environment variable CALIBRE_TEMP_DIR. I did:
$export CALIBRE_TEMP_DIR=/new/path/
but Calibre insists of using the old path (/tmp).

How can I set CALIBRE_TEMP_DIR on Linux?
Any ideas?

DiapDealer · 06-27-2016, 01:21 PM

That's only going to work for the current terminal you have open. You can't run the export command and then click on a calibre shortcut. You'd have to launch calibre from that same terminal by typing: "calibre" and hitting enter (no quotes).

Try just setting it and running calibre at the same time from a termnal:
CALIBRE_TEMP_DIR=/new/path calibre

If it works, you can just create a bash script to launch calibre. Or if you want something more permanent, add 'export CALIBRE_TEMP_DIR=/new/path' to the .profile file in your home directory. Then you can launch calibre however you want (after you log back in).

excaliber · 06-28-2016, 03:06 AM

Thanks for the quick response!

Try just setting it and running calibre at the same time from a termnal:
CALIBRE_TEMP_DIR=/new/path calibre

This has worked as you said! I haven't tried the other solution with the .profile file yet but I suppose it will work too.

DaltonST · 06-28-2016, 05:24 AM

@excaliber:

Since you use ZMI, please read the section in the Original Post entitled "Special Operating System Notes". The bottom half pertains to the path for temporary files. ZMI will not work unless you do what it says about temporary files and the related Python environment variables that must be consistent with the Calibre temp file environment variable.

DaltonST

excaliber · 06-28-2016, 05:43 AM

Since you use ZMI, please read the section in the Original Post entitled "Special Operating System Notes". The bottom half pertains to the path for temporary files. ZMI will not work unless you do what it says about temporary files and the related Python environment variables that must be consistent with the Calibre temp file environment variable.

I have read the "Special Operating System Notes" for ZMI. What does that mean? Am I stuck with some standard paths? Is this python, Calibre or ZMI related?
I have a around 50 GB for the root partition and around 1 TB for the home partition. I think some plugins in Calibre crash because they need some GB and and there is no space. I have started Calibre now in a terminal with a count pages job and it's writing to the new temp path in my home partition.

DaltonST · 06-28-2016, 07:22 AM

It means that you have to set extra environment variables specific to Python, and they must be consistent with your Calibre environment variables. ZMI is a Calibre plug-in, so it uses Python. Temporary files in Python are put in the directory based on the environment variables for Python temporary files.

Quote:

Am I stuck with some standard paths?

Absolutely not. That is the point. Just do what it says about Python environment variables.

DaltonST

DiapDealer · 06-28-2016, 07:57 AM

Quote:

Originally Posted by DaltonST

It means that you have to set extra environment variables specific to Python, and they must be consistent with your Calibre environment variables. ZMI is a Calibre plug-in, so it uses Python. Temporary files in Python are put in the directory based on the environment variables for Python temporary files.

To my knowledge, calibre (and its plugins) ignore any system python environment variables--at least when installed via the recommended calibre installation script. Calibre's python is isolated from any system-installed version of python. So why would setting a system python environment variable affect a calibre plugin? Unless I'm missing something?

DaltonST · 06-28-2016, 09:29 AM

Quote:

Originally Posted by DiapDealer

To my knowledge, calibre (and its plugins) ignore any system python environment variables--at least when installed via the recommended calibre installation script. Calibre's python is isolated from any system-installed version of python. So why would setting a system python environment variable affect a calibre plugin? Unless I'm missing something?

It has nothing to do with a "system-installed version of Python". It has to do with Calibre's Python. If you use any of my plug-ins or add-ons that generate calibredb subprocesses (ZMI, CALM, Q&S, and Library Codes), the instructions in their Original Posts must be followed viz a viz generic Python temporary file environment variables, or they will fail for a variety of reasons. In Windows, it would be a security/permissions issue, since the default temporary directory would likely be the directory where calibredb.exe exists.

DaltonST

DiapDealer · 06-28-2016, 10:22 AM

I'm still not following at all (but then I don't have to, either). Subprocess or no, calibredb won't know a thing about any generic Python temporary environment variables. Nor will calibre or calibre-debug. Calibre's Python (including plugins) is launched with specific instructions to ignore any outside Python environment variables. For good reason.

Never mind: My confusion lies with the fact that you keep implying that the environment variables need to be set for the PLUGIN to work. When in fact, the environment variables need to be set so your PROGRAM (and its Python subprocesses) interacts with a calibre plugin correctly.

eschwartz · 06-28-2016, 11:29 AM

Of course, the calibre plugin in question could always get the temporary directory directly from calibre ('from calibre.ptempfile import base_dir') and then do whatever it wants with that. e.g. by setting that as the value of $TMP in the environment of the subprocesses. Or whatever it is the plugin is doing.

(Maybe I am missing something? It seems to be indicated that the subprocess runs calibredb. And the statement "the default temporary directory would likely be the directory where calibredb.exe exists" is just completely and utterly wrong, or else no one would ever be able to run calibre, ever. And for good measure, a third "ever".)

...

For some reason, DaltonST's plugins always seem to have manual steps that should be unnecessary with a bit of automation?

DiapDealer · 06-28-2016, 12:25 PM

Quote:

Originally Posted by eschwartz

Maybe I am missing something? It seems to be indicated that the subprocess runs calibredb.

Just a WAG on my part, but I'm assuming info is being shared (via the temp directory) between his PROGRAM (which launches calibredb as a python subprocess) and his PLUGIN. So the program needs an env var set in order for his program to create files in the same place the plugin is going to look for them (or vice versa). But I agree, there has to be a better way of getting the job done than manually making sure the scratch directories of two different Python processes jive.

DaltonST · 06-28-2016, 04:21 PM

Quote:

Never mind: My confusion lies with the fact that you keep implying that the environment variables need to be set for the PLUGIN to work. When in fact, the environment variables need to be set so your PROGRAM (and its Python subprocesses) interacts with a calibre plugin correctly.

Nope. ZMI is a GUI Plug-in. No executable program is involved, other than calibredb.exe and calibre.exe. It does, however, use whatever versions of Python, APSW and Qt that Calibre is currently bundling. It has to. It is a GUI Plug-in.

Assume that the user uses a RAMDISK via CALIBRE_TEMP_DIR. If ZMI used the same temporary folder as CALIBRE_TEMP_DIR, the RAMDISK could potentially overflow while ZMI is processing, for example, 15,000 .pdf files containing images and text, but all at one time (as inadvisable as that is).

So, the base case is to make TMPDIR = CALIBRE_TEMP_DIR, but to handle the above scenario, they will have to make TMPDIR point to a hard drive, not to a RAMDISK. Calibre will work off of the RAMDISK, but just ZMI (a Calibre GUI plug-in) will work off of the hard drive.

TMPDIR is the "goto" first choice for Python when looking for where to create a temporary file. The user could pick and choose among several other environment variables and directories, of course. That is up to them.

It is just as trivial to set TMPDIR = CALIBRE_TEMP_DIR as it is trivial to set CALIBRE_TEMP_DIR = ...\some_path\...

The cost is trivial, and the flexibility given the user to manage their computer resources the way they wish is extremely high.

DaltonST

eschwartz · 06-28-2016, 04:46 PM

And you could choose to import several thousand books into calibre all at once, too -- and flood the temporary directory in the same way. What is your point?
A user who uses a ramdisk for their $TMP has to be careful with or without your plugin. It makes sense to have one preference, as opposed to unnecessarily complicating the setup instructions.

Or, you could just say

that a user will have to set $TMPDIR if they want to process many files at once. Rather than claiming that it is down to permission issues (which if you use calibre's temporary folder is not a problem, or alternatively it is a problem that breaks calibre in general). Or claiming that it will default to "C:\Program Files\Calibre2\" for no comprehensible reason.

There you go. Flexibility without any cost whatsoever.

DaltonST · 06-28-2016, 05:04 PM

Generic Python defaults to the current working directory of the program executing it as a last choice to create a temporary file. In Windows, for Calibre, that is "C:\Program Files\Calibre2\". Windows would throw a security/permissions error. Windows users are 85% of Calibre users. Simply reading the Original Post section about the ins-and-outs of environment variables and doing what they want to do eliminates that issue entirely.

eschwartz · 06-28-2016, 05:53 PM

Um, the default is %TEMP% which is created by your user account.

And Windows users are unlikely to have manually installed a ramdisk driver and redirected %TEMP% to it... but if they did, they would know how to deal with it. So there is no need to worry on behalf of Windows users that thousands of files will clobber their RAM.

And you still haven't explained what is wrong with relying on calibre's temp dir (which must be writable for calibre to work in general, plugin or no plugin).

...

But if you really, really want to impress your users by writing and enforcing complicated instructions, go right ahead. I have no beef in this, I am just commenting on what I regard as shaky logic.

06-27-2016, 12:48 PM	#1
excaliber Connoisseur Posts: 59 Karma: 10 Join Date: Nov 2013 Device: Samsung Galaxy Tab 2 10.1 P5110	How to set CALIBRE_TEMP_DIR on Linux? Hi all, I want to change the temporary directory used by calibre. According to the manual: https://manual.calibre-ebook.com/customize.html this is controlled by the environment variable CALIBRE_TEMP_DIR. I did: $export CALIBRE_TEMP_DIR=/new/path/ but Calibre insists of using the old path (/tmp). How can I set CALIBRE_TEMP_DIR on Linux? Any ideas?

06-28-2016, 10:22 AM	#9
DiapDealer Grand Sorcerer Posts: 28,561 Karma: 204127028 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	I'm still not following at all (but then I don't have to, either). Subprocess or no, calibredb won't know a thing about any generic Python temporary environment variables. Nor will calibre or calibre-debug. Calibre's Python (including plugins) is launched with specific instructions to ignore any outside Python environment variables. For good reason. Never mind: My confusion lies with the fact that you keep implying that the environment variables need to be set for the PLUGIN to work. When in fact, the environment variables need to be set so your PROGRAM (and its Python subprocesses) interacts with a calibre plugin correctly. Last edited by DiapDealer; 06-28-2016 at 10:40 AM.

06-28-2016, 11:29 AM	#10
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Of course, the calibre plugin in question could always get the temporary directory directly from calibre ('from calibre.ptempfile import base_dir') and then do whatever it wants with that. e.g. by setting that as the value of $TMP in the environment of the subprocesses. Or whatever it is the plugin is doing. (Maybe I am missing something? It seems to be indicated that the subprocess runs calibredb. And the statement "the default temporary directory would likely be the directory where calibredb.exe exists" is just completely and utterly wrong, or else no one would ever be able to run calibre, ever. And for good measure, a third "ever".) ... For some reason, DaltonST's plugins always seem to have manual steps that should be unnecessary with a bit of automation? Last edited by eschwartz; 06-28-2016 at 11:43 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Windows vs Linux Metadata Summary Lacking in Linux	TGPB	Library Management	5	04-10-2014 10:56 AM
[Linux] Calibre on Linux isn't detecting any of my Android devices	Amogh Harish	Devices	9	10-31-2012 05:45 AM
Linux calibre-server: How to set username/password?	qupfer	Calibre	0	12-08-2011 04:54 PM
Set time on Kindle keyboard with usbnetworking and linux terminal	bibiboo	Kindle Developer's Corner	6	11-21-2011 02:11 AM
Consortium is to set standards for Linux-based smartphones	Colin Dunstan	Lounge	0	11-14-2005 10:31 AM

06-27-2016, 01:21 PM	#2
DiapDealer Grand Sorcerer Posts: 28,561 Karma: 204127028 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	That's only going to work for the current terminal you have open. You can't run the export command and then click on a calibre shortcut. You'd have to launch calibre from that same terminal by typing: "calibre" and hitting enter (no quotes). Try just setting it and running calibre at the same time from a termnal: CALIBRE_TEMP_DIR=/new/path calibre If it works, you can just create a bash script to launch calibre. Or if you want something more permanent, add 'export CALIBRE_TEMP_DIR=/new/path' to the .profile file in your home directory. Then you can launch calibre however you want (after you log back in).

06-28-2016, 03:06 AM	#3
excaliber Connoisseur Posts: 59 Karma: 10 Join Date: Nov 2013 Device: Samsung Galaxy Tab 2 10.1 P5110	Thanks for the quick response! Try just setting it and running calibre at the same time from a termnal: CALIBRE_TEMP_DIR=/new/path calibre This has worked as you said! I haven't tried the other solution with the .profile file yet but I suppose it will work too.

06-28-2016, 05:24 AM	#4
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	@excaliber: Since you use ZMI, please read the section in the Original Post entitled "Special Operating System Notes". The bottom half pertains to the path for temporary files. ZMI will not work unless you do what it says about temporary files and the related Python environment variables that must be consistent with the Calibre temp file environment variable. DaltonST

06-28-2016, 05:43 AM	#5
excaliber Connoisseur Posts: 59 Karma: 10 Join Date: Nov 2013 Device: Samsung Galaxy Tab 2 10.1 P5110	Since you use ZMI, please read the section in the Original Post entitled "Special Operating System Notes". The bottom half pertains to the path for temporary files. ZMI will not work unless you do what it says about temporary files and the related Python environment variables that must be consistent with the Calibre temp file environment variable. I have read the "Special Operating System Notes" for ZMI. What does that mean? Am I stuck with some standard paths? Is this python, Calibre or ZMI related? I have a around 50 GB for the root partition and around 1 TB for the home partition. I think some plugins in Calibre crash because they need some GB and and there is no space. I have started Calibre now in a terminal with a count pages job and it's writing to the new temp path in my home partition.

06-28-2016, 04:46 PM	#13
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	And you could choose to import several thousand books into calibre all at once, too -- and flood the temporary directory in the same way. What is your point? A user who uses a ramdisk for their $TMP has to be careful with or without your plugin. It makes sense to have one preference, as opposed to unnecessarily complicating the setup instructions. Or, you could just say that a user will have to set $TMPDIR if they want to process many files at once. Rather than claiming that it is down to permission issues (which if you use calibre's temporary folder is not a problem, or alternatively it is a problem that breaks calibre in general). Or claiming that it will default to "C:\Program Files\Calibre2\" for no comprehensible reason. There you go. Flexibility without any cost whatsoever.

06-28-2016, 05:04 PM	#14
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	Generic Python defaults to the current working directory of the program executing it as a last choice to create a temporary file. In Windows, for Calibre, that is "C:\Program Files\Calibre2\". Windows would throw a security/permissions error. Windows users are 85% of Calibre users. Simply reading the Original Post section about the ins-and-outs of environment variables and doing what they want to do eliminates that issue entirely.

06-28-2016, 05:53 PM	#15
eschwartz Ex-Helpdesk Junkie Posts: 19,421 Karma: 85400180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Um, the default is %TEMP% which is created by your user account. And Windows users are unlikely to have manually installed a ramdisk driver and redirected %TEMP% to it... but if they did, they would know how to deal with it. So there is no need to worry on behalf of Windows users that thousands of files will clobber their RAM. And you still haven't explained what is wrong with relying on calibre's temp dir (which must be writable for calibre to work in general, plugin or no plugin). ... But if you really, really want to impress your users by writing and enforcing complicated instructions, go right ahead. I have no beef in this, I am just commenting on what I regard as shaky logic.

Advert

Advert