Save to disk using hardlinks, possible?

[CKLJ001]

Hullo folks,

I was wondering whether anyone knows how to use, or whether not it's even possible to use the save to disk template formats to create a hardlinked directory.

I organise my books within Calibre using a custom hierarchical genre column as described in the Calibre manual. When I want to pass on some books from a specific genre, I simple do a save to disk. (Which I learned to do using templates by reading the wonderful plugboard sticky thread!)

This works fine. However, the size of my library is now quite large. (I'm a manual writer. And manuals, because of their pic heavy nature, can be large - 200-300mb pdf's are not uncommon.)

Doing a save-to-disk to keep a folder hierarchy that mirrors my genre hierarchy is now starting to get prohibitive.

Because I use linux, all my backups are done using snapshot hardlinks. So I was wondering if I could do something similar, that is a save-to-disk of my genre hierarchy, using hardlinks instead of copying as save-to-disk currently does.

I tried doing a number of searches online and searching this forum, but can't seem to find anything.

I was wondering if anyone here had any advice of how I might go about doing this or even where I might start looking.

[BetterRed]

@CKLJ001 - The short answer is no, i.e. there is no option for calibre to create hardlinks to the library copies of files when it Saves to Disk. And I doubt that's going to change any time soon.

When Books are Saved they may not be copies of what is in the library. When a book's metadata is edited the format files are not changed, only the library database is updated. When a book is saved to disk there's an option to update the embedded metadata in the saved format files with what's in the database.

AFAIK hardlinks only work on the same volume, and many people would save to a different volume, particularly if they use StD for backup purposes. And some of the file systems calibre supports don't support hardlinks.

BR

[orebmur]

Hardlinking in Linux is an option only if the files reside on the same file system.

There are tools for deduplicating directory contents by hardlinking, such as the excellent https://jak-linux.org/projects/hardlink/ and it is included at least in Debian. Chances are your ${DISTRO} has binaries too. I am using hardlink in a different scenario and for me it works as expected.

But i'm not really sure if externally messing with hardlinks might result in Calibre to unexpectedly wreak havoc with your precious files...

[CKLJ001]

Hullo BetterRed and orebmur,

Once I read your replies and started to think about what you said, I realised you're correct and hardlinks would be a disaster. Given their nature, they would easily wreck Calibre's internal storage system.

Which makes the cheap option out - so it's back to the expensive option then -- more hardware!

orebmur, thanks for the link to Jak. I do use Debian on my systems. And thank you both for your replies.

Cheers.

[BetterRed]

@CKLJ001 - Question is : why do you need to retain a copy of the books in a different directory structure on the same drive as the library?

Or to put it another way: what can you do with a 'content mirror' of your calibre library in a different folder structure that you can't do with calibre itself?

BR

[CKLJ001]

BetterRed,

That's simple - I thought it would be a be way for others to access specific book genres via the file system without Calibre.

I love how Calibre makes it easier to organise books, electronic or dead tree. Others dont care as much. They just want easy access to them. That's why I thought it would be easier for them to use the file system.

Only I use Calibre to organise the books, but I need to let others access them. My idea was to keep the Calibre library and it's mirror on a network drive and use permission controls to lock others out of Calibre's library and only give them access to the mirrored hardlinked directory.

If you feel like laughing now feel free. Because it wasn't until I read yours and orebmur's post I realised what a disaster I'd create for myself if I had actually done that.

I know what hard links are, heck all my systems are linux. And yet . . . It's amazing how stupid a person can get while trying to be to clever by half!!!

[BetterRed]

@CKLJ001 - Have a look at this plugin ===>> CalibreSpy

You might be able to use it give those who don't care so much about organising books, and who just want to access them, a nice simple GUI to do just that, without duplicating the library.

BR

[CKLJ001]

BetterRed,

Thanks for the heads up that does looks like what I'm looking for.

[DaltonST]

Originally Posted by CKLJ001

BetterRed,
Thanks for the heads up that does looks like what I'm looking for.

CKLJ001:


CalibreSpy is "read-only" as regards a Library's books etc. However, its own "Library-specific Settings" are obviously not "read-only", and are stored in a CalibreSpy-specific table, _calibrespy_settings, in each Library's metadata.db.

There is a potential conflict caused by multiple users viewing the identical Calibre Library simultaneously with CalibreSpy, and simultaneously wanting to change the CalibreSpy "Library-specific Settings", such as for Font, Font Size, Colors, and even which Custom Columns to show (and in what order from left to right). I am evaluating solutions for the above, but in the meantime, it would be a good idea to tell your people to not mess with the "Library-specific Settings" that you have chosen for them.

Also, each command file that executes CalibreSpy via CLI implicitly references a particular user's Calibre configuration folder that contains its plugins etc. For Windows, it is: C:/Users/USERNAME/AppData/Roaming/calibre/plugins. That directory has a subdirectory, /calibrespy_cli, that contains the icon image files dynamically loaded by CalibreSpy. Each command file CLI execution of CalibreSpy must have access to an appropriate /calibre/plugins/calibrespy_cli directory.

I originally wrote CalibreSpy just for myself to use on my tiny Windows 10 Tablet running Calibre64 that I use as my e-reading device. In other words, a single-user on a single PC having multiple Libraries.

It seems that CalibreSpy may have tapped into other unfilled needs.


Added:I am testing enhancements to track multiple users who are viewing the identical Library simultaneously. This includes a single user who runs CalibreSpy twice for the identical Library. Essentially, it will be "first come, first served" for access to the "Customize CalibreSpy" button in the lower left corner. The 2nd or 3d or whatever CalibreSpy simultaneous instances for that identical Library will have their "Customize CalibreSpy" button hidden. You cannot click what you cannot see. There are some other sequelae, such as not saving the 'state' of the matrix at exit for the 2nd or 3d or whatever users, since the 1st user may have changed some Library-specific preferences while the other users were active. I will also provide the means to easily "reset" the multi-user 'status' for a Library if ever needed due to a PC crash or network disconnect or whatever. It will likely be a trivial Python script executed via calibre-debug.



DaltonST

[CKLJ001]

Hi DaltonST

Thank you for the warning. I will certainly be very careful before I do anything, let alone before I allow anyone else anywhere near my library!