Hardlinks in 0.9.4

rick_s · 10-31-2012, 09:44 AM

Hi,

as a frequent user of hardlinks I think this is a very useful item. I was wondering though.

A few months ago I made the suggestion (on a ticket) to use hardlinks for importing/copying books into calibre if they are on the same partition.

I was told this isn't possible because of the database format of calibre or something with streams. I don't recall exactly and didn't understand it then anyway.

So, if there can be hardlink copies within the calibre library like in 0.9.4 why not from outside for the import? Am I seeing something wrong?

I still think this would be very useful. First for the speed-up reason. And then you could keep old library structures without wasting space. And this would put an end to the recurring discussion and wishes of changing the calibre library format to accommodate preferences to a different storage way without compromising any of the design decisions for calibre.

kovidgoyal · 10-31-2012, 09:58 AM

Keeping private copies of ebook files is one of the design decisions of calibre, for reasons I have spelled out many times in the past. Hardlinks are used in internal file moves in 0.9.4, the hard link typically exists for less than a few milliseconds. And even that simple use of hardlinks has led to no less than three bug reports from people using calibre on various broken filesystems in the last couple of days alone.

Not to mention all the use cases that using hardlinks will break:

1) Hardlinks may or may not be applicable, depending on whether the files in question oare on the same volume. Queue the bug reports about "hard link sometimes created sometimes not"

2) People that like to maintain out of library files as a backup to calibre will be very annoyed if calibre making changes to a file will also change their out of library files. Queue the bug reports about how calibre destroyed people's backup files.

3) If people make changes to out of library files, there will be unexpected changes to in library files, not to mention file locking issues on windows. Queue the bug reports about how calibre refuses to change metadata saying a file is in use when the file obviously isn't.

All in all, absolutely not

10-31-2012, 09:44 AM	#1
rick_s Junior Member Posts: 1 Karma: 10 Join Date: Mar 2011 Device: none	Hardlinks in 0.9.4 Hi, as a frequent user of hardlinks I think this is a very useful item. I was wondering though. A few months ago I made the suggestion (on a ticket) to use hardlinks for importing/copying books into calibre if they are on the same partition. I was told this isn't possible because of the database format of calibre or something with streams. I don't recall exactly and didn't understand it then anyway. So, if there can be hardlink copies within the calibre library like in 0.9.4 why not from outside for the import? Am I seeing something wrong? I still think this would be very useful. First for the speed-up reason. And then you could keep old library structures without wasting space. And this would put an end to the recurring discussion and wishes of changing the calibre library format to accommodate preferences to a different storage way without compromising any of the design decisions for calibre.

10-31-2012, 09:58 AM	#2
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Keeping private copies of ebook files is one of the design decisions of calibre, for reasons I have spelled out many times in the past. Hardlinks are used in internal file moves in 0.9.4, the hard link typically exists for less than a few milliseconds. And even that simple use of hardlinks has led to no less than three bug reports from people using calibre on various broken filesystems in the last couple of days alone. Not to mention all the use cases that using hardlinks will break: 1) Hardlinks may or may not be applicable, depending on whether the files in question oare on the same volume. Queue the bug reports about "hard link sometimes created sometimes not" 2) People that like to maintain out of library files as a backup to calibre will be very annoyed if calibre making changes to a file will also change their out of library files. Queue the bug reports about how calibre destroyed people's backup files. 3) If people make changes to out of library files, there will be unexpected changes to in library files, not to mention file locking issues on windows. Queue the bug reports about how calibre refuses to change metadata saying a file is in use when the file obviously isn't. All in all, absolutely not Last edited by kovidgoyal; 10-31-2012 at 10:02 AM.

Advert