Supplementary material

darq · 08-02-2012, 06:03 PM

How do I handle supplementary materials in calibre?

Currently I use one folder per book where I keep extra files that belong to the book: source code, programs, data, pictures, metadata, coverimages, annotations etc. A few thousand files in total.

What is the workflow to keep this data together?

theducks · 08-02-2012, 09:19 PM

Quote:

Originally Posted by darq

How do I handle supplementary materials in calibre?

Currently I use one folder per book where I keep extra files that belong to the book: source code, programs, data, pictures, metadata, coverimages, annotations etc. A few thousand files in total.

What is the workflow to keep this data together?

I don't believe that Calibre complains about Extra files inside a HTML zip .
Now they (the files) will NOT follow, if converting AND HTML is one of the few formats that does NOT extract the format when adding to the Library.

darq · 08-03-2012, 12:37 PM

Hi theducks,

I see, so I for each folder I'd have to make a html file and put the book and the extra files into the right folder structure, zip them, and then import everything together?

itimpi · 08-03-2012, 02:19 PM

I think if you put all the supplementary material into a RAR ie and added that then Newsbin would treat it as though it was a book format and so it would be kept with the other formats.

darq · 08-03-2012, 05:40 PM

itimpi,

Just read your other reply on this thread
https://www.mobileread.com/forums/sho...d.php?t=186494
now I see, if you have multiple files in the archive it leaves them alone in one archive, but you can't have the uncompressed files around.

So its impossible to keep the files together in the file system, bad news for me, but thanks for your explanation. I have too many GB of files and metadata (e.g. txt files for OCRd books, and I need to keep them accessible in some way to other programs that I've written. Seems what I am trying to do is not possible at the moment, so I'll stay with my folders for the time being.

Agama · 08-04-2012, 03:07 AM

If you're on a Windows system then you can create a shortcut to the supplementary folder for a book within that book's calibre db folder. Select a book in your library and use "Open containing folder". Create a shortcut to the supplementary folder.

This shortcut can then be accessed again by using "Open containing folder".

One point to watch out for is that such shortcuts are not kept when copying the book to a different library. There may well be other caveats.

(This is of course 'bad practise' in the sense that it involves messing about within calibre's db folder, but it works for me.)

BetterRed · 08-05-2012, 02:33 AM

This issue comes up a lot

If Calibre had the ability to define the data type of a Custom Column as a Uniform Resource Identifier then one could could use it to contain the path to Supplementary Material on the local machine, on an FTP Server etc.

The base functionality is already in Calibre, in the ID values and Click to Open items in the Book Details

Sort of surprised Kovid doesn't have it, but maybe its not as simple as all that.

BR

darq · 08-07-2012, 05:28 PM

I can explain a bit more what I am trying to do, maybe there is a better way:

I have an extensive book and article collection. Many items are sorted hierarchically, one folder per item. Some of the folders contain a lot of metadata in a database file, a text file with the abstract, cover images. An OCR version of the book if the original contains no text.
Sometimes I annotate a pdf, in this case I keep the original version as well.

A lot of the data is generated by running a script that traverses the folders. If I move the files apart or pack them into a .rar, none of these scripts will be able to continue to work. If calibre needs to rename the folders, I can (barely) live with that, but I'd still have to keep the files together in their respective folders.

Another reason for me to keep the files together is that sometimes they are parts of the same book. Some scans are huge and split up into many individual files, I don't want one entry per file, and I don't want to store them in different folders and lose the association between the files.

I am sure that's a common issue, am I approaching this the right way?

BetterRed · 08-07-2012, 07:37 PM

Quote:

Originally Posted by darq

I am sure that's a common issue, am I approaching this the right way?

@darq - why are you using Calibre?

Perhaps you can get the features of Calibre you're after from the Command Line. My understanding is that some functions, eg Conversion, can be accessed from the command line without using Calibre's "Library Management" features that require the Library/Author/Book directory structure. http://manual.calibre-ebook.com/cli/cli-index.html

Otherwise, and assuming you do not have the Real Book in a ZIP format.

If you put a dummy.txt file and a shortcut (.lnk) file to the folder you want to access into a zip file and drop it into the Book (Edit Metadata drop into the formats box top right) then Calibre will rename it to "title - author.zip", and take care of the file. Make sure that .lnk file goes in the zip file and not the content of the folder its pointing at, depending on which archiver you use, that can be a little tricky.

Now if you open the book folder (Book Details - Click to open) and open the zip with your archiver (DONT UNZIP IT) then you should be able to click on the .lnk file inside the zip to open the referenced folder in another Windows Explorer window.

I have attached an example zip that should work on your system - download it, open it with your archiver, but dont unzip it (I use WinRAR),then double click Program Files.lnk in the zippy.zip - and I think you should see your Program Files folder open in a new Windows Explorer folder (assuming its on C

BR

theducks · 08-07-2012, 07:48 PM

He could also make a simple HTML file with a URL to each file

Code:

file:///c:/path/to/the%20file.txt

darq · 08-07-2012, 08:51 PM

@BetterRed: I've been using Calibre for some time for a few novels and I would like to use it for my technical library.
For that I currently use Zotero, but I have the feeling that it is not a good solution for books (and I am not a big fan of javascript).
Calibre is really easy to extend, and it has many nice plugins.
I like the way calibre shows the coverpages, that is extremely helpful because I can look for a book series by the color of their title, generally the interface of calibre is much nicer.

Thanks for the zip file, works!

BetterRed · 08-08-2012, 02:41 AM

You like much the same things about Calibre I like

Pleased the zippy.zip worked for you, wasn't 100% sure it would, thought it might run into Windows security minefields. Will that technique be adequate until something better comes along ?

BR

Agama · 08-09-2012, 07:25 AM

Quote:

Originally Posted by darq

Calibre is really easy to extend, and it has many nice plugins.

This has given me an idea. I have started on a plugin to deal with links to supplementary folders.

BetterRed · 08-09-2012, 08:23 AM

Quote:

Originally Posted by Agama

This has given me an idea. I have started on a plugin to deal with links to supplementary folders.

Good on yer cobber, happy to test when its ready

darq · 08-09-2012, 09:02 PM

looks like a powerful way to circumvent the directory limitation of Calibre.
Keep the directory structure, generate a zip or html with links and then import.
This could be done by an import script. The original files never get imported. Of course, this would break many plugins that won't find the extra files. I need to study this a bit

08-02-2012, 06:03 PM	#1
darq Junior Member Posts: 6 Karma: 20 Join Date: Aug 2012 Device: none	Supplementary material How do I handle supplementary materials in calibre? Currently I use one folder per book where I keep extra files that belong to the book: source code, programs, data, pictures, metadata, coverimages, annotations etc. A few thousand files in total. What is the workflow to keep this data together?

08-07-2012, 07:48 PM	#10
theducks Well trained by Cats Posts: 29,812 Karma: 54830978 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	He could also make a simple HTML file with a URL to each file Code: file:///c:/path/to/the%20file.txt

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Privacy of Material?	bayside94	Barnes & Noble NOOK	4	12-26-2010 10:54 PM
copyrighted material	GaylaM	General Discussions	53	09-28-2010 12:58 PM
Reference Material	Kumabjorn	General Discussions	11	07-12-2010 09:26 AM
Video material of A81	kathy6699	Alternative Devices	8	02-23-2010 04:19 AM

08-03-2012, 12:37 PM	#3
darq Junior Member Posts: 6 Karma: 20 Join Date: Aug 2012 Device: none	Hi theducks, I see, so I for each folder I'd have to make a html file and put the book and the extra files into the right folder structure, zip them, and then import everything together?

08-03-2012, 02:19 PM	#4
itimpi Wizard Posts: 4,552 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	I think if you put all the supplementary material into a RAR ie and added that then Newsbin would treat it as though it was a book format and so it would be kept with the other formats.

08-03-2012, 05:40 PM	#5
darq Junior Member Posts: 6 Karma: 20 Join Date: Aug 2012 Device: none	itimpi, Just read your other reply on this thread https://www.mobileread.com/forums/sho...d.php?t=186494 now I see, if you have multiple files in the archive it leaves them alone in one archive, but you can't have the uncompressed files around. So its impossible to keep the files together in the file system, bad news for me, but thanks for your explanation. I have too many GB of files and metadata (e.g. txt files for OCRd books, and I need to keep them accessible in some way to other programs that I've written. Seems what I am trying to do is not possible at the moment, so I'll stay with my folders for the time being.

08-04-2012, 03:07 AM	#6
Agama Guru Posts: 776 Karma: 2751519 Join Date: Jul 2010 Location: UK Device: PW2, Nexus7	If you're on a Windows system then you can create a shortcut to the supplementary folder for a book within that book's calibre db folder. Select a book in your library and use "Open containing folder". Create a shortcut to the supplementary folder. This shortcut can then be accessed again by using "Open containing folder". One point to watch out for is that such shortcuts are not kept when copying the book to a different library. There may well be other caveats. (This is of course 'bad practise' in the sense that it involves messing about within calibre's db folder, but it works for me.)

08-05-2012, 02:33 AM	#7
BetterRed null operator (he/him) Posts: 20,583 Karma: 26954694 Join Date: Mar 2012 Location: Sydney Australia Device: none	This issue comes up a lot If Calibre had the ability to define the data type of a Custom Column as a Uniform Resource Identifier then one could could use it to contain the path to Supplementary Material on the local machine, on an FTP Server etc. The base functionality is already in Calibre, in the ID values and Click to Open items in the Book Details Sort of surprised Kovid doesn't have it, but maybe its not as simple as all that. BR

08-07-2012, 05:28 PM	#8
darq Junior Member Posts: 6 Karma: 20 Join Date: Aug 2012 Device: none	I can explain a bit more what I am trying to do, maybe there is a better way: I have an extensive book and article collection. Many items are sorted hierarchically, one folder per item. Some of the folders contain a lot of metadata in a database file, a text file with the abstract, cover images. An OCR version of the book if the original contains no text. Sometimes I annotate a pdf, in this case I keep the original version as well. A lot of the data is generated by running a script that traverses the folders. If I move the files apart or pack them into a .rar, none of these scripts will be able to continue to work. If calibre needs to rename the folders, I can (barely) live with that, but I'd still have to keep the files together in their respective folders. Another reason for me to keep the files together is that sometimes they are parts of the same book. Some scans are huge and split up into many individual files, I don't want one entry per file, and I don't want to store them in different folders and lose the association between the files. I am sure that's a common issue, am I approaching this the right way?

08-07-2012, 08:51 PM	#11
darq Junior Member Posts: 6 Karma: 20 Join Date: Aug 2012 Device: none	@BetterRed: I've been using Calibre for some time for a few novels and I would like to use it for my technical library. For that I currently use Zotero, but I have the feeling that it is not a good solution for books (and I am not a big fan of javascript). Calibre is really easy to extend, and it has many nice plugins. I like the way calibre shows the coverpages, that is extremely helpful because I can look for a book series by the color of their title, generally the interface of calibre is much nicer. Thanks for the zip file, works!

08-08-2012, 02:41 AM	#12
BetterRed null operator (he/him) Posts: 20,583 Karma: 26954694 Join Date: Mar 2012 Location: Sydney Australia Device: none	You like much the same things about Calibre I like Pleased the zippy.zip worked for you, wasn't 100% sure it would, thought it might run into Windows security minefields. Will that technique be adequate until something better comes along ? BR

08-09-2012, 09:02 PM	#15
darq Junior Member Posts: 6 Karma: 20 Join Date: Aug 2012 Device: none	looks like a powerful way to circumvent the directory limitation of Calibre. Keep the directory structure, generate a zip or html with links and then import. This could be done by an import script. The original files never get imported. Of course, this would break many plugins that won't find the extra files. I need to study this a bit

Advert

Advert