Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 02-06-2012, 06:33 PM   #1
Egnaled Knarf
Junior Member
Egnaled Knarf began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jan 2010
Device: Phone running whatever
Lightbulb PATCH: ebook-convert: obviate need for write permission in source directory

I'm using part of the Calibre cli tools to implement a rudimentary library management system capable of converting documents on the fly to several popular formats.
Spoiler:
I can not use all of Calibre for this project because of the abysmal performance when confronted with large collections (> 90000 documents in this case).
While implementing these on-the-fly conversion functions I noticed ebook-convert (and with that Calibre proper as well as far as I can tell) needs write permission in the document source directory for at least conversions from TXT to other formats. Given that it only uses this permission to create a temporary file which is deleted a few moments later and given the risks inherent in allowing write access to the document source directory may I suggest applying this short patch to obviate this requirement? The patch does away with the versioned temporary file in the source directory, replacing it with a proper temporary file as create by Calibre's own TemporaryFile method. This is both cleaner as well as more readable, and works in situations where the user running ebook-convert (in my case www-data) does not have write permission to the library source.

The patch to fix this problem when converting TXT-related documents to other formats is attached to this message. It is of course possible this problem exists elsewhere in Calibre so more patching might be called for. The patch applies to Calibre 0.8.38 (and with some fuzz to earlier versions as well).
Egnaled Knarf is offline   Reply With Quote
Old 02-07-2012, 01:17 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,436
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Your patch will break txt files that reference images using relative paths (with markdown, for instance).
kovidgoyal is offline   Reply With Quote
 
Advertisement
Old 02-09-2012, 07:34 PM   #3
Egnaled Knarf
Junior Member
Egnaled Knarf began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jan 2010
Device: Phone running whatever
Quote:
Originally Posted by kovidgoyal View Post
Your patch will break txt files that reference images using relative paths (with markdown, for instance).
True, I had not thought of that - probably because I don't have any markdown-formatted texts. This breakage could be resolved by symlinking the contents of the source directory to the temporary directory. I'll go hunting for a suitably complex test case to add this to the patch.
Egnaled Knarf is offline   Reply With Quote
Old 02-09-2012, 11:38 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,436
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Symlinking isn't cross platform, and there's no way you can guarantee that the linked to resources are in the same directory as the txt file.
kovidgoyal is offline   Reply With Quote
Old 02-10-2012, 01:32 PM   #5
Egnaled Knarf
Junior Member
Egnaled Knarf began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jan 2010
Device: Phone running whatever
Quote:
Originally Posted by kovidgoyal View Post
Symlinking isn't cross platform, and there's no way you can guarantee that the linked to resources are in the same directory as the txt file.
Cross-platform - well, it runs on Linux and *BSD and such so it is cross-platform in the Microsoft-sense but I'm starting to see what you're getting at.
Given the (to me at least...) obvious advantages of keeping write access to the library source limited to the absolute minimum I can live with these limitations but the current patch is clearly not ready for general release. Since it does fit my needs I'll just keep patching...
Egnaled Knarf is offline   Reply With Quote
Old 02-10-2012, 11:43 PM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,436
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Why dont you simply copy the source file out of the library to a temp directory and then convert it and copy the result back.
kovidgoyal is offline   Reply With Quote
Old 02-17-2012, 06:27 AM   #7
Egnaled Knarf
Junior Member
Egnaled Knarf began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jan 2010
Device: Phone running whatever
Quote:
Originally Posted by kovidgoyal View Post
Why dont you simply copy the source file out of the library to a temp directory and then convert it and copy the result back.
While that would have most of the benefits of my method, it has all of the drawbacks with regard to unresolved relative links and it adds an additional file copy.
Egnaled Knarf is offline   Reply With Quote
Old 02-17-2012, 07:02 AM   #8
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,436
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
There are no relative links for documents in the calibre library, unless the user has been manually putting files in there. And since you only care about unix, create a hard link, that will mean no extra copy unless hte temp dir is mounted ona different partition.
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Write Permission wtw1936 Library Management 2 09-21-2011 01:40 AM
Open Source Font Directory for 1.9.5 Toastedpine Kobo Reader 3 06-29-2011 03:16 PM
ebook-convert: access current working directory inside recipe jens32 Recipes 3 06-03-2011 11:57 AM
Do you need permission to write about public places? ficbot Writers' Corner 14 12-28-2009 10:58 AM
Convert LIT bug & patch llasram Workshop 4 12-30-2007 05:05 PM


All times are GMT -4. The time now is 09:00 PM.


MobileRead.com is a privately owned, operated and funded community.