Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 01-01-2016, 06:05 PM   #16
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 16,794
Karma: 26315296
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo,Aura H2O,Glo HD,Aura ONE,Clara HD,Forma;tolino epos
Apart from the difficulty of making this robust across platforms, I would feel uncomfortable with an application changing permissions of files. This is treading in murky waters and means changes to the system that aren't expected.

Also, I can guarantee that there is someone out there relying on this behaviour. Someone is putting read-only files in the auto add directory knowing that they won't be processed immediately. I have no idea why they are doing this, but I am sure that someone is.
davidfor is offline   Reply With Quote
Old 01-01-2016, 10:55 PM   #17
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,410
Karma: 10323934
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Absolutely, auto-adding is already pretty magical (particularly on windows, with its crazy file locking semantics) -- however, the solution to something being magical is not to add more magic to it, it is to remove as much magic as is possible. Read the code for the calibre auto-adder and you will find that > 75% of it is to deal with problems like antivirus locking files, or files that are partial because they are in the process of being transferred from a slow source, such as a network. That is all work calibre does to make the process as predictable as possible, given the inherent limitations of filesystems and OSes. Speculatively chmoding files before trying to add them absolutely will not increase that predictability.

And yes, as davidfor points out, file permissions can be a signal just as much as they can be noise. I have to agree with him that having an application simply override file permissions is disconcerting to say the least.
kovidgoyal is offline   Reply With Quote
Advert
Old 01-02-2016, 07:28 AM   #18
Phssthpok
Evangelist
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 432
Karma: 95229
Join Date: Nov 2014
Device: Horrible Kindle
OK. So bottom line, it's not something for everyone (which is why I originally suggested making it a prefs option which I and others like me could turn on).

At the moment I have a low-quality solution (the batch file I described earlier) which pops up an annoying console window when it runs; I could rewrite it in in a real language to prevent the annoyance, but presumably the "right" solution would be to rewrite it as a plugin which would start and stop when Calibre starts and stop, and wake up every so often to do my nasty magic tricks...

So, would this make sense? If so, I'll probably start bugging people on the plugins forum for advice on how to do it, such as plugin lifecycle issues and how to get at the auto-add directory...

Last edited by Phssthpok; 01-02-2016 at 07:32 AM.
Phssthpok is offline   Reply With Quote
Old 01-02-2016, 08:10 AM   #19
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,410
Karma: 10323934
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Seems awfully complicated. Why dont you just create a script that will extract the files from your archives with proper permissions instead, if you cannot find any program that does it for you. If you cannot figure out how to do it directly, then make it a two part operation, one that extracts the files to a temp dir, fixes their permissions, and then moves them over to the auto add dir.

Then just run your script instead of using whatever unarchiver program you are using currently.
kovidgoyal is offline   Reply With Quote
Old 01-02-2016, 08:12 AM   #20
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,410
Karma: 10323934
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Here's a one liner that will extract the contents of a zip file into the current directory.

calibre-debug -c "import sys, zipfile; zipfile.ZipFile(sys.argv[-1]).extractall()" file.zip
kovidgoyal is offline   Reply With Quote
Advert
Old 01-02-2016, 08:16 AM   #21
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,410
Karma: 10323934
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
And here's a one liner that will mark all files int he current directory as writable (assuming chmod works)

calibre-debug -c "import os, stat; from calibre import walk; [os.chmod(f, stat.S_IRWXU) for f in walk('.')]"
kovidgoyal is offline   Reply With Quote
Old 01-02-2016, 08:59 AM   #22
Phssthpok
Evangelist
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 432
Karma: 95229
Join Date: Nov 2014
Device: Horrible Kindle
Quote:
Originally Posted by kovidgoyal View Post
Seems awfully complicated. Why dont you just create a script that will extract the files from your archives with proper permissions instead, if you cannot find any program that does it for you. If you cannot figure out how to do it directly, then make it a two part operation, one that extracts the files to a temp dir, fixes their permissions, and then moves them over to the auto add dir.

Then just run your script instead of using whatever unarchiver program you are using currently.
So, write a special-purpose program which lists the contents of a zip file, lets me select the files I want (because I usually don't want everything that's in the archive, so I also need a GUI), and then extracts them into files with the right permissions. And what about other formats (rar, 7z, and what-all else that people use)? It gets messy, it turns into a non-trivial project.

Plan B, using a temp directory and all that as both you and betterred have suggested means that I have to (a) extract the files, and then (b) manually run something to fix the permissions. Or have (b) run (a), wait for process (a) to terminate, then continue with the moving etc.

In either case, the other issue is I have to remember to run my special program instead of just double-clicking on an archive. I might as well not bother using auto-add at all -- just extract into a temp directory and then drag&drop into Calibre.

At the moment I have a solution which involves no extra effort on my part -- I just extract the files into the auto-add directory. Every 30 minutes a batch file runs that checks if there are any files, fixes the permissions, creates and deletes a file, and goes back to sleep. Like I said, I could rewrite it easily enough and run it from startup or install it as a service -- but then it would be running even when Calibre ain't. So the plug-in solution seems better.

I'm imagining a plugin that kicks off a thread which will basically just sit in a loop which, with a bit of hand-waving, looks something like this:
Code:
while keepRunning:
    gotfiles = False
    for file in os.listdir(gprefs['auto_add_path']):
        os.chmod(file,stat.S_IWRITE|stat.S_IREAD)
        gotFiles = True
    if gotFiles:
        flagFile = os.path.join(gprefs['auto_add_path'],'foo.xxx')
        open(flagFile,'a').close()
        os.remove(flagFile)
    time.sleep(loopDelay)
The big question is, is this feasible? Am I on the right lines?

[edit: Just caught up with your second one-liner, which would simplify the above code somewhat. Thanks.]

Last edited by Phssthpok; 01-02-2016 at 09:38 AM.
Phssthpok is offline   Reply With Quote
Old 01-02-2016, 05:14 PM   #23
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 13,289
Karma: 10793754
Join Date: Mar 2012
Location: Sydney Australia
Device: none
@Phssthpok - FWIW - if you put a shortcut to calibre.exe your Windows Send To folder then you can send a file with any of the common attributes set - readonly, system, hidden, archive. Works from a compressed volume, not sure about an encrypted volume.

You can send multiple 'books', if you want to pick and choose what to send, the check boxes in Windows file manager are useful. For me at least this is a 'better' option than Autoadd.

BR
BetterRed is offline   Reply With Quote
Old 01-02-2016, 10:58 PM   #24
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,410
Karma: 10323934
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
You really need to embrace the power fo scripting. If you want a gui to extract the files, use one. Have your script launch a GUI program and wait for it to terminate. After it terminates have it fix permissions and copy the files. No extra effort on your part the and required script is five lines long.
kovidgoyal is offline   Reply With Quote
Old 01-02-2016, 11:14 PM   #25
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,278
Karma: 83106403
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Or simply add the magic directly to calibre via a source checkout.

As I understand the main objection is that it would run contrary to peoples' expectations.
So if you, personally, want calibre to speculatively chmod fils before trying to add them, then you, personally, can tell calibre to do so.


But I would feel nervous about adding it upstream -- what, more options, and this one to do something that is inadvisable to begin with?
There is such thing as feature creep, and features designed for one specific user are feature creep. Besides, the existence of an option would encourage people to use it!
eschwartz is offline   Reply With Quote
Old 01-03-2016, 05:14 AM   #26
Phssthpok
Evangelist
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 432
Karma: 95229
Join Date: Nov 2014
Device: Horrible Kindle
Quote:
Originally Posted by kovidgoyal View Post
You really need to embrace the power fo scripting. If you want a gui to extract the files, use one. Have your script launch a GUI program and wait for it to terminate. After it terminates have it fix permissions and copy the files. No extra effort on your part the and required script is five lines long.
Indeed, this was exactly the plan B I described earlier. Like I said, my only real objection to this is I have to remember to use the script to process the file rather than double-clicking and extracting. (And besides, I thought it might be fun to learn a bit about writing plugins.)

All I really want to know at this point is, are there any foreseeable problems with the plugin approach I'm suggesting?
Phssthpok is offline   Reply With Quote
Old 01-03-2016, 05:15 AM   #27
Phssthpok
Evangelist
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 432
Karma: 95229
Join Date: Nov 2014
Device: Horrible Kindle
Quote:
Originally Posted by eschwartz View Post
Or simply add the magic directly to calibre via a source checkout.
Aaargh! No thanks! Then I'd have to re-insert the change and rebuild every time a new version comes out...
Phssthpok is offline   Reply With Quote
Old 01-03-2016, 05:19 AM   #28
Phssthpok
Evangelist
Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.Phssthpok knows how to set a laser printer to stun.
 
Posts: 432
Karma: 95229
Join Date: Nov 2014
Device: Horrible Kindle
Quote:
Originally Posted by BetterRed View Post
@Phssthpok - FWIW - if you put a shortcut to calibre.exe your Windows Send To folder then you can send a file with any of the common attributes set - readonly, system, hidden, archive. Works from a compressed volume, not sure about an encrypted volume.

You can send multiple 'books', if you want to pick and choose what to send, the check boxes in Windows file manager are useful. For me at least this is a 'better' option than Autoadd.

BR
Good idea -- but "compressed volume" limits me to zipfiles AFAIK. At the moment I use 7-zip, which seems to understand every format known to man.
Phssthpok is offline   Reply With Quote
Old 01-03-2016, 05:27 AM   #29
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 34,410
Karma: 10323934
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by Phssthpok View Post
All I really want to know at this point is, are there any foreseeable problems with the plugin approach I'm suggesting?
Like I said its way more complicated than it needs to be and you are messing with an already fragile process. But sure, in theory you could do it in a plugin. There is no hard boundary between core calibre code and plugins, so you can do pretty much anything in a plugin.
kovidgoyal is offline   Reply With Quote
Old 01-03-2016, 06:14 AM   #30
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 13,289
Karma: 10793754
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by Phssthpok View Post
Good idea -- but "compressed volume" limits me to zipfiles AFAIK. At the moment I use 7-zip, which seems to understand every format known to man.
I don't where you got that idea, volume compression is a function of the file system. Folder/file compression is sometimes done by the operating system but more often by regular application programs.

My on-site backup disks are compressed volumes, on them I have zip, rar, 7z, sfx, iso, tar.gz, cbr, cbz, docx, epub etc etc. All of which I can uncompress with WinRAR (or Peazip or 7zip), to the same or another compressed volume or a regular non-compressed volume. And if I was minded I could install some plugins into my file manager and get it to do them.

BR
BetterRed is offline   Reply With Quote
Reply

Tags
auto add

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre Auto Add james14 Library Management 10 05-31-2014 02:49 AM
How-to Auto-add cover.jpg? buralex Library Management 6 03-05-2013 03:25 AM
Is this possible: Auto-add book/magazine then auto e-mail to device? runningwithbulls Library Management 5 09-10-2012 12:27 PM
Help with new feature: Auto-Add Folder kbullkar Library Management 12 03-30-2012 06:15 AM
need auto add help bulldogmo Calibre 10 02-15-2012 10:38 PM


All times are GMT -4. The time now is 03:46 PM.


MobileRead.com is a privately owned, operated and funded community.