Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Closed Thread
 
Thread Tools Search this Thread
Old 04-01-2013, 09:31 AM   #16
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,689
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Quote:
Originally Posted by BetterRed View Post
Maybe because there's no need to change the Calibre code in order to achieve that objective.

This programmer simply layers folder symlinks onto Calibre's Library/Author/Title folder structure to create an entirely different view of the same data. No need to touch any Calibre code and no need to store the data twice.

Pretty sure that each of the desktop environments on which Calibre executes will support folder symlinks.

Why would I change someone else's code to do something for which it wasn't designed, especially when there's an alternative loosely couple solution sitting there waiting to be used. Real programmers are inherently lazy.

BR
Does symlinks even exist for Windows?
Remember, calibre runs on 3 OS platforms.


Also, how do you manage the USER actions? If I have books all over the filesystem, how does the USER, remember/know WHICH are known to calibre (since Calibre is no longer the Manager, but has been reduced to a simple auditor of user mis-actions)

If I had MY way, this could be resolved by having a system user/owner/process "Calibre" running. You pass files to Calibre, it take ownership, and files them in its user space (Look at BOINC for examples).
theducks is offline  
Old 04-01-2013, 11:38 AM   #17
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,458
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by theducks View Post
Does symlinks even exist for Windows?
Remember, calibre runs on 3 OS platforms.
Yep it's in Windows, been there since NT 5.1 if you were one of the 'in-crowd' but it wasn't exposed until Vista - MKLINK command. If you want the code for XP (NT5.2) go to bottom of this page - the source is there too http://schinagl.priv.at/nt/hardlinks...sion.html#3740

Quote:
Originally Posted by theducks View Post
Also, how do you manage the USER actions? If I have books all over the filesystem, how does the USER, remember/know WHICH are known to calibre (since Calibre is no longer the Manager, but has been reduced to a simple auditor of user mis-actions)
The books (formats, cover and opf) are stored in the Calibre folders in the normal fashion. I am NOT doing what some people do with metadata.db - ie have Calibre access the data via symbolic links.

Calibre maintains a standard set of Calibre Library folders with standard contents in the standard way, nothing is added to, changed in or deleted from those folders other than via bog standard Calibre. But the book folders are the target of the of the folder symlinks that are maintained and used by the software I have written.

For my purposes the Calibre Library/Author/Title folders are the database, I'm primarily interested in the data that's IN my 'books' rather than the data ABOUT my books.

BR

Last edited by BetterRed; 04-01-2013 at 12:25 PM.
BetterRed is offline  
Advert
Old 04-01-2013, 03:07 PM   #18
EowynCarter
Wizard
EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.
 
Posts: 4,332
Karma: 4000000
Join Date: Oct 2008
Location: Paris
Device: Cybooks; Sony PRS-T1
Quote:
Originally Posted by Dopedangel View Post
Fact is most people don't care if they did someone would have done it by now. That thread and argument was from a few years ago this argument has been going on for a few years even before that.
In all those years it was not important enough for anyone to branch off calibre as its opensource or submit a patch etc. I understand why you want it I would have been in the same boat as you 4-5 years ago if I hadn't bit the bullet and transferred all my ebooks to calibre. I haven't regretted that decision.
Well, I went the DIY road and made my own app. The attitude here discouraged me to try and improve calibre.

As a develpper, I do understand kovid's choices, i understand changing it is some work. But I still believe it have to be done.
EowynCarter is offline  
Old 04-01-2013, 03:22 PM   #19
Adoby
Handy Elephant
Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.
 
Adoby's Avatar
 
Posts: 1,736
Karma: 26785668
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus
What exactly is the wish?

Read-only files in a custom folder tree? Would it be enough to generate a fake directory tree, on command, using something similar to what save templates does, but with symlinks, like BetterRed described?

One obvious problem is the current optimization with lazy update of metadata in the ebooks, metadata is only written to books when the books leave calibre or when manually launched. To change that would slow down calibre a lot. So to generate the fake tree you would first have to write metadata to books, then create all the symlinks.
Adoby is offline  
Old 04-01-2013, 03:23 PM   #20
EowynCarter
Wizard
EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.
 
Posts: 4,332
Karma: 4000000
Join Date: Oct 2008
Location: Paris
Device: Cybooks; Sony PRS-T1
Quote:
Originally Posted by Adoby View Post
What exactly is the wish?

Read-only files in a custom folder tree? Would it be enough to generate a fake directory tree, on command, using something similar to what save templates does, but with symlinks, like BetterRed described?

One obvious problem is the current optimization with lazy update of metadata in the ebooks, metadata is only written to books when the books leave calibre or when manually launched. To change that would slow down calibre a lot. So to generate the fake tree you would first have to write metadata to books, then create all the symlinks.
My original need is -> use file explorer to copy past file from PC to e-reader.
I don't want to be forced to do that via any sotfware.
EowynCarter is offline  
Advert
Old 04-01-2013, 04:30 PM   #21
Adoby
Handy Elephant
Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.
 
Adoby's Avatar
 
Posts: 1,736
Karma: 26785668
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus
OK.

And you don't consider the file explorer to be software?

If calibre implemented a custom tree view, with support for drag and drop, but otherwise like send to device or save to disc, that would be enough?

Last edited by Adoby; 04-01-2013 at 04:50 PM.
Adoby is offline  
Old 04-01-2013, 06:45 PM   #22
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,458
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by Adoby View Post
What exactly is the wish?

Read-only files in a custom folder tree? Would it be enough to generate a fake directory tree, on command, using something similar to what save templates does, but with symlinks, like BetterRed described?
The alternate directory tree I create is real, nothing "fake" about it, gives direct access to the data just like any other directory entry in a POSIX compliant file system. A symlink is just a directory entry, they're a bit like file forks (hardlinks), but they work on directories and files, and deleting all symlinks wont delete the original as happens with file forks (hardlinks), and symlinks can span volumes, whereas file forks (hardlinks)are normally limited to a single volume.

Quote:
Originally Posted by Adoby View Post
One obvious problem is the current optimization with lazy update of metadata in the ebooks, metadata is only written to books when the books leave calibre or when manually launched. To change that would slow down calibre a lot.
The format files are not the only place where Calibre stores the metadata, in fact its the last place Calibre stores the metadata; unless you meticulously Polish or Modify books after the metadata is updated, but given Calibre does not provide a way to ensure that you do, then the metadata in the format files is, as you imply, inherently unreliable.

However, no programmer worthy of the title would consider going to a format file to get metadata if they are accessing Calibre Library folders. They would get it from the .opf files or from the metadata database. Not only is it more or less 'guaranteed' to be current, it is also complete, the metadata written to the format files is usually a subset of what's in the database/opf files. And if all you have is PDF's or TXT's you wont even get that.

BR

PS : As an experiment I just created a symlink clone of my Test Calibre Library which has 37 books in it. I "pointed" Calibre at the clone and Calibre worked just as it does on the original, same function, same speed... blah, blah, blah.

It took more time to tell Calibre about the clone than it did to create it... which was done with a single drag/drop in a 2 pane file manager using Link Shell extension.

Last edited by BetterRed; 04-02-2013 at 05:54 AM. Reason: fix inconstitent terms
BetterRed is offline  
Old 04-01-2013, 08:20 PM   #23
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,458
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
I should probably add that my software uses an 'out of the box' OS feature to monitor my major Calibre library folder trees for additions, changes and removals done via Calibre, or Sigil on an ePUB, or Word on an RTF etc etc. Its probably the same 'out of the box' OS feature that Calibre uses to monitor the folder nominated in the Add Books Automatic configuration.

Since the monitoring is done asynchronously my alternate views of my major libraries are kept in near real time synch with the Calibres view of those libraries. I measure time in intervals of a few seconds - in this instance "near real time" is typically < 200 msecs - plenty fast enough for me.

I keep other data... direct and via folder and file symlinks... in the same folders that contain the symlinks to Calibre's book folders. Including videos, radio podcasts, spreadsheets, presentations, emails, blog posts, recorded telephone conversations...

If this sounds all too complicated for you, then it probably is. Its also why I do not even want, let alone expect, Calibre or any other End User application to do it for me. Heck I don't even want the OS or file system to do it - we've been there too - WinFS

Agama & Lynx Lynx, if you happen to be reading this, you might remember my Supplementary Custom Column idea that we kicked around last year, the above describes its replacement.

The End

BR

Last edited by BetterRed; 04-01-2013 at 09:03 PM.
BetterRed is offline  
Old 04-02-2013, 01:50 AM   #24
alanHd
Addict
alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.alanHd ought to be getting tired of karma fortunes by now.
 
alanHd's Avatar
 
Posts: 374
Karma: 1408579
Join Date: Jul 2012
Location: UK
Device: Kindle Touch, Ipod Touch, Ipad Air
BetterRed, is your software available for all?
alanHd is offline  
Old 04-02-2013, 02:40 AM   #25
EowynCarter
Wizard
EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.EowynCarter ought to be getting tired of karma fortunes by now.
 
Posts: 4,332
Karma: 4000000
Join Date: Oct 2008
Location: Paris
Device: Cybooks; Sony PRS-T1
Quote:
Originally Posted by Adoby View Post
OK.

And you don't consider the file explorer to be software?

If calibre implemented a custom tree view, with support for drag and drop, but otherwise like send to device or save to disc, that would be enough?
Well, software built in the OS
So I don't have to bother starting anything just to copy one book.
I use both windows and linux, it's nice to be able to acces my books in both OS without having to to some serious mess.

I have multiple versions of a book (even for minor edits i like to keep the original, just in case.) That calibre won't deal with either.
EowynCarter is offline  
Old 04-02-2013, 05:46 AM   #26
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,458
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by alanHd View Post
BetterRed, is your software available for all?
No mate, its far too subject to my whims, wants and idiosyncrasies. It'll live out its days in my skunkworks.

PM me if you want ideas on wrapping Calibre up your own skunk skin

BR

Last edited by BetterRed; 04-02-2013 at 06:01 AM.
BetterRed is offline  
Old 04-02-2013, 06:28 AM   #27
Adoby
Handy Elephant
Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.
 
Adoby's Avatar
 
Posts: 1,736
Karma: 26785668
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus
One very easy way to do it could be to automatically run a program when you exit calibre. Launch calibre from a script. The program then updates the alternative folder tree with new/updated links. I do something similar to sync a backup copy of my calibre library using rsync. Takes seconds usually.

There could also be a separate "resource" folder tree with files that calibre is not suited to manage. If names of folders/files match metadata in calibre these resource folders/files are also linked in the alternative folder tree.

By linking less space is used and no data has to be copied, so it should be much faster than an actual sync.

To make sure the calibre library is not corrupted the linked files would have to be read only.
Adoby is offline  
Old 04-02-2013, 07:18 AM   #28
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,458
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by Adoby View Post
To make sure the calibre library is not corrupted the linked files would have to be read only.
If you change the attributes of a file in a symlink folder, the attribute is also changed in the target folder - this is true on Windows, on Linux, on OS/X. Why - because the file is the identical inode in both folders.

A consequence of making the files in a book folder read only would be that Calibre, Format Editors such as Sigil, Word, Notepad++ etc and image editors like Gimp, Irfanview etc will not be able to update the opf, the format files or the covers due to permission faults

I guess one could run a script that set the read only attribute off on all the files in all the libraries before starting calibre - as I said one could, but not this one, thanks.

It could be done with ACLs that set different permissions for different user accounts, then one could spend ones time logging in and out of different user accounts to protect oneself from oneself - same as last time, one could, but not this one, thanks.

BR
BetterRed is offline  
Old 04-02-2013, 08:12 AM   #29
Adoby
Handy Elephant
Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.Adoby ought to be getting tired of karma fortunes by now.
 
Adoby's Avatar
 
Posts: 1,736
Karma: 26785668
Join Date: Dec 2009
Location: Southern Sweden, far out in the quiet woods
Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus
Yes, not easy to do elegant and bulletproof. In Linux this could be handled by using sudo to run calibre and other programs as another user, and increase password timeout. Possible, but still clunky.
Adoby is offline  
Old 04-02-2013, 09:38 AM   #30
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,458
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Its simple Adoby,

If you don't trust yourself and/or you're fault intolerant then don't do what I do.

And nor should you question what I do... thank you very much!

BR
BetterRed is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Folder Structure on Device laos369 Sony Reader Dev Corner 6 05-25-2012 11:23 PM
The ability to edit epub file from other software droople Sigil 9 09-25-2010 10:07 AM
Folder structure an the reader thomasmorus Calibre 2 05-23-2010 07:51 PM
Folder structure doubt Oletros iRex 3 01-01-2009 07:00 AM
SD Folder structure ctitanic Amazon Kindle 1 12-30-2007 12:48 PM


All times are GMT -4. The time now is 08:28 PM.


MobileRead.com is a privately owned, operated and funded community.