Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Development

Notices

Reply
 
Thread Tools Search this Thread
Old 05-17-2011, 09:00 AM   #1
hakan42
Zealot
hakan42 is on a distinguished road
 
hakan42's Avatar
 
Posts: 136
Karma: 60
Join Date: Jul 2009
Location: Munich, Germany
Device: Nook Classic rooted; Galaxy S IV with Aldiko, other older devices
Pluginize file naming strategy ?

Hi,

before anyone points me to FAQs, yes, I have read PLEASE READ BEFORE POSTING - The purpose of this subforum and Want to change the folder structure of the Calibre library? and am aware of most of the arguments pro / contra.

My special use case ( content from different shops / sources aggregated and available on one server because *I* bought them for *myself*, but should be stored in separate locations for legal reasons ) would be best served by manipulating the storage strategy, tough.

So, my question is: If I would put in the effort to pluginize the directory naming strategy (much like the way metadata sources work now), and later on provided additional plugins that serve my use case, would that work be considered for inclusion in the main source tree or would I have to maintain my private tree for the foreseeable future, including the effort of building windows binary installation packages?


Regards,
Hakan
hakan42 is offline   Reply With Quote
Old 05-17-2011, 10:05 AM   #2
user_none
Sigil & calibre developer
user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.user_none ought to be getting tired of karma fortunes by now.
 
user_none's Avatar
 
Posts: 2,488
Karma: 1063785
Join Date: Jan 2009
Location: Florida, USA
Device: Nook STR
If you do the work, you maintain the work and you can show that the changes are an improvement then it will be considered for inclusion.

However, you are going to be hard pressed to show that changing the library structure to something else is an improvement. Your also going to be hard pressed to show that pluginizing this is an improvement. In most cases not special cases.

I would look over the existing threads about the library structure pull in all reasons for and against then start a non-development section thread to discuss. Go over each for and against and make a clear argument as to why your changes re an improvement. Then take in the responses.and re work your proposal.

Again the majority of users and developers are happy with the current implementation. There is little, not 0, chance of getting this accepted.
user_none is offline   Reply With Quote
Advert
Old 05-17-2011, 10:07 AM   #3
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by hakan42 View Post
So, my question is: If I would put in the effort to pluginize the directory naming strategy (much like the way metadata sources work now), and later on provided additional plugins that serve my use case, would that work be considered for inclusion in the main source tree
You might want to read this post and the related thread:
https://www.mobileread.com/forums/sho...94&postcount=8
Starson17 is offline   Reply With Quote
Old 05-17-2011, 10:43 AM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,775
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I have no objections in principle to having the directory naming configurable. However, various caveats apply:

1) It would have to be done in a way that does not break metadata backup/db restore/the cover storage APIs (probably others as well)
2) The default semantics must remain as they are
3) This kind of change will require *a lot* of testing, on multiple platforms, as it has the potential for data loss.

I'm somewhat puzzled by your use case, however. If you are serving up books from a server, what earthly difference does it make where they are stored?
kovidgoyal is offline   Reply With Quote
Old 05-17-2011, 11:15 AM   #5
hakan42
Zealot
hakan42 is on a distinguished road
 
hakan42's Avatar
 
Posts: 136
Karma: 60
Join Date: Jul 2009
Location: Munich, Germany
Device: Nook Classic rooted; Galaxy S IV with Aldiko, other older devices
Quote:
Originally Posted by user_none View Post
If you do the work, you maintain the work and you can show that the changes are an improvement then it will be considered for inclusion.
Yes, I am willing to do the initial work and the maintenance. Actually, I am locally modifying database2.py and config.py since ages (since August 2009 to be precise) now to keep my estimated 100 lines of changes working.

Quote:
Originally Posted by user_none View Post
I would look over the existing threads about the library structure pull in all reasons for and against then start a non-development section thread to discuss. Go over each for and against and make a clear argument as to why your changes re an improvement. Then take in the responses.and re work your proposal.
Actually, I opened this thread intentionally here in the development area because I am willing to do the work and don't want to get swamped by the usual threads of the form "I want directories to be named 'LN FN' instead of 'FN LN'". This sums up about 80% of the directory naming threads, the rest being "I want to have the genre in the directory name" type of threads. Both get answered by the (very valid) argument of "It's not your directory tree, it belongs to calibre".

The code I am willing to create and maintain would keep all those guys off your back. You could just tell them that if they really really want to modify their storage, they can create / use a plugin, but the main calibre developers only supports the "default" configuration (which would work identically to the current code).

Quote:
Originally Posted by user_none View Post
Again the majority of users and developers are happy with the current implementation. There is little, not 0, chance of getting this accepted.
After some time, the majority accepts the argument that the storage tree of calibre is not theirs to muck around in it (which is a correct argument for most use cases). My use case(s), and the points where they differ from the mainstream calibre operation, I'll enumerate in my answer to kovid's posting.

Regards,
Hakan
hakan42 is offline   Reply With Quote
Advert
Old 05-17-2011, 01:47 PM   #6
hakan42
Zealot
hakan42 is on a distinguished road
 
hakan42's Avatar
 
Posts: 136
Karma: 60
Join Date: Jul 2009
Location: Munich, Germany
Device: Nook Classic rooted; Galaxy S IV with Aldiko, other older devices
Quote:
Originally Posted by kovidgoyal View Post
I have no objections in principle to having the directory naming configurable. However, various caveats apply:

1) It would have to be done in a way that does not break metadata backup/db restore/the cover storage APIs (probably others as well)
2) The default semantics must remain as they are
3) This kind of change will require *a lot* of testing, on multiple platforms, as it has the potential for data loss.

I'm somewhat puzzled by your use case, however. If you are serving up books from a server, what earthly difference does it make where they are stored?
I can do testing on a Windows desktop, X on Gentoo Linux and a Macbook. Hopefully, this will cover the majority of the installations out there.

And my intention is to keep the default semantics if no plugin to modify strategy is configured.

And for the "where they are stored" part, one example are the ebooks from the http://www.blacklibrary.com. If you read their license rules, they demand that you provably remove all their files from your system if they decide to revoke their license to you. For some time, I actually thought about putting all the stuff I buy from them into a completely separate calibre instance, but then again, I'm not sure that their license would even allow me to store the epubs in calibre, anyway In that case, the proof that I removed everything would be done by showing their lawyer the directory where their stuff is, telling them that nothing of their intellectual property is in my other directories and kindly asking them to refrain from rifling through all my other stuff...


Regards,
Hakan
hakan42 is offline   Reply With Quote
Old 05-17-2011, 03:03 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,775
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I seriously doubt that a lawyer would consider being shown an empty directory as proof of anything.
kovidgoyal is offline   Reply With Quote
Old 10-06-2012, 06:08 AM   #8
hakan42
Zealot
hakan42 is on a distinguished road
 
hakan42's Avatar
 
Posts: 136
Karma: 60
Join Date: Jul 2009
Location: Munich, Germany
Device: Nook Classic rooted; Galaxy S IV with Aldiko, other older devices
Quote:
Originally Posted by kovidgoyal View Post
I have no objections in principle to having the directory naming configurable. However, various caveats apply:

1) It would have to be done in a way that does not break metadata backup/db restore/the cover storage APIs (probably others as well)
2) The default semantics must remain as they are
3) This kind of change will require *a lot* of testing, on multiple platforms, as it has the potential for data loss.

I'm somewhat puzzled by your use case, however. If you are serving up books from a server, what earthly difference does it make where they are stored?
Hi,

I re-applied my changes to the currently distributed 0.9.1 tag and sent you a merge proposal.

If accepted, this could be used to create plugins to put the source (see my main use case) or series information into the path of the books without having to modify the core of calibre anymore.

Also, if accepted, I would continue to work on this to allow graphical configuration of my plugins, which at this moment have a bit of hard-coded information in them (e.g. name of the column to use for "source" and so on).

Regards,
Hakan
hakan42 is offline   Reply With Quote
Old 10-22-2012, 02:58 AM   #9
Agama
Guru
Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.Agama ought to be getting tired of karma fortunes by now.
 
Agama's Avatar
 
Posts: 776
Karma: 2751519
Join Date: Jul 2010
Location: UK
Device: PW2, Nexus7
Quote:
Originally Posted by hakan42 View Post
My special use case ( content from different shops / sources aggregated and available on one server because *I* bought them for *myself*, but should be stored in separate locations for legal reasons ) would be best served by manipulating the storage strategy, tough.
Can't you achieve this by simply using multiple libraries or do your books all need to show up in the same library? I use 5 libraries with calibre, which gives me 5 separate folder locations on my hard disk.
Agama is offline   Reply With Quote
Old 10-22-2012, 09:27 AM   #10
hakan42
Zealot
hakan42 is on a distinguished road
 
hakan42's Avatar
 
Posts: 136
Karma: 60
Join Date: Jul 2009
Location: Munich, Germany
Device: Nook Classic rooted; Galaxy S IV with Aldiko, other older devices
Sure, I could separate with multiple libraries. Buuuuuuuut...

On one hand, I have about a dozen book sources which I want to keep separated, so a-library-for-each gets unwieldy pretty quick.

On the other hand, calibre2opds can't merge multiple libraries into one opds directory so configuration on my reader devices gets complicated.

And, on the gripping hand, the OCD afflicted software engineer in me wanted a configurable solution which would also consider Series information and whatever the user might consider necessary . So, main reason is my OCD
hakan42 is offline   Reply With Quote
Old 10-22-2012, 10:40 AM   #11
itimpi
Wizard
itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.itimpi ought to be getting tired of karma fortunes by now.
 
Posts: 4,552
Karma: 950151
Join Date: Nov 2008
Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)
Quote:
Originally Posted by hakan42 View Post
On the other hand, calibre2opds can't merge multiple libraries into one opds directory so configuration on my reader devices gets complicated.
I agree that there is no capability in calibre2opds to generate a single catalog covering multiple libraries (although it might be possible to develop this if a good enough case can be made).

what there already is is a capability in Calibre2Opds to link catalogs. You can have entries in the top level of a catalog pointing to other catalogs (set up via the 'Custom Catalog/External Links' tab). You could therefore use this capability to handle your top level which contains the pointers to other libraries, and then create a catalog for each library. This was primarily envisaged as handling the case when you wanted to point to other catalogs provided by third parties, but I see no reason why it cannot be used by an individual with multiple libraries. If you do this you would want to also make use of the 'Profiles' facility so that you can store the settings you want for each individual library. Would this perhaps meet your needs?
itimpi is offline   Reply With Quote
Old 10-22-2012, 01:27 PM   #12
hakan42
Zealot
hakan42 is on a distinguished road
 
hakan42's Avatar
 
Posts: 136
Karma: 60
Join Date: Jul 2009
Location: Munich, Germany
Device: Nook Classic rooted; Galaxy S IV with Aldiko, other older devices
I have to experiment a little bit with the current calibre2opds, maybe on the weekend. My installation is still 2.x

I could imagine that my largish set of libraries (if I would go with the one shop / one library setup) would make searching / browsing for a certain book harder though. Let's see how the handling would be...
hakan42 is offline   Reply With Quote
Reply

Tags
database, directory name, directory_file_names, plugins

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
File naming on Kindle tomsem Calibre 2 09-15-2010 01:05 AM
File naming convention for best import result? Belfaborac Calibre 1 06-07-2010 09:14 AM
Functions for directory & file naming LARdT Calibre 4 01-09-2010 12:06 PM
File naming strangeness HarryT iRex 2 06-05-2009 11:39 AM
Naming PDF's malduin Sony Reader 7 09-02-2008 09:46 AM


All times are GMT -4. The time now is 11:32 AM.


MobileRead.com is a privately owned, operated and funded community.