Pluginize file naming strategy ?

[hakan42]

Hi,

before anyone points me to FAQs, yes, I have read PLEASE READ BEFORE POSTING - The purpose of this subforum and Want to change the folder structure of the Calibre library? and am aware of most of the arguments pro / contra.

My special use case ( content from different shops / sources aggregated and available on one server because *I* bought them for *myself*, but should be stored in separate locations for legal reasons ) would be best served by manipulating the storage strategy, tough.

So, my question is: If I would put in the effort to pluginize the directory naming strategy (much like the way metadata sources work now), and later on provided additional plugins that serve my use case, would that work be considered for inclusion in the main source tree or would I have to maintain my private tree for the foreseeable future, including the effort of building windows binary installation packages?


Regards,
Hakan

[user_none]

If you do the work, you maintain the work and you can show that the changes are an improvement then it will be considered for inclusion.

However, you are going to be hard pressed to show that changing the library structure to something else is an improvement. Your also going to be hard pressed to show that pluginizing this is an improvement. In most cases not special cases.

I would look over the existing threads about the library structure pull in all reasons for and against then start a non-development section thread to discuss. Go over each for and against and make a clear argument as to why your changes re an improvement. Then take in the responses.and re work your proposal.

Again the majority of users and developers are happy with the current implementation. There is little, not 0, chance of getting this accepted.

[Starson17]

Originally Posted by hakan42

So, my question is: If I would put in the effort to pluginize the directory naming strategy (much like the way metadata sources work now), and later on provided additional plugins that serve my use case, would that work be considered for inclusion in the main source tree

You might want to read this post and the related thread:
http://www.mobileread.com/forums/sho...94&postcount=8

[kovidgoyal]

I have no objections in principle to having the directory naming configurable. However, various caveats apply:

1) It would have to be done in a way that does not break metadata backup/db restore/the cover storage APIs (probably others as well)
2) The default semantics must remain as they are
3) This kind of change will require *a lot* of testing, on multiple platforms, as it has the potential for data loss.

I'm somewhat puzzled by your use case, however. If you are serving up books from a server, what earthly difference does it make where they are stored?

[hakan42]

Originally Posted by user_none

If you do the work, you maintain the work and you can show that the changes are an improvement then it will be considered for inclusion.

Yes, I am willing to do the initial work and the maintenance. Actually, I am locally modifying database2.py and config.py since ages (since August 2009 to be precise) now to keep my estimated 100 lines of changes working.

Originally Posted by user_none

I would look over the existing threads about the library structure pull in all reasons for and against then start a non-development section thread to discuss. Go over each for and against and make a clear argument as to why your changes re an improvement. Then take in the responses.and re work your proposal.

Actually, I opened this thread intentionally here in the development area because I am willing to do the work and don't want to get swamped by the usual threads of the form "I want directories to be named 'LN FN' instead of 'FN LN'". This sums up about 80% of the directory naming threads, the rest being "I want to have the genre in the directory name" type of threads. Both get answered by the (very valid) argument of "It's not your directory tree, it belongs to calibre".

The code I am willing to create and maintain would keep all those guys off your back. You could just tell them that if they really really want to modify their storage, they can create / use a plugin, but the main calibre developers only supports the "default" configuration (which would work identically to the current code).

Originally Posted by user_none

Again the majority of users and developers are happy with the current implementation. There is little, not 0, chance of getting this accepted.

After some time, the majority accepts the argument that the storage tree of calibre is not theirs to muck around in it (which is a correct argument for most use cases). My use case(s), and the points where they differ from the mainstream calibre operation, I'll enumerate in my answer to kovid's posting.

Regards,
Hakan

[hakan42]

Originally Posted by kovidgoyal

I have no objections in principle to having the directory naming configurable. However, various caveats apply:

1) It would have to be done in a way that does not break metadata backup/db restore/the cover storage APIs (probably others as well)
2) The default semantics must remain as they are
3) This kind of change will require *a lot* of testing, on multiple platforms, as it has the potential for data loss.

I'm somewhat puzzled by your use case, however. If you are serving up books from a server, what earthly difference does it make where they are stored?

I can do testing on a Windows desktop, X on Gentoo Linux and a Macbook. Hopefully, this will cover the majority of the installations out there.

And my intention is to keep the default semantics if no plugin to modify strategy is configured.

And for the "where they are stored" part, one example are the ebooks from the http://www.blacklibrary.com. If you read their license rules, they demand that you provably remove all their files from your system if they decide to revoke their license to you. For some time, I actually thought about putting all the stuff I buy from them into a completely separate calibre instance, but then again, I'm not sure that their license would even allow me to store the epubs in calibre, anyway In that case, the proof that I removed everything would be done by showing their lawyer the directory where their stuff is, telling them that nothing of their intellectual property is in my other directories and kindly asking them to refrain from rifling through all my other stuff...


Regards,
Hakan

[kovidgoyal]

I seriously doubt that a lawyer would consider being shown an empty directory as proof of anything.

[hakan42]

Originally Posted by kovidgoyal

I have no objections in principle to having the directory naming configurable. However, various caveats apply:

1) It would have to be done in a way that does not break metadata backup/db restore/the cover storage APIs (probably others as well)
2) The default semantics must remain as they are
3) This kind of change will require *a lot* of testing, on multiple platforms, as it has the potential for data loss.

I'm somewhat puzzled by your use case, however. If you are serving up books from a server, what earthly difference does it make where they are stored?

Hi,

I re-applied my changes to the currently distributed 0.9.1 tag and sent you a merge proposal.

If accepted, this could be used to create plugins to put the source (see my main use case) or series information into the path of the books without having to modify the core of calibre anymore.

Also, if accepted, I would continue to work on this to allow graphical configuration of my plugins, which at this moment have a bit of hard-coded information in them (e.g. name of the column to use for "source" and so on).

Regards,
Hakan

[Agama]

Originally Posted by hakan42

My special use case ( content from different shops / sources aggregated and available on one server because *I* bought them for *myself*, but should be stored in separate locations for legal reasons ) would be best served by manipulating the storage strategy, tough.

Can't you achieve this by simply using multiple libraries or do your books all need to show up in the same library? I use 5 libraries with calibre, which gives me 5 separate folder locations on my hard disk.

[hakan42]

Sure, I could separate with multiple libraries. Buuuuuuuut...

On one hand, I have about a dozen book sources which I want to keep separated, so a-library-for-each gets unwieldy pretty quick.

On the other hand, calibre2opds can't merge multiple libraries into one opds directory so configuration on my reader devices gets complicated.

And, on the gripping hand, the OCD afflicted software engineer in me wanted a configurable solution which would also consider Series information and whatever the user might consider necessary . So, main reason is my OCD

[itimpi]

Originally Posted by hakan42

On the other hand, calibre2opds can't merge multiple libraries into one opds directory so configuration on my reader devices gets complicated.

I agree that there is no capability in calibre2opds to generate a single catalog covering multiple libraries (although it might be possible to develop this if a good enough case can be made).

what there already is is a capability in Calibre2Opds to link catalogs. You can have entries in the top level of a catalog pointing to other catalogs (set up via the 'Custom Catalog/External Links' tab). You could therefore use this capability to handle your top level which contains the pointers to other libraries, and then create a catalog for each library. This was primarily envisaged as handling the case when you wanted to point to other catalogs provided by third parties, but I see no reason why it cannot be used by an individual with multiple libraries. If you do this you would want to also make use of the 'Profiles' facility so that you can store the settings you want for each individual library. Would this perhaps meet your needs?

[hakan42]

I have to experiment a little bit with the current calibre2opds, maybe on the weekend. My installation is still 2.x

I could imagine that my largish set of libraries (if I would go with the one shop / one library setup) would make searching / browsing for a certain book harder though. Let's see how the handling would be...