Right Forum??

[karta]

I am in the proces of trying to catalog a rather large etext collection, and looking for methods to learn and to share.

However....

I ony read etexts wit dektops and a laptop. I have a reader, but find it of little use.

My pimary intrest is NOT using any library software, but rather scripts to categorize etexts into category directories. I do use calibre, but only to get titles and metadata.

Is this the right forum?

Other etext forums, at least from what i have seen appear moirbund. Any pointers would be appreciated...

[pdurrant]

You're making your task much harder by not using calibre to store your etexts. You can define custom metadata, and using calibre is so much less error prone than moving texts between dictionaries.

What are your objections to using calibre as a store for the texts and metadata?

[Dr. Drib]

^ I completely agree with Paul.

Calibre would be a great choice, and you can manipulate the Metadata in just about any way you may wish. Why do you prefer to use a script, rather than solving this with a software solution?

One of your other questions (asked twice) on whether this is the 'right' forum: Probably not.

[frahse]

Originally Posted by Dr. Drib

^ I completely agree with Paul.

Calibre would be a great choice, and you can manipulate the Metadata in just about any way you may wish. Why do you prefer to use a script, rather than solving this with a software solution?

One of your other questions (asked twice) on whether this is the 'right' forum: Probably not.

Well, maybe it is... The people here have information and skills galore!

Still, the question and objectives need to be worked out to give an idea as to what is really needed. I think I am hearing a predetermined preference of method as opposed to just a desired starting point and end result.

[karta]

Though I like Calibre, I really dislike the idea of requiring a GUI interface to acess my ever growing and shifting collections.

Crurrently, I use text files to store data in a large directory, and grep portions of titles to tell me where they are. The bulk of my non-fiction is on optical media.

Fiction is mostly across a couple hard drives, and is probably around 200,000 titles if not more. I am consolidating that now, and my ultimate nightmare would be loss of data due to software failure or deprecation.

Currently I use calbre on books with ambiguous or damaged titles, and export the etext, jpg, and opf files it generates to a category directory.
Flush, rinse, repeat.

Currently each author gets a single category, but in time I intend to fix that with symlinks.

The final steps would be a database with all relevant info on the etexts, but *not* the etexts themselves. Just a location pointer. MySQL or mariaDB com to mind.

The reason for the scripts is to automate as much as possible the processes for categorization, elimination of duplicates (for fiction I archive only .epub, and will eliminate by script any extra title that matches the .epub title, for example). another script will come in and then fix title formats,and then roughly categorize the etexts, as well as eliminate some garbage authors.

I am particularly intersted in mining the metadata out of the etexts themselves. With an epub it should be failry simple to batch process them and yank out the .opf files to the metadata. I can dothat by hand quicker than Calibre. other file formats are a bit more opaque, and I have had minimal success with scraping PDFs for info using scripts.

Most of all the system needs to be portble across Win and Linux. so i use Perl, and may switch to Ruby in the near future. With those languages i can build the database interface, into perhaps something like Perl based GC-Star

This is long term project, and for now I am still building on the basics. Like looking for a good author->category list to assst in my processing.
And perhaps a script for grabbing titles like what Calibre uses.

I can get access to the ABEBooks API, which is particularly useful, as a large portion of my collection is pre-ISBN .

PLUS, there is the additional requirment that I will be needing when I finally get around to cataloguing my modest personal collection of *real* books (around 5,000).

Or, to put it much more simply - as we are dealing with terabytes of data, do not see calibre or any typical reader software as being more than a tool in a larger chain. And there, GUI software just gets in the way - unless Calibre can be run from the command line with personalized scripts.

[pdurrant]

Originally Posted by karta

unless Calibre can be run from the command line with personalized scripts.

calibre can be run from the command line. It's also open source.

For what you describe (since you seem to be happy to learn different tools/languages), you'd probably be better off working from the calibre base. Adapting it to easily work with offline references to books should be trivial. You could, of course, do that without any programming with just the use of custom columns (which can be created and manipulated from the command line).


And I hate to ask a silly question, but you do have backups of your texts, don't you?