Trying to do a few things in Calibre

[Starson17]

Originally Posted by ATimson

I think the reason people have problems grasping that is because that's not how it works; it's not a black box, it's a transparent one. A truly black box would be keeping the files as binary blobs in the database.

You can open binary blobs, black boxes and transparent ones. You just don't need to.

It's also not how, say, music library software works. Yes, software like iTunes can keep your files organized on disk, but it doesn't force you to like Calibre does.

I agree that music software is different, but music software I've used doesn't have the ability to run a remote server or export with a filename that I choose to a directory I choose with naming based on the metadata while supporting multiple different players, each with a different naming convention. I just don't see any forcing going on, other than that it needs its own copy of the ebook to serve out.

I understand that people are used to going directly to the folder their books are in, and when they see Calibre using a folder, they try to do the same thing, and object when they don't see the folder and file names they want. That's how it initially struck me, too, but that's not Calibre's fault.

[nerys]

You want some GREAT music editing software that does all those things? a GREAT program that is free called ID3-TagIT

You can "fill" meta data from the file name and vice versa. Name and structure (storage) based on the meta data. Very very cool software! (you need .net so make sure you have that) and it lets you putz with those oddball tags fields that only the big names feel a need to use :-) hehe

and it IS calibre's fault. Per say only because that's what it does. "normally" software does not "mess" with your files. it just "does" its thing and access the files from where they lie. Thats how itunes works thats how zune works that how a lot of other software works.

Calibre takes it unto itself and "REDOES" the very file structure the way it wants to instead of "leaving it alone" the way it was like most software. This is why when I use new software I disconnect my primary store and give it "sample" files to "see" what its going to do.

This is not an intrinsically bad thing but IT IS for a fact not "normal" behavior so it is calibre's fault that people experience this. ie it could have like other software LEFT the file structure alone.

[ATimson]

Originally Posted by Starson17

I just don't see any forcing going on, other than that it needs its own copy of the ebook to serve out.

I can't disagree with the literal content of that, but I think that the "but" in there is far more significant than you and Kovid do.

But I doubt I'm going to convince either of you of that anytime soon.

[hidari]

Originally Posted by nerys

>>>>Why?

You know its amazing how easy my system is. Say I want to read Kress's Probability moon.

its so easy I click Start then click Search. I select my external drive and type kress or moon and WOW half a second alter their it is. I drag and drop the file onto my reader's mem card and away I go.

or I click on My computer goto external drive click books click kress drag and drop probability moon. time consumed? less time than it takes to OPEN Calibre (NOT a cut on calibre mind you its a VERY nice program just not efficient for the way I read)

when you think "browse and explore" the calibre system works GREAT and my system sucks.

Looks like you already have your own system, So don't use Calibre as a library just as a book converter.

You may rub people the right way, that happens; however, your complaints seem childish.

Don't like Calibre(or if it does not live up to your standards), Don't use it.

[Starson17]

Originally Posted by ATimson

I can't disagree with the literal content of that, but I think that the "but" in there is far more significant than you and Kovid do.

But I doubt I'm going to convince either of you of that anytime soon.

I recognize that it's significant, but that's the way it's designed. A little disk space seems to be a small price to pay.

I just checked. I have 14,000 books, 50% with covers, all with metadata and most with at least 3 formats (about 35,000 files). It uses about 18 GB. That's less than one of my 50 GB BluRay movies. With TB drives running about U.S. 100$ it costs me $1.80 for all my book storage.

[nerys]

I am not concerned with space but with alterations to my structure. NOTE hidari please read the whole thread. I did not complain. I noticed something I "desired" to happen and simply asked if it was possible. I did not complain till people "whined" about me not submitting to their idea of the "right" way to do it.

The right way is my way :-) thats the nature of this kind of thing. When people come off pompous and holier than thou toward me because I do it at all different than they perceive as right. Well that tends to not sit so well with me. :-)

A cardinal NONO in software is to screw with peoples files. Calibre violates this unspoken rule. No big deal really but it is "non standard" and any semi experienced user is smart enough to not let this be a problem ie you test first before exposing your 35,000 book library to a new program that is unknown to you. (to use your example starson comment NOT directed at you)

I have a book in a folder of a specific nature.

I put that book in calibre

unless that book and that folder are precisely inline with how calibre is designed to handle them IT CHANGES THEM without asking me.

that is the very definition of "forcing"

Now keep in mind I have no problem with calibre. I have issue with how its creator responded to me but hey he is human I am human I deal with it.

The software is what it is. Like any selection if it does what I need great if not I move along. In this case it does SOME of what I need and it was "close enough" to what I needed that I figured it was worth an inquiry as to whether I could "make it" what I need.

This is not about right and wrong. This is about what I want. Period. End of discussion. If I can make it do it my way. Great bonus. If no. Great no problem.

but I will not change the way I do it without a VERY compelling reason. If calibre loaded in 5 seconds and had a much more advanced UI it might work for ME to put in the effort to bend to its way of doing things.

Calibre's way is not one tiny bit better than my way. Its just a "different" way and "different" is not worth changing for.

How a person interacts with their files is a very personal thing. We evolve to do it "just" such a way that we are comfortable with. I will NOT change how I do this because some program wants it differently. Not without a compelling and WORTHWHILE reason.

as it stands its UI is not even close to ideal so I was hoping to bend it to MY way. I am no programmer so I am at the mercy of those who can program.

So far no one has come up with a better more compelling method than a standard windows explorer file structure hierarchy. Its clean its faster than I can click and its efficient and consistent across the board.

what I did not need was the attitude.

[Kristo]

You are suffering through your music library like a nicotine addict with emphasize, trying to enjoy a simple cigarette. To help you I will direct your attention to MusicBrainz Picard tagger and MediaMonkey.

Use the MusicBrainz Picard tagger to automatically identify and clean up the metadata tags in your digital music collections and be sure to peruse through some of the additional scripts out there to make use of as well. It's not sexy but it is a drag 'n drop work horse.

I now suggest MediaMonkey to manage your physical library. Picard has done the tedious part gloriously but the organization and missing tidbits of information can be handled nicely by the monkey. If you still need iTunes as an interface for an iphone/ipod then just delete it's library and then rediscovery it.

Either of these programs will allow you to easily adjust metadata but mediamonkey will be a better choice to select multiple tracks to then right click and edit metadata. Changed the name of the album, click the album art tab and select one picture for all. Done. PS Your way is like fingernails on a chalk board but this is how I would do your way for you.

Your current way of searching for and then dragging and dropping it onto your card shouldn't have be altered in anyway. The book would still be named the same as it was (presumable author-title) before, right?

Although I don't share your exact position, I too have a large amount of ebooks in my collection. My intention with calibre was to manage the entire thing. I didn't mind the file system it used, nor did I mind the prospect of having tags for all. The problem I do share with you is the time involved in adding all these books to calibre.

I have added large amounts at once but this has proven to be too much for calibre and/or my system. I have added smaller amounts at the same time but the percent of properly tagged books wasn't much to write home about--It seems to really like to classify my books as audio books for some reason and ascribe ridiculously large titles to it. Thus, wrong and faulty tags aplenty, becomes only slightly less time consuming than searching amazon myself for the metadata pertaining to each book. This is why I have scaled back my aspirations of the expansive impeccable library and now only add the authors I am reading throughout the week.

What cought me initially about this post was the idea to utilize two library. One would be the Agglomerate Library consisting of the unrevised bulk-tagged books and the other would be the Consummate Library. I need a way to sort the books that have various formats or rather the ones that don't have various formats.

I donated a small amount to kovidgoyal/calibre but in lieu of this post and kovidgoyal interaction with you, I'm feeling a little regret for the donation. Very childish. Don't respond in this manner. If someone doesn't want to sleep inside your sleeping bag, rather under it; point out that half the bag will serve as insulation from the ground while the other half is made to perform as well as a typical blanket, with a zipper! If they just want to use it as a pillow and ask for any tips; don't treat them as if rape and your mother were involved in the story. I'm certain I will donate towards the program again but your level of respect will reflect the amount.

[Kristo]

Originally Posted by nerys

A cardinal NONO in software is to screw with peoples files. Calibre violates this unspoken rule. No big deal really but it is "non standard" and any semi experienced user is smart enough to not let this be a problem ie you test first before exposing your 35,000 book library to a new program that is unknown to you. (to use your example starson comment NOT directed at you)

I have a book in a folder of a specific nature.

I put that book in calibre

unless that book and that folder are precisely inline with how calibre is designed to handle them IT CHANGES THEM without asking me.

that is the very definition of "forcing"

I doesn't do anything to your book. It makes its own copy. At least that's what's goin on over on this end.

[nerys]

Media Monkey is awesome! I love media monkey. Itunes is mandatory sadly. Media Monkey will not properly handle podcasts or videos or audible for the little ipod it worked fine for the 4th gen ipod but pukes on the 5th gen ipod so clearly apple once again putz'd with something to break third party software. Hopefully media monkey will update and I can once again ditch itunes.. I need itunes for that sadly. I DO NOT use itunes to manage my music. I manage my music and use itunes strictly to "move" the information to the ipod. I also do not use media money for managing my music. I manage my music just fine using my folder structures. Works great. I use media monkey for light metadata editing and playback while editing.

For heavy duty editing such as songs with no metadata or bulk edits I use ID3-TagIt (media moneky can do similar work but is much slower)

------------

alas most of my books are in txt and pdf format and neither of which seem to have any metadata (so far none have had data already when I put them in calibre) it does do a good job of getting it though I have had it come up with some verboes titles :-)

I was largely intrigued by the SUMMARY included with the metadata but then I realized there is no easy way for me to READ the summary except IN calibre which would be extremely cumbersome beyond a few dozen titles since the original txt and pdf file do not hold metadata and the lrf file is not readable on my computer unless I install the unwanted sony software (is there a simple freeware lrf reader anywhere?)

I wish I could just go epub for the sony but the firmware has extreme issues for my model sony so I won't upgrade.

[Starson17]

Originally Posted by Kristo

I too have a large amount of ebooks in my collection. My intention with calibre was to manage the entire thing. I didn't mind the file system it used, nor did I mind the prospect of having tags for all. The problem I do share with you is the time involved in adding all these books to calibre.

I have added large amounts at once but this has proven to be too much for calibre and/or my system. I have added smaller amounts at the same time but the percent of properly tagged books wasn't much to write home about--It seems to really like to classify my books as audio books for some reason and ascribe ridiculously large titles to it. Thus, wrong and faulty tags aplenty, becomes only slightly less time consuming than searching amazon myself for the metadata pertaining to each book. This is why I have scaled back my aspirations of the expansive impeccable library and now only add the authors I am reading throughout the week.

Yes, entering large numbers of books can be tedious, and properly tagging them incredibly time consuming. To help with that, I've written several minor improvements that are now in Calibre or being reviewed. The first was a lock on the title/author so that you can get metadata automatically, but not change the title/author. Yes, it was getting audio books for me too, which I didn't mind too much, but I did miind when it added the name of the reader for the audio book to the title and changed the title. The lock stops that. (It's in the main Preferences screen (don't overwrite author/title by default).

Second, I've written code that automatically sorts all books you add into existing books as new formats by matching author and fuzzy matching title. This speeded up entry for me by a factor of 10. I had spent 2 months entering my first half of the library and only 2 days entering the last half. It was particularly annoying when I added 3 formats of the same book, and it made 3 records, not one. I hope this module will soon be added to the main branch. It's still under review. Finally, I'm working on a "merge records" module, so you can merge two books into a single record when the author name differs slightly. Yes, entry can be a pain, but people are working on improving it.

[DoctorOhh]

Originally Posted by nerys

and it IS calibre's fault. Per say only because that's what it does. "normally" software does not "mess" with your files. it just "does" its thing and access the files from where they lie. Thats how itunes works thats how zune works that how a lot of other software works.

Calibre takes it unto itself and "REDOES" the very file structure the way it wants to instead of "leaving it alone" the way it was like most software. This is why when I use new software I disconnect my primary store and give it "sample" files to "see" what its going to do.

I must be using a different version of Calibre then you do. Calibre has never messed with my files or my file structure. What you state is just wrong.

Originally Posted by nerys

I am not concerned with space but with alterations to my structure.

Again with the "alterations to my structure" thing. Calibre just doesn't mess with anyone's file structure.

Originally Posted by nerys

A cardinal NONO in software is to screw with peoples files. Calibre violates this unspoken rule.

I have a book in a folder of a specific nature.

I put that book in calibre

unless that book and that folder are precisely inline with how calibre is designed to handle them IT CHANGES THEM without asking me.

that is the very definition of "forcing"

So if I understand correctly, if Calibre just hid the books you put in it from you (in a db or encrypted) then it wouldn't bother you. Since this program does not change your files or file structure one bit.

It seems the fact that you can see how Calibre is storing them for use is what drives you nuts.

Originally Posted by nerys

but I will not change the way I do it without a VERY compelling reason. If calibre loaded in 5 seconds and had a much more advanced UI it might work for ME to put in the effort to bend to its way of doing things.

Why are you adding books to Calibre at all? Everything can be done from the command line including news fetching. Just create some batch files and the "slow UI" is a thing of the past.

Originally Posted by nerys

Calibre's way is not one tiny bit better than my way. Its just a "different" way and "different" is not worth changing for.

Calibre never asks or requires you to change the way you do things. Your file structure and search tools are the same whether you use Calibre or not. You already stated that space wasn't an issue. Without space being an issue then your argument that it changes your structure is moot.

Originally Posted by nerys

How a person interacts with their files is a very personal thing. We evolve to do it "just" such a way that we are comfortable with. I will NOT change how I do this because some program wants it differently. Not without a compelling and WORTHWHILE reason.

The fact that Calibre structures its data one way and you structure your data another way does not compel you to change your data. You keep going over the same non points.

Originally Posted by nerys

So far no one has come up with a better more compelling method than a standard windows explorer file structure hierarchy. Its clean its faster than I can click and its efficient and consistent across the board.

Post after post and hardly a constructive point muttered.

Originally Posted by nerys

what I did not need was the attitude.

But all you have is an attitude, anal retentive to the point that you keep insisting that a program is trying to compel you to change.

Good Luck to you in accomplishing your goals.

[DoctorOhh]

Originally Posted by nerys

I was largely intrigued by the SUMMARY included with the metadata but then I realized there is no easy way for me to READ the summary except IN calibre which would be extremely cumbersome beyond a few dozen titles since the original txt and pdf file do not hold metadata and the lrf file is not readable on my computer unless I install the unwanted sony software (is there a simple freeware lrf reader anywhere?)

I place the metadata in the lrf when I convert my files to that format. This places the summary on the first page beyond the cover. I attached an example.

It is nice having the summary, series etc at the beginning of the book.

Plus if your Calibre Library has summaries on every book then creating a catalog creates one book with all of the titles, series, summaries, etc in one convenient package.

The lrf viewer within Calibre works for any lrf on your drive. I have lrf associated with the lrf viewer. It does not view the drm'd lrx.

[nerys]

Hmm I wonder what I am missing. When I tell it to view an LRF nothing happens. I mean literally nothing. I wonder if I have a bad install. I will try reinstalling again. Or maybe since I DID have the sony software installed (needed to install the driver for the reader and the only way to do that was to install the sony software) when I uninstalled the sony software it might have left the association blank??

I tell it NOT to add the summary as first page of the LRF since if I am viewing it on the reader I have already decided I am going to read it and have no need for the summary.

One of the reasons I was was intrigued by Calibre is I can view the summary "in" calibre before I decide to put it on the reader.

and I am not sure how you mean it does not change the folder structure. I "watch" it change the folder structure based on the meta data it downloads in real time.

When Itunes or media Monkey or zune get new Metadata for an mp3 they DO NOT change the file name of the MP3 they DO NOT change the folder its located in or the position in the tree its located in. They do NOT create a duplicate set of files (note a conversion is not a duplicate to me)

Calibre does. Try changing the "author" name of the book in calibre while the books folder you assigned to calibre is open.

it will alter the folder name in REAL TIME as you alter the meta data. How you could possibly not notice this is beyond me.

WORSE it puts each book in its own sub folder now. so instead of

Root/author/list of books I now get

root/author fluid/list of folders of book names fluid/single book

fluid meaning calibre changes this file or folder name as the metadata changes.

so instead of a single author folder and all of his books I get a single altered author folder (sometimes) and a LIST OF FOLDERS one for each book (why?)

Changing the metadata should not entail changing the file name or the folder name/structure.

again not a big deal but it is a fact. Even the ilk known as itunes ASKS you first. do you want to move all the files it finds to a single library or LEAVE THEM WHERE THEY ARE and database them as they lie.

again I am not complaining about calibre its fine for what it does. Convert my files and thats how I use it. I am arguing with you guys over topics YOU guys brought up. I simply started this conversation with a few requests. YOU ALL turned it into this long whining session. not me.

[Kristo]

Nerys, upon the new install, be sure to designate the library to /program files/calibre2/Library. I think this may be helpful to you. This is how mine is set up.

My old library, the uncalibrized books, are still in /My Documents/My Anthenaeum/blah/blahblah. They are not altered in anyway. When I add books to calibre, calibre simply creates its own copy and stores it out of sight. Calibres library will be altered in real time but your library will just sit and wait patiently until something wants to know what its got.

Perhaps you could set up calibre to convert all the documents as you add them. That way, the calibre library won't be full of duplicate document that are found in your original library. This will still allow you to 'quick search' the calibre library from the start menu and drag 'n drop onto the card.

I don't think you have been out of line nor do I think you have been complaining, but rather, you have been defending your original query and subsequently, the precise way you utilize your library. Your explanation of how you would like to implement calibre didn't encompass much of calibre's phenomenal features so you were introduced to some features and both sides had failed to fully communicate their positions/system strengths.

Without addressing the ltf thing(what is this? I don't know.), I would like to see if I understand a possible utilization of calibre for you.

Place calibres library somewhere pertaining to system use rather than end user perusing, along the lines of /program files/calibre.
As needed, enter books into calibre but do the whole folder pertaining to that author. Sooner than never you will have a decent amount of quality ebooks within the calibre library.
As this collection grows you will find more and more useful elements of calibre. For instance having your library accessible from anywhere in the world and lets not forget that calibre will be growing at the same time.
You will still be dragging and dropping from the search results so that won't change and the integrity of your current library won't be altered so nothing lost there either.
You will learn a few tricks that will improve your utilization of calibre and I am sure by the end of summer you will find nearly all of calibres features instrumental in the way you use your reader—or I'm wrong and you have lost nothing more than time and the respect for whats-his-name.

That's all I've got to say about that.

Edit: Consider less capslocks. I feel like I'm being yelled at in an email from a great aunt-somebody. No intention to insult you, just say'n is all.

[Starson17]

Originally Posted by nerys

Calibre does. Try changing the "author" name of the book in calibre while the books folder you assigned to calibre is open.

it will alter the folder name in REAL TIME as you alter the meta data. How you could possibly not notice this is beyond me.

Once upon a time, Calibre kept each book in its database, hidden from prying eyes. When a user wanted to read a book, he selected the books he wanted, connected his Sony reader and they were sent over.

The users added books, peace prevailed, users (and the database) grew fat and ....

ALL WAS HAPPY IN THE LAND.

As time went on, however, a black cloud grew on the horizon. Soon reports came in from the frontier - something was happening to those who had prospered. Fear and confusion spread - the evil demon "Limited Resources" had struck. Far and wide the fat and happy databases were being slaughtered. They were too fat and too slow to escape the demon's evil influence.

Riding to the rescue was the White Knight of "New Design!" Let not the fat databases die the death of eternal slowness as they flee the evil demon's power. Henceforth, all shall be nimble as they shall no longer keep their precious books within. Books shall be kept in a secret cave, and only the location within the cave shall be kept by each database. Upon request, each precious book shall magically appear beside the database with whatever name and in whatever shape or form(at) is desired!

Now each database was nimble as none had to bring with it all the heavy precious books. Each could twist and dodge to avoid the evil demon with ease and once again .....

ALL WAS HAPPY IN THE LAND.

Word of the wonderful database began to spread, but as it did, rumors also began to spread. The secret cave and the precious books within had been found! One said: "I think the precious books should be stored in jars." Another said: "No I think the books should be stored on shelves." A third said: "No, they should be stored in chests!"

Panic began to spread as each new user worried about how the precious books were stored in the cave. The White Knight "New Design" tried to calm the worries of the crowds and dispel the confusion. "Remember that each database can magically call up the precious books from within the cave!" "Worry not about how they are stored within. Each can be called to appear in whatever guise thou wishest."

The old men of the village nodded sagely, remembering the evil times afore when there had been no magic cave to find or worry about and all had suffered the depredations of the evil demon. ... Yet still there were those who feared. Perhaps someday, however, again ....

ALL (WILL BE) HAPPY IN THE LAND.