Pros & Cons of using the Calibre GUI

[sealbeater]

Originally Posted by BetterRed

@sealbeater - template based custom columns can make startup times slow, in particular templates that necessitate access beyond the metadata database - i.e. to the file system or network.

My biggest library, of a ~140,000 books, located on a Toshiba 2TB Sata III drive, does not have any non-metadata based template based custom columns. Starting calibre with that library in the All Books VL takes 7-10 seconds. That's on a 16GB i7 with a 256GB NVme system drive running Windows 10 Pro.

BR

Thank you, I'll keep that in mind. I'm starting fresh and the only columns I've added thus are the DDC/LLC/OCLC_OWI and Extra Author Detail.

[sealbeater]

Originally Posted by un_pogaz

Wut

I say exactly the opposite.
The management of Author names, serial names and their numbering (via Serie Managere), Tags, Publishers, Duplicate (via Find Duplicate) in mass, with dedicated modules, is very easy .

Is also capable of importing Metadata online (Metadata Source Plugins) it may be slower than a fully automatic script, but is here.

You wanted to send books not present on the e-reader => Virtual library with "ondevice:false", Ctrl+A, D send on the devices.
The Virtual library is one of the most powerful tools of Calibre, there are so many possibilities.

Not to mention the custom columns, gold mine and ultimate customization and filing tool.

And all this in a very complete and pleasant graphical interface, make all options, even the most advanced, accessible for the neophyte.
Don't forget that the main purpose of a software like Calibre is to make life easier for the uninitiated.

But one of the big things that Calibre makes very easy is the uniformity of the metadata.
And when you've tasted this (I tell you, I've experienced it), Calibre becomes irreplaceable.

Calibre is thus on the contrary, overpowerfull and he succeeds more than his role.

I appriciate your well thought out response, truly. However, I'm not a newbie with Calibre. I've been using it since at least 2008. I chose to move off of it for a better, scripted, automated metadata lookup file name renaming solution.

Originally Posted by un_pogaz

Your judgment of Calibre's abilities may come from the fact that you had used for a very long time a homemade/personal creation method that worked very well.
Learning a new tool is never easy at first, but if you do it right, you don't even realize you're using it every day anymore, and it's proof that it's done well.

If you would prefer search your books by their file name, chek the Calibre template language.
And nothing prevents you from always using your scripts before sending them in Calibre.

It's not a new tool. It's just not a tool that I need in any way. I'm not alone, the others who feel this way just aren't on this forum.

Following text even no more valid, because argument retracted

Spoiler:

By the way, I laugh. You wanted to discuss the drawbacks of the GUI, but what was the only one you had raised: that the initiation time is long?
You say 15 min (I hope you exaggerated), but you seem to be the only one in this case

You are not running Windows, so I can't say XP/ME/Vista. Only possibilities :
1) Your PC is made of cardboard
2) Your Library has settings that slow it down
3) Remote access to your Library is slow
4) You are impatient
In any case, it doesn't come directly from Calibre, but probaly from your side.
Talked of inconvenience is therefore a bit inappropriate and exaggerating for my taste.

Perhaps. We'll see. This was some time ago on a different computer. This isn't the fastest but should be good enough. Once I'm at .5 mil, we'll see what's what.

edit: Actually I'm pretty sure it was this one so strike that "different computer" comment.

[sealbeater]

Originally Posted by JSWolf

I doubt this to be true. You can use the Reading List plugin to put books on your Reader as soon as you plug it in. You cannot do this without Calibre. Also, finding and organizing is much easier.

Pretty sure I could script that with udev. Don't assume yours is the only way.

Originally Posted by JSWolf

If you have to hide what you are doing from your bosses, then you should not be doing this at work.

What I do is my business and I'll thank you to let me be the one to mind it. However, I prefer to have a screen full of text when I'm doing stuff rather than a big window full of book covers. To me, being able to manage and sort and preprocess and do all my other magic without needing a gui is of far more value to me than Calibre could ever be.

Originally Posted by JSWolf

As for tagging and organizing better/faster without Calibre, not possible.

You'll forgive me but you have no idea of the world I live in. You don't know what you are talking about.

Originally Posted by JSWolf

For one thing if you wanted to organize your eBooks in collections on the devices you have that collections are compatible with Calibre, then you cannot do it better/faster manually.

I dont' use collections but when i did, i used the PRS+ option that sorted directories into collections. Now that I have series info in my filenames, I only need to search for that.

Originally Posted by JSWolf

Even if you cannot manage collections with Calibre on your current device, you can present your eBooks to you in many different ways like by title, author, or series. How you do so is up to you. Once you setup your directories by hand, you are stuck with that way only.



This one is correct.

I respectfully disagree. I can always change how I present my data and rerun my script against my library.

[sealbeater]

Originally Posted by stumped

i am still missing the explanation of how sealbeater will ever manage to actually read that two million book collection

I believe I said the following:

Originally Posted by sealbeater

As for finding the time, I subscribe to the "anti-library" approach.

for the google impaired:

“The writer Umberto Eco belongs to that small class of scholars who are encyclopedic, insightful, and nondull. He is the owner of a large personal library (containing thirty thousand books), and separates visitors into two categories: those who react with “Wow! Signore, professore dottore Eco, what a library you have ! How many of these books have you read?” and the others - a very small minority - who get the point that a private library is not an ego-boosting appendage but a research tool. Read books are far less valuable than unread ones. The library should contain as much of what you don’t know as your financial means, mortgage rates and the currently tight real-estate market allows you to put there. You will accumulate more knowledge and more books as you grow older, and the growing number of unread books on the shelves will look at you menancingly. Indeed, the more you know, the larger the rows of unread books. Let us call this collection of unread books an antilibrary.” - Nassim Nicholas Taleb, The Black Swan: The Impact of the Highly Improbable

Originally Posted by stumped

maybe he reads VERY VERY fast, or maybe he has a severe case of undiagnosed kleptomania. my money is on the latter.

Your money would be lost. I haven't stolen a book since...jr high? I do however, read very very fast and have ways of reading even faster.

Originally Posted by stumped

I used to have a well paid job also, but paying for 2 million books at say $5 per book is ten million dollars. not something your average very well paid job can fund.

so I allege piracy, also.

I've been well paid and collecting books for a long long time.

Originally Posted by stumped

and a final debunk - from post 9- "10k books is not a even a 1/3rd of what is on my e-reader". Seriously? what mass produced e-reader handles 30k books. Is this one he built in his spare time, while not reading his 2 million books ?

Final debunk? What have you debunked previously?

As for what mass produced e-reader handles 30k books, my Onyx. There's a thread in the Onyx forum about it.

[sealbeater]

Originally Posted by theducks

Since, these books are 'at home' and hidden from the boss, I will assume they are not a work related Library (my old company HAD a real, technical library for the use by the engineering staff).

Exactly! My jobs have had the same.

Originally Posted by theducks

I can't imagine a 'private individual' needing to deal with that many BOOKS (I believe at least 1 of our members, catalogs papers. those can be small, short reads.)

Well, I guess that's one of the differences between you and I.

Originally Posted by theducks

Calibre can't deal with IMPORTING massive amounts of books in a single go. That's it. Feed it batches and it will chomp them down with ease.

I agree but I distinctly remember running into issues after half a mil. It got to the point where I would rather blow out my then .5 mill calibre library and start over with the scripts.

Originally Posted by theducks

The big task, is hammering the lousy metadata into something of beauty , that take TLC and time per book. (When I first got Calibre, I imported all the books (single format) on the Baen CD's. That took a bit of time. It took DAYS Weeks to properly Tag the series and metadata.

And...interestingly enough, that's where my scripts are coming in handy. Not *mine* but the guy who wrote it and my incorporations. They so far are proving to be a very handy way to "pre-process" my books. So far, the metadata lookups in calibre have been much easier as I always have the correct ISBN, regardless of my regex missing some books.

Originally Posted by theducks

While I have never seen startup times as low as reported (numerous systems. nothin higher than a core duo ), from the time I double click the icon on the desktop, to full user control of the GUI, is under a minute. No fancy SSD needed.

I really didn't want to get into technical specs.

AMD FX(tm)-8150 Eight-Core Processor
SSD
32G of ram
ZFS

Originally Posted by theducks

I also tend to leave Calibre running in background, so the OS discovering the USB device, is the more time consuming part of the transfer process. So the startup time argument is

I do too but honestly, why bother when i can do the same stuff calibre can (enough for my purposes) without it?

[sealbeater]

One of the ways Calibre sucks for large libraries? Slow metadata lookup. The author has gone on record saying that the metadata lookups are slowed down to not hammer the websites. My way, i can just use screen and spawn my script as many times as I like.

[davidfor]

Originally Posted by sealbeater

One of the ways Calibre sucks for large libraries? Slow metadata lookup. The author has gone on record saying that the metadata lookups are slowed down to not hammer the websites. My way, i can just use screen and spawn my script as many times as I like.

Which means you are hammering the websites to get the metadata. And that can lead to blacklisting your IP by those sites. It will depend on the site, but Goodreads didn't like 150 lookups in about a five minute period for me last night.

[davidfor]

Originally Posted by sealbeater

What I do is my business and I'll thank you to let me be the one to mind it. However, I prefer to have a screen full of text when I'm doing stuff rather than a big window full of book covers. To me, being able to manage and sort and preprocess and do all my other magic without needing a gui is of far more value to me than Calibre could ever be.

And strangely enough, I do have a screen full of text when doing stuff in calibre. There's this big list of the books that occupies 80% of the calibre window. And the rest is nice and convenient for me to click on.

And I know your response will be you use 100% of the window. But, if I was doing this one a command line, I'd be using two windows next to each other. One would be where I listed the books, the other where I showed more details of one book, or edit it or something else like that. I could do it all in one command window, but that means a lot of scrolling and repeating the same command to redisplay the list I'm working with. Which is exactly what I am doing now to look at problems on various servers I am working with.

[Terisa de morgan]

Originally Posted by sealbeater

To manage a large amount of books. When I ran it, I had .5 mill books and it managed but took a long time to open. This may have changed, this may have been an effect of the machine at the time, we'll see. I'm adding books now so I'll report if I have an issue.


As for doing other things like deduplication, it's best to use the OS tools rather than calibre.

Ok, I get about books. I cannot say anything about that because:

1. I don't have such a quantity of books
2. I have A LOT of custom columns (direct input and calculated), which slows down the library.

And deduplication... it's ok if your file name are standardized, in other way, not so great (I've tested both ways).

[Divingduck]

Originally Posted by BetterRed

@sealbeater - template based custom columns can make startup times slow, in particular templates that necessitate access beyond the metadata database - i.e. to the file system or network.

My biggest library, of a ~140,000 books, located on a Toshiba 2TB Sata III drive, does not have any non-metadata based template based custom columns. Starting calibre with that library in the All Books VL takes 7-10 seconds. That's on a 16GB i7 with a 256GB NVme system drive running Windows 10 Pro.

BR

Can't see this too. More records, lots of custom columns and running on older systems.

[un_pogaz]

Originally Posted by sealbeater

I appriciate your well thought out response, truly. However, I'm not a newbie with Calibre. I've been using it since at least 2008. I chose to move off of it for a better, scripted, automated metadata lookup file name renaming solution.

Oh okay.
My answer was inspired by the fact that "complaining" about loading time is more a remark from a newbie.

I would insist a little by advise you to try Calibre again. Apart from this new PC, Calibre probably improved a lot.
But, I let you judge which tools you use.

Originally Posted by sealbeater

As for doing other things like deduplication, it's best to use the OS tools rather than calibre.

Find Duplicate plugin read metadatas and has a syntactic tolerance margin.
And it works not only for books, but also for metadata (Authors, Tag...), which makes it possible to unified the Library.
If the first point is possible with OS tools, the second seems more difficult.

Various comments by various peoples.

Personally, I totally approve the disproportionate size of your Library. I too share this archiving culture/strategy.
I don't have as much as you, but I think Calibre can handle if we organize ourselves right.
So try it, you go pushed Calibre in its limits, your return will always be useful for the software.

[sealbeater]

Originally Posted by davidfor

Which means you are hammering the websites to get the metadata. And that can lead to blacklisting your IP by those sites. It will depend on the site, but Goodreads didn't like 150 lookups in about a five minute period for me last night.

That's why I proxy all of my requests though Tor.

People have thought about these things, you know.

[sealbeater]

Originally Posted by davidfor

And strangely enough, I do have a screen full of text when doing stuff in calibre. There's this big list of the books that occupies 80% of the calibre window. And the rest is nice and convenient for me to click on.

That's cute but they aren't quite the same thing.

Originally Posted by davidfor

And I know your response will be you use 100% of the window.

That actually wasn't my response.

Originally Posted by davidfor

But, if I was doing this one a command line, I'd be using two windows next to each other. One would be where I listed the books, the other where I showed more details of one book, or edit it or something else like that. I could do it all in one command window, but that means a lot of scrolling and repeating the same command to redisplay the list I'm working with. Which is exactly what I am doing now to look at problems on various servers I am working with.

I guess you are unfortunately stuck in your paradigm.

All I have to do is run two commands, depending on what I am doing:

ebook-reader-prep.sh

and

ebook-organize.sh

It's not much work at all, I just like to monitor the output when I am at work.

[sealbeater]

Originally Posted by un_pogaz

Oh okay.
My answer was inspired by the fact that "complaining" about loading time is more a remark from a newbie.

I would insist a little by advise you to try Calibre again. Apart from this new PC, Calibre probably improved a lot.
But, I let you judge which tools you use.

Well, I am. Starting fresh with data that has been processed beforehand, so I'm hopeful.

Originally Posted by un_pogaz

Find Duplicate plugin read metadatas and has a syntactic tolerance margin.
And it works not only for books, but also for metadata (Authors, Tag...), which makes it possible to unified the Library.
If the first point is possible with OS tools, the second seems more difficult.

Well, my approach is to have all the metadata needed available in the filename, my regex puts the ISBN in the right place, regardless of any misses. I had bad luck finding dupes with calibre due to the number of records. It's better to import with clean data.

Originally Posted by un_pogaz

Personally, I totally approve the disproportionate size of your Library. I too share this archiving culture/strategy.
I don't have as much as you, but I think Calibre can handle if we organize ourselves right.
So try it, you go pushed Calibre in its limits, your return will always be useful for the software.

Thank you.

[gbm]

As some one who started out using a mainframe @1k clock, magnetic core memory and using a teletype/paper tape reader as a syscon give me a GUI on a VDT. Sure I use cl when doing operating system chores in Linux when I must, but GUI means less chance of making a fatal error.

But use what you want just remember other people have different needs and ability.

bernie