[GUI Plugin] Quality Check

[papagelos]

Whoa!

I am overwhelmed by the astronomical number of options for this addon.
I have no idea where to start, since I am also new to Calibre.

For us new guys, is there some kind of button called "RUN ALL TESTS", so we dont have to click 300 different tools, and then also an option to "AUTOMATICALLY CORRECT OBVIOUS GLARING ERRORS", so we dont have to care about that, and then the other errors can be a Y/N prompt, as "Would you like the program to change author format to LastName, Firstname for all Entries?"

I have 15000+ books in my library, any sort of automation is good at this stage

[kiwidude]

There are multiple reasons why there is no "Run all tests" feature. For a start there are a number of tests which are mutually exclusive - some people want authors FN LN, others want them LN, FN. Some people want metadata jackets, others hate them. Other tests may be of no interest whatsoever as not being something you care about. So such a feature would need to have the ability to run a selection of tests rather than all.

The next problem is that many of the tests can produce "noise" - false positives mixed in with the genuine books of concern. So a "Run all tests" would produce absolute reams of output which would swamp users missing the wheat from the chaff. I recently added the ability to specify exclusions for a test which might be extended to help but it would force users into using them.

There is also the amount of time taken - as the tests I have added have gotten more complex (including scanning the content of each html page in an epub for instance) so has some of their running times.

The biggest issue by far however is working out a UI to trying to present the results of these tests. Producing some giant report wouldn't be very useful and painful to try to work with. Stepping through one test after another would be very long winded and painful. I haven't given it much thought because it all involves much pain.

With all that said, I do see some value in perhaps offering the ability to produce a summay report for a selection of tests. It would run all the tests and produce a summary (might just be a simple text log to begin with) that would tell you how many "issues" you have after running each test. You could copy that to Notepad or whatever and then individually run just those tests which showed issues you were interested in and resolve or exclude them. Note I am just thinking out loud here, not promising to implement!

In answer to your other question about where to start - well it depends on what is of interest to you and that may evolve over time as your library gets progressively cleaned up. Personally I would start with your book metadata (in conjunction with the Find Duplicates plugin) to make sure your titles, authors and series are rationalised/correct. If you don't care about stuff like comments, publishers etc then simply don't run those options (or leave them to a later iteration).

Then presuming ePub formats are your key master format ike they are for me and many others, start hitting some of those options and if desired use the Modify ePub plugin to fix them. Personally the only ones I use regularly is to identify/remove <address> tags (since they screw up book formatting) and remove metadata jackets (as they look fugly).

Just experiment and have fun with it - there are no rules about how OCD you have to be about your library, the plugin should help with a range of obsessivenes...

[papagelos]

Wow Kiwi, that is quite an answer.

Thanks for taking the time to address my concerns, and even if I still have no idea what a "metadata jacket" that you put such importance at, I do understand the predicament of presenting the output data in a useful way.

I still feel there should be some sort of "advanced mode" in this addon that presents ALL those options, and a "simple" mode for people like me so we do not get discouraged by all these options, 90% of which we do not understand anyways.

I was hoping there was some definition of "error" regarding a book or a library and that your addon could automatically correct said error, but from what I understand from your post it all comes down to preference and there is nothing your addon can do automagically to satisfy noobs like myself as well as advanced librarians if left to its own AI.

I wish I was as smart as you, and could make head or tails of all these tools you include. But for now I shall have to remain a guy that likes to read books and just has a passing interest in whether his library has thousands of errors or not, and you will remain the guy with the brain that provides other guys with brains good utilities

[theducks]

Originally Posted by papagelos

Wow Kiwi, that is quite an answer.

Thanks for taking the time to address my concerns, and even if I still have no idea what a "metadata jacket" that you put such importance at, I do understand the predicament of presenting the output data in a useful way.

Think the 'Dust cover' of a Hard bound book on Steroids.

All is really a bad idea
There are things that may be wrong, but might be correct for YOU.
But Kiwidude pointed out the Biggie. The report (really a filtered book List) can only show ONE type of problem at a time (gets cleared when another check runs)

And lets face it, Many of those checks don't apply (or they care less) to most folk.
Important to ME:
Title case, Author Sort, No XPGT margins, Jacket problems, (No) Smart tags.

All the rest

[kiwidude]

Originally Posted by papagelos

I still feel there should be some sort of "advanced mode" in this addon that presents ALL those options, and a "simple" mode for people like me so we do not get discouraged by all these options, 90% of which we do not understand anyways.

You can do this yourself, if you go to Customise plugin on the menu for this plugin. It allows you to select which sub options you want displayed. So you could begin by unticking all the ones that mean absolutely nothing to you and end up with much simpler menus.

Don't forget there is a Help file included with the plugin (choose Help on the Quality Check menu) which describes each of those checks in detail as to what they do and why you might want to use it. I suggest you bring that up (it displays in a browser) and decide based on that what menu options might be of interest to you.

I was hoping there was some definition of "error" regarding a book or a library and that your addon could automatically correct said error, but from what I understand from your post it all comes down to preference and there is nothing your addon can do automagically to satisfy noobs like myself as well as advanced librarians if left to its own AI.

As theducks has said, one mans trash is another mans treasure . What is important to one person to be considered an "error" is not of interest to someone else. This plugin attempts to cater to all tastes. Like I said above the most important thing in my opinion is to get your titles, authors and series metadata sorted out - as this will help you get rid of duplicates, searches to work more effectively, recover books that you thought "lost" because they are incorrectly categorised etc and most directly impacts your ability to manage a large collection and find the right book to read.

The other options like in the Check ePub menu are more about the quality inside the ePub itself, which are useful to identify for some people and completely irrelevant to others

[jackie_w]

@papagelos,
Stop worrying. If you're struggling to understand what all these 'errors' may be, the chances are that they're not important enough to YOU to be called 'errors' -- or at least not yet

If you're new to ebooks, forget about 'errors'. If you have OCD tendencies they will gradually become obvious as you read more ebooks. If you get to the point where some little thing starts to really irritate you, just remember kiwidude's plugin. It might help you track down the rogue books so you can fix them. If you're a relaxed easygoing type you may never need it.

[theducks]

Originally Posted by kiwidude

As theducks has said, one mans trash is another mans treasure . What is important to one person to be considered an "error" is not of interest to someone else. This plugin attempts to cater to all tastes. Like I said above the most important thing in my opinion is to get your titles, authors and series metadata sorted out - as this will help you get rid of duplicates, searches to work more effectively, recover books that you thought "lost" because they are incorrectly categorised etc and most directly impacts your ability to manage a large collection and find the right book to read.

I like your way of saying that better.

[P.K.Dick]

Hello all

I can't run this extension, it always stop at 99% with

calibre, version 0.8.47
ERREUR : Exception non gérée: <b>AttributeError</b>:'NoneType' object has no attribute 'split'

Traceback (most recent call last):
File "calibre_plugins.quality_check.dialogs", line 49, in do_book_check
File "calibre_plugins.quality_check.check_metadata" , line 101, in evaluate_book
AttributeError: 'NoneType' object has no attribute 'split'


what can I do ?

[kiwidude]

@P.K.Dick - welcome to MobileRead.

Looks like you have an author set on one of your books which is causing the check to fall over. I would love to know exactly what case this is. The line of code above is where it is trying to take the author(s) stored in the calibre database for that particular book and turn them into a list. See if you can find out which book that is using a process of elimination and search restrictions to isolate it down and then give me the details so I can handle it in the plugin. Sort your books by author and visually scan down to see if anything obvious stands out, but it looks like some sort of corruption to me.

If you can't figure it out I will do a special build of the plugin for you that will print a log of the books as it processes them.

[P.K.Dick]

Thank you

I found the book, the author seems to be empty, but when I copy it to a temporary library, the extensions works. So I kept "the bad one"
What information can i give you ?

[kiwidude]

Interesting - you are saying the Author field is actually blank, rather than containing "Unknown"? Does it get changed to "Unknown" when you copy to the other library?

I thought every book had to have an author, and calibre will always change it to "Unknown" if it is not specified. Perhaps Kovid may know the answer to this one - are you by any chance a long term user of calibre, and this book might have been imported with a very old version before this behaviour got "fixed" into its current implementation?

[P.K.Dick]

Yes the Author Field seems to be blank (or unknown characters ?) and it's a "recent" book (max 1 month) in my library.
And it works when this book is copied to an other library. (it doesn't remains blank !!!)

For the record, I have, using the Author's editor deleted a few "bad" Authors -> F2, backspace and tab

It looks that the database contains values unexpected for you; and that théy are corrected during the copy.

[P.K.Dick]

I reproduced the problem by erasing the author using the method up

[kiwidude]

Hmmm... that's what I find interesting, I can't reproduce this. If I hit F2, backspace (so the author field is blank) and then hit either Tab or Enter, then the Author field goes to "Unknown". You are saying you have a way to "erase" the author so that it goes "blank"? If so, could you describe it in detail, as it sounds like a calibre bug to me.

[P.K.Dick]

I use the "manage authors, tag etc"