[GUI Plugin] Quality Check

[theducks]

Originally Posted by kiwidude

@theducks - Re your "oddity" test case - that should get found when you run the duplicate series check though right? The way the check works currently there is an "assumption" in this check that you have eliminated your duplicates first.

As for your [123] case - well that is a perfect example of the limitations of this sort of check, absolutely nothing I can do about it I can think of. Serves you right for getting "funky" with your omnibus

[Pinecone]

I would use 1.2.3 or 3.2.1 or 1.23 or 3.21 for collections of books.

That way they would sort in the proper order with the other individual books.

But I have just put them in as 1

[theducks]

Originally Posted by Pinecone

I would use 1.2.3 or 3.2.1 or 1.23 or 3.21 for collections of books.

That way they would sort in the proper order with the other individual books.

But I have just put them in as 1

You can't use 2 decimal point marks in a index.
valid:
.123
1.23
12.3
123

[drMerry]

I love the option.
Thanks
Will have to use it in sub-functions.

I ran it twice. once on my epub files and once on all files.

I got these reports:
log was not available for first notice.
second notice had as log:

calibre, version 0.8.19
ERROR: Unhandled exception: <b>MemoryError</b>:

Traceback (most recent call last):
  File "calibre_plugins.quality_check.action", line 96, in perform_check
  File "calibre_plugins.quality_check.check_metadata", line 39, in perform_check
  File "calibre_plugins.quality_check.check_metadata", line 242, in check_series_gaps
MemoryError

This lib has 17526 books
1764 series
7865 books having series
largest series 239 books
about 600 series with one book (all possible indexes)

Spoiler:

5429 books with series_index 0

There seems to be a bug in calibre telling all books to have a series_index (series_index:true returns all books)
6305 books with series_index 1

[kiwidude]

@drMerry - can you zip up your metadata.db and PM me a link as you have done previously so I can run this myself?

[drMerry]

at least someone is happy with my bugs

[kiwidude]

Originally Posted by drMerry

at least someone is happy with my bugs

Who said I was happy?

So as far as I can see there isn't a bug in the plugin, it is just the same issue mentioned above of some "unusual" shall we say series index numbers. So for instance you have a series entry in there of:
De Jonge Lijsters [199304]

So straight away that is going to add 199303 missing numbers in the series. Which combined with other instances of this blows up the logging with the memory error you had above.

I think what I am going to do is out some sort of threshold check in there which will ignore series which appear to not be "start from 1" type series indexes. Easiest will be to take the maximum number in that series, and if it exceeds say 3 digits (i.e. 1000 or greater) then it won't attempt the series gap check and just mention it in the log instead. At the same time I will make a change to cater for duplicate series index values which theducks had in his library.

[kiwidude]

Changes in this release:

• Tune the check for series with gaps to exclude series with indexes >= 1000 and handle duplicates

@drMerry - I found the uber culprit for your issue - there is one series in there that has a single entry with a series index of 10,000,000. Now you will just see series like that ignored on the report.

[drMerry]

Originally Posted by kiwidude

Changes in this release:

• Tune the check for series with gaps to exclude series with indexes >= 1000 and handle duplicates

@drMerry - I found the uber culprit for your issue - there is one series in there that has a single entry with a series index of 10,000,000. Now you will just see series like that ignored on the report.

hmmm, And I thought I had all the books in that series. Seems I have to look for some missing parts

The report is confusing in this case.
It tells me I had a lot of gaps. But in my opinion that is not true, I have a lot of missing books.

If I have a series containing 1, 3, 6
I have 2 gaps and 3 missing books counting till the max provided index number.
Afterwards I thought you did mean items with books (and I realized that you probably called it gaps to overcome the missing books in the series after the main index found).

I should suggest to change the function to count the gaps or (and?) change the dialog to display
Found x missing items (up to biggest index found per series) in y series


or

Found x missing items in y series
(no. of Items are calculated based on main index found)




Another 'problem' is that I can not find missing indexes of my series with index based on year. (e.g. (not in my lib) Guinness book of records 2000, 2001, 2003).
To fix this problem I should propose a config function:
Skip gaps larger than: x

That option would tell the plugin to not count missing indexes if more than x missing items in one gap.

assuming x is set to 1999

Guinness book of records 2000, 2001, 2003
would return 1

Guinness book of records 1, 2000, 2001, 2003
would return 2000

But this is an advanced function. For possible another future release.
I'm very happy with the current version and (again quick) fix.

Thanks for the great work again.

[kiwidude]

@drMerry - I don't know what you mean by this gaps/missing books thing. They are *gaps* in your series index, it is nothing to do with whether you have book formats on empty books or not. If you have book entries for 1,3,6 only, it will tell you the gaps are 2,4,5. If you have 1,3,6 with book formats and an empty book entry for 2, it will tell you that 4,5 are gaps. Whether a book entry does or does not have a format is irrelevant.

If you want to see what books in a series you don't have that exist in your calibre library, you would just do a search for formats:false.

[drMerry]

No that is not my point.
The point is a discussion about gap vs book (meaning entry).

If I have 3 entries (index):
Book a (1)
Book b (3)
Book c (6)

I'm missing 3 books (2, 4, 5)
But I have 2 gaps {2}. {4,5}

Entries  3
Missing  3
Gaps     2
1-.-3-.-.-6
.-X-.-X-X-.
.-X-.-XxX-.

[kiwidude]

Ahh, I understand what you were saying now. A gap of 2 in your above example is actually a useless piece of information to a user. I didn't use the word "missing book" because to many users (who don't accumulate formats) there will be confusion over whether that means a book entry missing a format or not. Personally if someone asked me how many gaps I had in a series my instinct would still be to say 3 rather than 2.

If others think it is worth renaming I can do that in a future release.

[drMerry]

Originally Posted by kiwidude

I didn't use the word "missing book" because to many users (who don't accumulate formats) there will be confusion over whether that means a book entry missing a format or not.

I understand. No bug == no problem.
I can imagine only one case where the distinction between gaps and items would be useful: series with same name and index like Volume/Issue

Series 101
Series 102
....
Series 152
Series 201
Series 203
...
Series 252

In this case, you should expect 1 gap and 48 missing items
If you get 2 gaps and 49 missing items, you're more warned than in case of just 48 gaps.

(Think of 8 series, you have to calculate the number of missing items, but if you have 10 gaps, you know for sure there is something wrong)

But this is only be useful if there are users that use their index in this way. (in my case it isn't because I use the fraction to distinct the issues)

[drMerry]

When books have duplicates, they are marked as missing.

For the series I found, all books had duplicates except 1 or two.

So I had series like:

1
1
2
2
3
3
4
5
5

I think here 4 is marked as missing.
Is this on purpose?

[drMerry]

update.
Also with a series like:
1
1
2
2
3
3
4
4

It is reported as having a gap