[GUI Plugin] Count Pages

[kiwidude]

Changes in this plugin:

• Now requires calibre 0.8.57
• Store configuration settings in the calibre library database rather than a json file, to allow reuse from different computers (not simultaneously!)
• Add a support option to the configuration dialog allowing viewing the plugin data stored in the database
• Remove the additional menu items for individual word/page counts added in v1.4.0 as cluttered the interface

I've got a number of plugin updates that will be released today based around calibre 0.8.57, taking advantage of a new feature to store plugin settings inside the calibre library. The benefit of this is for users that store their library in a shared network location and use calibre from multiple machines (though never at the same time - that limitation hasn't changed!). Your plugin configuration settings which are specific to each library (such as custom column names, book exemptions, reading lists etc) will be automatically kept in sync when calibre is next opened on each machine.

Specifically for this Count Pages plugin it means you will no longer have to repeatedly setup the page/word custom column names for each library on each machine.

The other change was to rip out the additional menu options I introduced in 1.4.0. Having tried it for a few weeks it was an experiment which annoyed me from a usability viewpoint - it gave too many options to think about every time you dropped down the menu. If you only want word or page count, the behaviour is as it was previously - just configure the associated custom column for that option, and the single menu option works out what you want from that.

EDIT: I just found an issue I didn't catch before release - for the few of you that downloaded before I spotted it, please just hit the Install button again (version number is the same).

[Amalthia]

I was curious to know if others are able to see their Word Count column on Calibre's content server? I'm not sure if it doesn't show up because it's an integer column?

[Boekenworm]

I have a suggestion for enhancing the Count Pages Calibre plugin.
When counting the number of pages and words in an ebook, it would be very easy to also calculate how "difficult" it is to read.
Two formulas, Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FK) can be used to measure this. The only thing you need to do to calculate these, except counting the number of words, is counting the number of sentences and syllables in the text. These values could be stored in custom tags.

[kiwidude]

@Boekenworm - welcome to MobileRead.

I vaguely recall someone else mentioning this a long time ago, which is discussed here. It is an interesting suggestion for this plugin. I am guessing there are other similar type of statistics out there that others may know of - Gunning Fog I saw mentioned, any others?

[JSWolf]

I've just gotten around to installing this and I think it's quite useful to know how large an eBook is for when I am deciding what to read next. Sometimes I will pick one based on the size. I really like that the ePub page numbers are the same as what ADE shows. Well done!

[kiwidude]

Here is a fairly significantly changed version of the plugin that I would appreciate some feedback on. As per the request above it now includes the option to generate readability statistics, be it Flesch Reading Ease, Flesch-Kincaid Grade Level or Gunning Fog Index.

Note these statistics are calculated based on the assumption that the book is in English, I have no interest in adding support for other languages. It uses a heavily modified tiny subset of the NLTK library (Natural Language Toolkit).

So you can now add up to a further three custom columns to display these additional statistics should you so choose, as per the screenshot below. Create columns of type floating point number to store the value, and use a format specifier of something like {0:.1f} to display the value to one decimal place when configuring your custom column.

Currently the readability statistics are calculated across the entire book (but only if you have any readability analysis columns enabled). It is just as fast to compute two or three as it is to compute one statistic, so don't think that by only doing one of them you will make it faster! I haven't yet decided whether I should instead do some sort of subset sampling of the book and if so what guise that should take - e.g. first 5,000 characters or % based or whatever. Feel free to suggest if you have an opinion - or if you don't mind the speed since it runs in the background anyways then that is fine too.

You may also find it mildly interesting to look at the log to see the more detailed statistics gathered. You may also notice a small difference in the word counts (if you normally calculate word count) between the existing calibre algorithm vs the NLTK one. Not something I am inclined to do anything about given the low importance but mentioned here in case anyone notices, as they do have different approaches. I have also made a slight change in how the calibre document is stripped of its html tags before counting which will bump up the word counts slightly (previously sentences would get joined) so don't be surprised to see a slightly different word count than any you compute currently.

Let me know if any issues, if there are no problems I will release it in a few days...

[theducks]

When I first went to customize the PI, the box shown was smooshed down (previous PI version sized?).

If you sample, you might want to skip the first N% since that may have the books 'boilerplate' sections .

BTW the Calibre Job wheel still says 'Count Page/Word Statistics'

6.75 min to do all 3 stats on 7 titles on a P4 under XP

[kiwidude]

Thx for giving it a whirl. I haven't renamed anything in terms of jobs or menu options as you noticed, it is getting harder and harder to come up with meaningful names. Plus given that readability is based around word statistics I can live with the current names unless someone dazzles me with better ones. I get tired of doing all the thinking sometimes

So what was your conclusion on that timing - was it so slow that it had you gnashing your beak? I agree on the boilerplate comment, but it is just one example of the difficulty of blind sampling to make sure your sample is representative. One of the reasons why I wussed out for now and let it compute on the whole book - heck you only do it once

[theducks]

Originally Posted by kiwidude

Thx for giving it a whirl. I haven't renamed anything in terms of jobs or menu options as you noticed, it is getting harder and harder to come up with meaningful names. Plus given that readability is based around word statistics I can live with the current names unless someone dazzles me with better ones. I get tired of doing all the thinking sometimes

So what was your conclusion on that timing - was it so slow that it had you gnashing your beak? I agree on the boilerplate comment, but it is just one example of the difficulty of blind sampling to make sure your sample is representative. One of the reasons why I wussed out for now and let it compute on the whole book - heck you only do it once

I am used to slow on my systems (that is why I have more than 1 and a KVM to switch them all)

"Scoring Books"

BTW If you were wussing out, you would not be writing these PI

Right now, I am having fun looking at the scores

BTW: I keep forgetting to look at the Logs... Is there a PI to look at the Last Log ?

[kiwidude]

Hmmm, don't get me started thinking about logs. There are a number of plugins that could do with such a feature - Quality Check particularly screams for it. It would be handy to have a modeless window allowing you to review them on.

But to answer your question, no they get ditched.

And yes if I had my time again I wouldn't have called the plugin "Count Pages" given it now does so much more than that. At least keeping Count Pages in the name of the job it ties into the plugin name.

[DoctorOhh]

Originally Posted by kiwidude

Here is a fairly significantly changed version of the plugin that I would appreciate some feedback on. As per the request above it now includes the option to generate readability statistics, be it Flesch Reading Ease, Flesch-Kincaid Grade Level or Gunning Fog Index.

I tested this on my 9500 book epub library. It took almost 6 or 7 hours to reach 100% but never gave me the screen to write the data. I even let it go for another 7 or 8 hours. The cause was the fact I included (without thinking) all of my ePubs, one of which was the 1913 edition of Webster's Dictionary. To it's credit it did not error and I'm guessing it would have finished someday. Obviously in hindsight I including a dictionary in the Flesch-Kincaid Grade Level calculation is just plain dumb.

On a side note the Flesch-Kincaid Grade Level number was interesting, but there are so many erroneous books listed from -0.4 to 4 (no doubt properly calculated) it makes it virtually useless to me. Still it is interesting and the test was a plus for me because I now have the ADE page counts listed to match my Sony eReaders.

Thanks for listening to folks and expanding the features of many of your plugins.

[kiwidude]

Thanks for the feedback Walt. Sounds like you have lots of books by Dr Seuss

[odinokij]

Count Pages plugin for calibre fails me for comic books in format cbr and cbz. It estimates about 1/6 of the actual pages.

These formats are .rar (cbr) or .zip (cbz) files with the jpgs of the scanned pages. It should be easy to know how many pages (jpgs) has the compressed file.

The versions I use are:
- Calibre 0.8.59 for windows
- Count Pages plugin 1.5.0


Thanks a lot.

[kiwidude]

@odinokij - welcome to MobileRead.

At the moment this plugin only has one special case handling for PDF formats - in all other cases it will attempt to convert to epub and then estimate counts based on that. I can appreciate how for CBR or CBZ formats another special exemption could be made using a rule like you mention to just count the image files. I don't have any CBR/CBZ myself and no-one else has mentioned it which is why it hasn't happened as yet.

I guess the only other complication around that which comes to mind is what would happen if you had configured other statistics like counting words or the readability. I think in this case those statistics should be skipped completely and only counting pages should be possible on CBR/CBZ.

I will take a look as part of the next release.

[odinokij]

Thanks for your work.