[GUI Plugin] Count Pages

[NiLuJe]

IIRC, the second method was designed with Mobi in mind, which somewhat alleviated the potential issue of really messy code . With more complex formats, it's probably less useful & accurate.

[davidfor]

Originally Posted by NiLuJe

IIRC, the second method was designed with Mobi in mind, which somewhat alleviated the potential issue of really messy code . With more complex formats, it's probably less useful & accurate.

The code is probably pre-azw3, so that might be right. I'll look at adding a format check on that bit.

[rpgmaker]

Originally Posted by davidfor

I have had a look at this, and it is working as designed. And the problem is the messy code in the book.

The APNX page calculation does the page count in two ways. Then it takes the maximum of the two.

The first way assumes a line is 70 characters long and a page has 31 lines. The plugin counts the paragraphs and text in each paragraph ignoring tags within the paragraph. This works out to 912 pages.

The other method, called "fast count", is to simply divide the total length of the HTML code by 2400. This works out to be 4937 pages. This is high because the paragraph tags have long styles and there are lots of spans with long styles wrapping only a few characters.

The plugin uses the maximum of these calculations. There is a comment in the code that the maximum is used in case it is a "really weird document" and that the first method underestimates the page count.

You mention that a previous version of the book gave a lower page count. That should be because the code in it was cleaner and the first method was probably the higher value.

I figured. Thanks for looking into it. It's the only book that has ever given me any issues with this plugin so I'm just gonna let it be.

[Iceybones]

My books have verified goodreads Ids and when I go through the goodreads plugin I can see that goodreads does have a page count listed for all my books but 9 times out of 10 the Count Pages plugin comes back "FAILED TO GET PAGE COUNT FROM WEBSITE". I can't figure out why. Any ideas? If I keep rerunning the search it will pick up a few more here and there but still very rare for it to find one.

[davidfor]

Originally Posted by Iceybones

My books have verified goodreads Ids and when I go through the goodreads plugin I can see that goodreads does have a page count listed for all my books but 9 times out of 10 the Count Pages plugin comes back "FAILED TO GET PAGE COUNT FROM WEBSITE". I can't figure out why. Any ideas? If I keep rerunning the search it will pick up a few more here and there but still very rare for it to find one.

Are there any more details? There should be more than that in the log and it hopefully includes an error message about why the books failed.

[Iceybones]

Hey Davidfor. Thanks for the quick reply. Looks like the reason is "Failed to make details query:...". I uninstalled it and reinstalled it a few times and just tried it again and it worked like a charm for the first 20 books, now it's back to not working/only successful on 1 out of every 50 or so. Perhaps I'm hitting a reCaptcha?

[JSWolf]

Just a quick note. The page count for eBooks on GoodReads is wrong. The staff refuse to accept the correct page number for eBooks and use some version pBook page numbers and that is 100% wrong.

[davidfor]

Originally Posted by Iceybones

Hey Davidfor. Thanks for the quick reply. Looks like the reason is "Failed to make details query:...". I uninstalled it and reinstalled it a few times and just tried it again and it worked like a charm for the first 20 books, now it's back to not working/only successful on 1 out of every 50 or so. Perhaps I'm hitting a reCaptcha?

Goodreads does do some speed limiting. Doing a lot of queries in a short period of time can get you put on their black list for a while. This is the most likely reason. I have reproduced it here by running two lots of 75 books one after another. The only real solution is to not do as many queries, or maybe add a delay. But, the error message is useless, so I might do some testing to see what it really is.

[davidfor]

Originally Posted by JSWolf

Just a quick note. The page count for eBooks on GoodReads is wrong. The staff refuse to accept the correct page number for eBooks and use some version pBook page numbers and that is 100% wrong.

Looking at some of my books, Goodreads has a page count for each edition it has. So, yes, the do have the page count for a paper book, but they also have counts for other paper books and ebooks.

[Iceybones]

davidfor: Ok that's understandable. Thanks. I'm having it calculate all my books right now and putting the page count into a separate column, just so I get a relative idea. Then I'll just have to do the goodreads column in small batches.

[kjdavies]

I'm running calibre 3.28 (I'll update soon) and Count Pages plugin 1.8.2 on a Windows 10, 64-bit. It appears Count Pages copies the book files to a temporary directory on the system drive. Is it possible to not do this? Can the files be examined in-place?

I'm working with a large number of biggish files (comic book collection, 100MB files aren't uncommon) and the solid state system drive isn't that big. Discovering the 70GB free space was suddenly gone was a bit of an unpleasant shock...

[BetterRed]

@kjdavies - those temporary files should have been deleted as it counted each book.

By default calibre uses the default location for temporary files (e.g. on it's Windows C:\Users\<users>\AppData\Local\Temp), and it uses it extensively, e.g. when adding and converting books. It's pretty good at cleaning up, but remnants can get left behind that normal 'system cleanup' processes take of.

You can give calibre a dedicated folder for its temporaries on another drive via the CALIBRE_TEMP_DIR environment variable, see User Manual: Environment variables

If you give calibre a dedicated location for its temporaries you'll have to make provision for any additional cleanup.

Curious - what format are the comics? I only count words and I never run Count Pages against CBZ or CBR (or audio/video formats etc). Maybe the plugin needs an Exclude Formats setting.

BR

[kjdavies]

Originally Posted by BetterRed

@kjdavies - those temporary files should have been deleted as it counted each book.

By default calibre uses the default location for temporary files (e.g. on it's Windows C:\Users\<users>\AppData\Local\Temp), and it uses it extensively, e.g. when adding and converting books. It's pretty good at cleaning up, but remnants can get left behind that normal 'system cleanup' processes take of.

You can give calibre a dedicated folder for its temporaries on another drive via the CALIBRE_TEMP_DIR environment variable, see User Manual: Environment variables

If you give calibre a dedicated location for its temporaries you'll have to make provision for any additional cleanup.

Curious - what format are the comics? I only count words and I never run Count Pages against CBZ or CBR (or audio/video formats etc). Maybe the plugin needs an Exclude Formats setting.

BR

Very informative, thank you.

In this case calibre did clean up after itself, but not until I'd discovered it filled the system drive. I closed calibre and the space was recovered.

I've set CALIBRE_TEMP_DIR as a system variable, so that should work after restarting.

I'm counting pages in (in this case) CBZ files, but I do have CBR (that I usually convert to CBZ). Page count is equal to the number of images.

Overall I'm happy with how this plugin works, and would prefer that CBZ and CBR continue to be supported. I just noticed that it ate my system drive.

I've also noticed that queuing the files takes many times longer (not measured, but observed) than the page count. If it could read in place it would finish almost immediately. This obviously might not be true of other formats such as PDF and EPUB.

[davidfor]

Originally Posted by kjdavies

In this case calibre did clean up after itself, but not until I'd discovered it filled the system drive. I closed calibre and the space was recovered.

I've set CALIBRE_TEMP_DIR as a system variable, so that should work after restarting.

I'm counting pages in (in this case) CBZ files, but I do have CBR (that I usually convert to CBZ). Page count is equal to the number of images.

Overall I'm happy with how this plugin works, and would prefer that CBZ and CBR continue to be supported. I just noticed that it ate my system drive.

I've also noticed that queuing the files takes many times longer (not measured, but observed) than the page count. If it could read in place it would finish almost immediately. This obviously might not be true of other formats such as PDF and EPUB.

The plugin shouldn't be copying the files. What it needs to do is unpack the files so that the contents can be accessed. As BR said, the temp files will be cleaned up, but probably not until the complete count job has been done. If this is an issue, the simplest thing to do is to run the count in smaller batches. I will have a look at the CBR/CBZ count method as it should be doable without unpacking the files. At the moment, it is probably using a calibre routine to do this and that might not be as efficient for these types of files.

From my experience, the counting takes much longer than setting up the count job. But, that is probably related to the file types. I am usually running the plugin on epubs and I know it takes some time for each of these. I haven't counted enough CBR/CBZ to see what happens there. In any case, I will not be changing the plugin to handle these differently.

[BetterRed]

@kjdavies - Whilst doing an in-place file count for CBZ and CBR would appear to be relatively simple, it may not be in practice. As you've probably noticed the plugin's author hasn't been active for some time; a couple of active plugin devs (jackie_w and davidfor) have kept the plugin compatible with core developments and added a few enhancements since.

I just did a test on a couple of very large CBZ's (50-800MB) counting pages, it took a while. My impression is, it copied the CBZ to temp, unpacked the copy to a folder, then did a file count on the folder. My file manager displays file counts for similar (same) zips more-or-less instantly.

@davidfor - we sort of x-posted.

BR