[GUI Plugin] Count Pages

[kjdavies]

Originally Posted by kjdavies

I've also noticed that queuing the files takes many times longer (not measured, but observed) than the page count. If it could read in place it would finish almost immediately. This obviously might not be true of other formats such as PDF and EPUB.

It occurs to me that at least part of the reason this takes so long -- and why I can't reboot my PC right now -- is that I have a large copy job running from one USB drive to another, while my calibre library lives on a third drive.

Page counting would still be faster if it used the files in place rather than copying them while queuing, but the gain would not be as much as I thought when I first realized what was happening.

[BetterRed]

Originally Posted by kjdavies

It occurs to me that at least part of the reason this takes so long -- and why I can't reboot my PC right now -- is that I have a large copy job running from one USB drive to another, while my calibre library lives on a third drive.

Page counting would still be faster if it used the files in place rather than copying them while queuing, but the gain would not be as much as I thought when I first realized what was happening.

@kjdavies: Plugin zip files are the source code, so you could change the plugin to do the in-line count for CBZ files.

BR

[kjdavies]

Originally Posted by davidfor

The plugin shouldn't be copying the files. What it needs to do is unpack the files so that the contents can be accessed. As BR said, the temp files will be cleaned up, but probably not until the complete count job has been done. If this is an issue, the simplest thing to do is to run the count in smaller batches. I will have a look at the CBR/CBZ count method as it should be doable without unpacking the files. At the moment, it is probably using a calibre routine to do this and that might not be as efficient for these types of files.

From my experience, the counting takes much longer than setting up the count job. But, that is probably related to the file types. I am usually running the plugin on epubs and I know it takes some time for each of these. I haven't counted enough CBR/CBZ to see what happens there. In any case, I will not be changing the plugin to handle these differently.

I'm coming at calibre mostly as a user than a programmer, so my observations are relatively simplistic...

What you describe, though, makes a fair bit of sense.

[kjdavies]

Originally Posted by BetterRed

@kjdavies: Plugin zip files are the source code, so you could change the plugin to do the in-line count for CBZ files.

BR

Ah, cool. Okay, I'll take a look. I have to think that reading the ZIP file metadata would be way more efficient than what it's doing now.

Time to brush off my Python, I suppose.

[kjdavies]

Hmm. get_cbz_page_count() appears to do what I would have expected: opens the CBZ file as a ZIP file, iterates through the files in the ZIP file and increments a page counter if the file extension looks like it's a comic book page.

And it looks like the place it's called from -- do_statistics_for_book() -- checks to see if the page count can be gotten via this function (i.e. it can short cut).

As far as I can tell, it shouldn't be unpacking anything.

When I look in do_count_statistics(), though, the first thing it does is queue the job via ParallelJob(), regardless of file type. I'll have to see what this does when I get back.

[Dongbei]

I'm trying to download the pages for a book and I'm getting this error:

calibre, version 3.30.0
WARNING: Page/word/statistics warnings: Could not analyse some statistics in 1 of 1 books, for reasons shown in details below. [book name removed] (No identifiers for selected download sources)

This is for a DRM-free Kindle book that I exported to MOBI using Calibre. I've tried it on two books and had the same error in both cases. Is there an easy fix for this?

[davidfor]

Originally Posted by Dongbei

I'm trying to download the pages for a book and I'm getting this error:

calibre, version 3.30.0
WARNING: Page/word/statistics warnings: Could not analyse some statistics in 1 of 1 books, for reasons shown in details below. [book name removed] (No identifiers for selected download sources)

This is for a DRM-free Kindle book that I exported to MOBI using Calibre. I've tried it on two books and had the same error in both cases. Is there an easy fix for this?

The error message tells you everything. The plugin can download the page count from two different sites (Goodreads and a Polish site). But, to do that, it needs to be able to identify the book on those sites. This is done by "identifiers" that are usually set when downloading the metadata. You need to set the identifiers before you can use the page count download. The first post of this thread gives some details of this.

[Dongbei]

Great! That fixed it perfectly! However, now I can't actually see the page numbers on my Kindle app for iPad or my Kindle app for Mac. How can I get them to show up? Thank you so much!!!

[davidfor]

Originally Posted by Dongbei

Great! That fixed it perfectly! However, now I can't actually see the page numbers on my Kindle app for iPad or my Kindle app for Mac. How can I get them to show up? Thank you so much!!!

There is an option in the Kindle driver for page counts, but apart from that, I have no idea.

[Dongbei]

Originally Posted by davidfor

There is an option in the Kindle driver for page counts, but apart from that, I have no idea.

Which Kindle driver are you referring to? I followed the guide on page 1 in selecting “send page counts” but it doesn’t seem to make a difference.

[davidfor]

Originally Posted by Dongbei

Which Kindle driver are you referring to? I followed the guide on page 1 in selecting “send page counts” but it doesn’t seem to make a difference.

That's pretty much all I know about it. But, that is for the Kindle ereader. I don't know anything about how the apps work.

[jhowell]

Originally Posted by Dongbei

Which Kindle driver are you referring to? I followed the guide on page 1 in selecting “send page counts” but it doesn’t seem to make a difference.

Page numbers for MOBI format are stored in a separate .apnx file associated with the book. On e-ink Kindle devices this file is uploaded by calibre along with the main book file and a cover image thumbnail. There is no way (that I know of) to associate an .apnx file with a book sideloaded to any of the Kindle apps on other platforms.

A possible work-around is to use KFX format instead of MOBI. KFX is supported by recent versions of the various Kindle apps and supports page numbers without a separate .apnx file. To create KFX format you need install the optional KFX Output plugin and Amazon's Kindle Previewer. You can configure that plugin to get the page count from a custom column that you specify when converting books to KFX format.

[Dongbei]

Originally Posted by jhowell

Page numbers for MOBI format are stored in a separate .apnx file associated with the book. On e-ink Kindle devices this file is uploaded by calibre along with the main book file and a cover image thumbnail. There is no way (that I know of) to associate an .apnx file with a book sideloaded to any of the Kindle apps on other platforms.

A possible work-around is to use KFX format instead of MOBI. KFX is supported by recent versions of the various Kindle apps and supports page numbers without a separate .apnx file. To create KFX format you need install the optional KFX Output plugin and Amazon's Kindle Previewer. You can configure that plugin to get the page count from a custom column that you specify when converting books to KFX format.

That worked perfectly! Thanks so much!

[kjdavies]

Originally Posted by kjdavies

Hmm. get_cbz_page_count() appears to do what I would have expected: opens the CBZ file as a ZIP file, iterates through the files in the ZIP file and increments a page counter if the file extension looks like it's a comic book page.

And it looks like the place it's called from -- do_statistics_for_book() -- checks to see if the page count can be gotten via this function (i.e. it can short cut).

As far as I can tell, it shouldn't be unpacking anything.

When I look in do_count_statistics(), though, the first thing it does is queue the job via ParallelJob(), regardless of file type. I'll have to see what this does when I get back.

Okay, I'm not entirely certain where it's doing the copy -- I suspect it's part of queuing the job -- but it certainly is copying the files. I see (well, saw, it was yesterday and I've since restarted calibre) log files in C:\Users\kjdavies\AppData\Local\Temp\calibre_qfrfd n that tell me they're processing CBZ files under that directory.

[BetterRed]

@kjdavies - Thanks for keeping us informed

BR