[GUI Plugin] Count Pages

[BeccaPrice]

sorry, Kiwidude - I'd closed down the error message when I was typing and couldn't remember the exact wording... something like no epub or mobi format was available or something like that.

I just went back to re-check the exact wording, and it worked just fine this time, so who knows?

I've got a lot of "one page" books in my library that I need to reconvert and run the page count plugin for, and see whether just running it over again fixes the problem. Or, I can say the hell with it and let it be. Page count for me is a nice-to-have, but not really necessary.

[kiwidude]

You should only see that message if you choose the "Estimate from content" option and you don't have an epub or mobi format. Or if you choose the "Download from Goodreads" option with a word count column (and no epub/mobi). If there is some other combination that you are seeing it on, that is a bug that should be fixed, so if you see it again let me know and the details of what formats and which option you chose.

I'll take a look sometime at the mobi page count algorithm stuff, maybe next weekend. I've become a word-count fan myself, took some getting used to the different scale but it offers a better consistent indication of book size. I still use page count though, just so if I take a peek trying to figure out what to read next on my Kindle I have some idea of how big it is

[BeccaPrice]

some books I'm seeing the message that reads "no mobi or epub file found, or no file on Goodreads" (or words to that effect) when I click the "get from Goodreads" option.

I've found that, for most of these books, the magic ritual is to convert mobi-to-mobi, and then run the plugin a second time. Then - mostly - I'll get a page count from Goodreads.

I can't believe that The Outlaws by Griffin has 652 pages, and no Goodreads id to get a more accurate page count.

Interestingly, for Soul Music, the estiimate page count came up with 400+ pages, and when I got the data from Goodreads, the count was 288, which is far more reasonable. I've noticed this for other books, that the estimate is way higher than the actual page count. Do I have the plugin configured wrong somehow?

[KoGs]

Is there no way to make the page count correspond to the number of kobo pages?

Also, none of my books are getting any data from GoodReads. How do I disable word count?

[kiwidude]

Originally Posted by KoGs

Is there no way to make the page count correspond to the number of kobo pages?

This was asked for fairly recently and my answer remains the same. Someone needs to reverse engineer exactly what "formula" the Kobo uses to determine a number of pages. I am not a mind-reader, nor do I own a Kobo

Also, none of my books are getting any data from GoodReads.

Read the first post in the thread very carefully. You can only get a page count if you have a gooreads id for the book, and that particular edition has a page count specified for it on the web page (just click on the Goodreads link in the book details panel to go direct to the linked web page). If you don't see a page count, try switching editions (done with the Goodreads Sync plugin).

How do I disable word count?

Just blank out the custom column name you configured for it in the configuration for this plugin.

[kiwidude]

Originally Posted by BeccaPrice

some books I'm seeing the message that reads "no mobi or epub file found, or no file on Goodreads" (or words to that effect) when I click the "get from Goodreads" option.

I've found that, for most of these books, the magic ritual is to convert mobi-to-mobi, and then run the plugin a second time. Then - mostly - I'll get a page count from Goodreads.

I'm afraid you again haven't given me much to go on. In the error dialog that appears, clicking on the "Show details" will show exactly why each book was rejected. I need specific details, exactly what options you have configured for this plugin (a screenshot of the config page will do or the plugin json configuration file contents). What options you chose. What formats the book had at the time. Any and all log/error output.

Also make sure you are running the very latest version - by any chance did you install the beta which did have a bug in this area? Make sure you install the plugin again (ignore that the version numbers will show the same, I didn't change them for final release) and restart calibre.

I can't believe that The Outlaws by Griffin has 652 pages, and no Goodreads id to get a more accurate page count.

If you don't have a goodreads id for the book, the plugin has no ability to download a page count so you will only be able to get the estimated count option. I'm not quite sure what your point is here. If the book has an edition on goodreads with a page count, then link to it and the lugin can download the count for it.

Interestingly, for Soul Music, the estiimate page count came up with 400+ pages, and when I got the data from Goodreads, the count was 288, which is far more reasonable. I've noticed this for other books, that the estimate is way higher than the actual page count. Do I have the plugin configured wrong somehow?

This has been discussed numerous times throughout the thread. There is no "one number" which indicates the page count for a book. Even if you use this plugin's ability to download from goodreads, you will see for most books that there are a large number of editions, and most will vary in page counts. Sometimes extremely significantly. Some would have been large print editions, differences between paperback and hardback, quite possibly data entry errors, you name it. So there is absolutely no point in trying to extrapolate between the differences. In fact when switching editions it is difficult at times (because the printed editions vary so widely) to decide which one to pick.

I chose to create a new page count column that I populate alonside the estimated count as I transition to using Goodreads based. For the most part the estimates (based on ePub for me) are in the right ballpark. For an at a glance idea as to whether one book is significantly larger than another, they do their job.

However word count is a better indication of the size of the book and personally is the only column I have visible, though as I say I populate page count for Kindle usage. However even word count just like page count can get "fooled", by books containing previes of upcoming books, special editions containing extras, interviews etc.

[KoGs]

Alright, I'll see what I can do.

One, long ago, Marsh had tried to train her
in the fine art of using bronze to read
Allomantic pulses. She hadn't realized at the
time just how large a task he'd begun.
        When an Allomancer burned a metal,
they gave off an invisible, drumlike beat that
only another Allomancer burning bronze
could sense. The rhythm of these pulses--
how quickly the beats came, the way they
"sounded"--told exactly what metal was
being burned.
        It took practice, and was difficult, but
Vin was getting better at reading the pulses.
She focused. Breeze was burning brass--
the internal, mental Pushing metal. And . . .
        Vin focused harder. She could feel a
pattern washing over her, a double dum-dum
beat with each pulse. They felt oriented to 
her right. The pulses were washing against 
something else, something that was sucking 
them in.

This is the amount of text that can be displayed on one Kobo screen on the medium font. Each line is also justified.

        Elend. Breeze was focused on Elend.
Not surprising, considering the current 
discussion. Breeze was always Pushing on
the people he interacted with.
        Satisfied, Vin sat back. But then she
paused. Marsh implied there was much
more to bronze than many people thought. I
wonder. . ..
          She squeezed her eyes shut--
ignoring the fact that any of the others who
saw her would think her actions strange--
and focused again on the Allomantic pulses.
She flared the bronze, concentrating so hard
she felt she'd give herself a headache. There
was a. . .vibration to the pulses. But what 
that could mean, she wasn't certain.
         Focus! she told herself. However, the
pulses stubbornly refused to yield any further
information.
         Fine, she thought. I'll cheat. She
turned off her tin--she almost always had it

These 2 screens are considered page 231 on the Kobo. Though even 2 screens is not exact since sometimes it's 2 screens per page, sometimes it's 3.

her right. The pulses were washing against something else, 
something that was sucking them in.

        Elend. Breeze was focused on Elend. Not surprising, 
considering the current discussion. Breeze was always Pushing 
on the people he interacted with.
        Satisfied, Vin sat back. But then she paused. Marsh 
implied there was much more to bronze than many people 
thought. I wonder. . ..
          She squeezed her eyes shut--ignoring the fact that any of 
the others who saw her would think her actions strange--and 
focused again on the Allomantic pulses. She flared the bronze, 
concentrating so hard she felt she'd give herself a headache. 
There was a. . .vibration to the pulses. But what that could mean, 
she wasn't certain.
         Focus! she told herself. However, the pulses stubbornly 
refused to yield any further information.
         Fine, she thought. I'll cheat. She turned off her tin--she 
almost always had it on a little bit--then reached inside and
burned the fourteenth metal.  Duralumin.
          The Allomantic pulses became so loud. . .so
powerful. . .she swore she could feel their vibrations shaking her
apart. They pounded like beats from a massive drum set right
beside her. But she got something from them.
         Anxiety, nervousness, worry, insecurity, anxiety,
nervousness, worry--
         It was gone, her bronze expended in one massive flar of
power. Vin opened her eyes; no one in the room was looking at
her except OreSeur.
         She felt drained.  The headache she'd predicted before 
now came in full force, thudding inside her head like the tiny
brother of the drum she'd now banished. However, she held to

This is the text that fits on one Kobo screen choosing the small font. Every 1-2 screens constitutes a page change.

        Vin focused harder. She
could feel a pattern washing
over her, a double dum-dum
beat with each pulse. They felt
oriented to her right. The pulses
were washing against something
else, something that was
sucking them in.
         Elend. Breeze was
focused on Elend. Not
surprising, considering the 
current discussion. Breeze was
always Pushing on the people
he interacted with.
         Satisfied, Vin sat back.
But then she paused.  Marsh

And that is the screen of the largest size. It takes between 5 and 6 of these screens before the page changes.

[kiwidude]

@KoGs - I appreciate the effort but I'm afraid that isn't going to be sufficient. At the very best all I would be able to do based on that is come up with another "estimate", which is just going to be not overly dissimilar to what is computed already.

The APNX/page count functionality is primarily targeted at Kindle users, since it is the only way to get a page count for your own books to appear on the Kindle.

The secondary set of users is those who just want a way to within Calibre compare the relative sizes of different books. That's where I see Kobo users fitting in.

However unless someone can determine the exact formula applied to come up with the page count on the device, I don't see what value some other estimated number is going to add? Without that it isn't going to be any more "accurate" than the existing algorithm, at best it will just skew the numbers either up or down slightly. Which doesn't to me sound worth the effort.

On top of which, there is the very real difference between what constitutes a "screen" and what constitutes a "page". If I understand your post above, what you have provided are a "screen" worth of information. Now I know nothing about the Kobo, maybe it works differently to other devices, but on a Kindle at least the page count is not in any way related to the number of "screens" displayed.

What I could possibly do is expose the variables used in the calculation to allow a plugin user to tweak them. So if someone thinks the numbers are overall skewed in the wrong direction they could "fiddle" them to a different number of "characters per page" or whatever. However I see it as rearranging deck chairs on the Titanic in that it will just skew the numbers slightly up or down, and will still result in some books being significantly underestimated and others overestimated.

[KoGs]

That's understandable. The screen and page discrepancies seems to only occur when converting mobi to epub (the kobo format). Books that are originally in the epub format do not seem to have this problem as one screen (regardless of font size) corresponds with page count quite accurately.

[kiwidude]

@Kogs - a book having originally been in mobi and then converted to ePub vs a book that was always in ePub should not make a difference to the count? If you have a specific example where this is not the case, feel free to PM me some links to take a look.

The one possible explanation for what you are seeing is if you had your count pages preferences set to "Prefer mobi" rather than "Prefer ePub". So if after doing a mobi -> ePub conversion you ran the Count pages plugin, you would still be seeing a count derived from the mobi, not from the ePub.

And (as I have said previously) the algorithm within calibre for computing mobi page counts (that this plugin calls) does not work very consistently because it gets fooled by books using <blockquote> or <div> tags for paragraphs. Converting mobi->ePub alone will not change that. However if you have "Prefer ePub" set, then this plugin will not use the calibre page count code and will instead use my own ePub page counting logic which caters for those conditions.

For the next version of this plugin I will add a modified version of the calibre logic to cope with blockquote/div.I will also drop the "Prefer epub/mobi" setting, which was a partial workaround required for consistency with the APNX generation (i.e. what actually ends up on your Kindle). That is no longer needed because a user can now specify a custom column as the page count source, rather than using calibre's mobi page count which has the issues above. So at least that way a plugin user will get consistent counts regardless of which of the two formats was computed against.

The final piece of the puzzle for some users is the standalone APNX file generator plugin, for users not using Send to Device. SauliusP posted in the Development forum his intent to update that to also support a custom column. In some ways it might make sense instead (or also) to simply add a menu option within this one to offer the same features, removing the need for an extra plugin. As even using a custom column as the source, you still need to populate it, which is what this plugin does.

[Zetmolm]

Kiwidude, is the word count for other formats than epub and mobi still on your agenda?
No pressure intended, but I would consider that a very welcome addition to a great plugin, so I'd like to know if this is something I can look forward to.

[kiwidude]

@Zetmolm - IIRC you wanted it for txt files, was that right? To be brutally honest it had completely dropped off the radar, since I only use ePub and Mobi myself. I can't remember what I said about this last time, I shall have to look back through the thread.

[KoGs]

Oh sorry, I meant on the Kobo itself, not the calibre program.

Books I download from kobobooks are in the epub format and when I add them to my kobo, each screen counts as a page. But when I download books from Amazon which are in the mobi format and convert to epub using Calibre, this is where the discrepancy between screens and pages occur.

I was just mentioning fyi because you mentioned you didn't know anything about Kobo; I'm not actually complaining about the Calibre program. Obviously I haven't been able to load a mobi book onto my Kobo to see if it's the conversion itself that changes this or it's inherent in the mobi book.

[Zetmolm]

Originally Posted by kiwidude

@Zetmolm - IIRC you wanted it for txt files, was that right? To be brutally honest it had completely dropped off the radar, since I only use ePub and Mobi myself. I can't remember what I said about this last time, I shall have to look back through the thread.

I would want it for fb2 rather than txt. I have quite a lot of books in fb2 format (the de facto standard format in Russia - not that I'm from there, though). As I understood it you were considering something like an on-the-fly conversion to epub (woithout actually saving the epub) to do the word count. Would be great if you can implement something.

[kiwidude]

@Zetmolm - of course, it was you that requested fb2 and Vaessa that requested txt... I just looked back at what I wrote about this previously here. It seems I had ideas back then which I subsequently forgot about

I can't see a major downside to what I proposed (other than it being possibly slightly slower for mobi only users) as it looks similar to what I just stated above. Basically completely drop using the APNX algorithm for mobi books. Instead always convert to ePub format (if not already ePub) and use my existing algorithm for that. Which format it chooses to convert to ePub comes from your preferred input format order. So it should additionally cope with fb2, txt, lit or whatever - heck even pdf if you were silly enough to try it and that was your only or preferred format (we all know how slow and badly pdf -> epub conversions work out).

I shall take a look hopefully this weekend.