[GUI Plugin] Count Pages

[kiwidude]

@GeckoFriend - afraid that is the way it is, the book effectively has to be viewable in calibre's ebook viewer which DRM'd books aren't.

If I was ever to change the plugin to support downloading page counts from websites like Goodreads then it wouldn't be an issue and this plugin would work with any book, DRM'd or not.

However I am not going to look at supporting that until user_none changes the APNX plugin to support retrieval from a custom column - an idea he appeared initially receptive to but seems to have put on the backburner since. Otherwise the downloaded page count retrieved would not be able to be displayed on the Kindle, which as a Kindle owner is a motivating factor for me spending significant hours doing the development.

[GeckoFriend]

Thanks for the reply and information (and of couse for writing the plugin in the first place!).

I guess I now have an incentive to read Apprentice Al and figure it out.

[MyNameIsMrBurns]

I find this plugin incredibly helpful, but I'm curious -- is there a possibility of adding whichever algorithm the Nook uses as an option? Comparing one book, my Nook gives a pagecount of 1038, the APNX algorithm gives 1490, and the Adobe algorithm gives 2300. I'm not sure how the Nook calculates its pagecount.

[kiwidude]

@MyNameIsMrBurns - I'm afraid that unless someone can figure out what their page count algorithm is (and me not being a Nook owner means I have no chance) then that would be a no

[kiwidude]

This beta adds an option to download the actual page count for a book edition from the Goodreads website, rather than only being able to estimate it based on ePub/Mobi content.

Kovid accepted a change I made last night to calibre which will allow the APNX file produced when sending a book to your device to pull the page number from a custom column rather than always calculating it.

So the combination of the two now means it is possible to automate the retrieval of page count for a book with this plugin, and have that displayed on your Kindle with the next calibre release of 0.8.40.

Note that I have made an intentional limitation with this functionality - you can only download the page count for a book that has a goodreads identifier. How do you get a goodreads identifier? Either by using the Goodreads metadata download plugin, or by using the Goodreads Sync plugin (or manually typing it into the ids field on Edit Metadata). If the edition you are linked to has no page count, then you will need to link to a different edition.

Why this limitation? Without it, the plugin would have to search the website to find a match based on ISBN/Title/Author just like metadata download. Which means a whole bunch of replicated code I can't be bothered with, and a lack of control over which edition gets selected (it could match the wrong book), whereas at least with single metadata download the user has a UI to choose. This simplified the implementation, which means I get to spend more of my weekend relaxing

There is no intention of adding similar Amazon website support at this point, since historically Goodreads data has been a superset of Amazon anyways. There are also the complications of multiple regions, possible current website changes etc.

I agonised over how best to integrate this download option into the UI, and a separate menu option was my decision. You can configure the default action for just clicking on the button (the plugin had no menu previously) so users who don't want the new functionality will find nothing needs to change. I contemplated whether to "fallback" to using an estimation for page count when none found via Goodreads, but decided against it - at least this way if you get no page count you more obviously know the cause is a bad goodreads id linked and can either rectify it or compute it.

Any feedback appreciated before I officially release it.

[Noughty]

I'm glad to see this new feature. Goodreads is my main source for all metadata so new types of data are always welcome.

Testing now.

I tried downloading for 18 books all with goodreads ID. I got warning that 12 of them couldn't be scanned (since there is no epub/mobi) or have no goodreads ID. I chose to download page count so it shouldn't scan it. It also shouldn't say they don't have ID, since I choose only the ones with id.

Another test with 20 other books. This time 8 can't be scanned or downloaded. And other 12 fail to download.

3rd try. It always gives the warning if there isn't an epub/mobi version about failure to scan or about missing ID.

4th try. Only one book with mobi format. Of course no warning message since there is mobi/epub.

This is how log looks:

Logfile for book ID 1412 (Dreaming - EPUB)
FAILED TO GET PAGE COUNT FROM GOODREADS

Failed to initialize plugin: u'C:\\Users\\PC\\AppData\\Roaming\\calibre\\plugin s\\Goodreads Metadata.zip'
Failed to initialize plugin: u'C:\\Users\\PC\\AppData\\Roaming\\calibre\\plugin s\\Goodreads Covers.zip'
Goodreads book url: u'http://www.goodreads.com/book/show/[book id]'
Traceback (most recent call last):
File "calibre_plugins.count_pages.jobs", line 115, in do_statistics_for_book
File "genericpath.py", line 18, in exists
TypeError: coercing to Unicode: need string or buffer, NoneType found

I checked, ID is good, I opened using view book. Page number is also written.

5th try with a pdf format. A warning as always with no epub/mobi. And nothing happens. It ignores it. So if it doesn't have an epub/mobi format it ignores the book and doesn't even try to download.

Am I doing something wrong or is there a bug or something?

I'm also interested and probably already know the answer, but better to check. Does it check other editions? Since often page count is in one but not another edition. Especially ebook versions often have page number left out.

[kiwidude]

@Noughty - currently the rules haven't changed about formats supported by this plugin - i.e. you must have a epub or mobi format for a book. The reason for that is because if you have word count enabled the plugin still needs to parse the contents of the book, so the same restrictions apply of needing an epub or mobi.

Now there is a case where a user does not have a word count column and wants to download a page count, irrespective of book format. I just haven't bothered trying to code for it yet . So your example of having a PDF file for instance.

So in terms of all those "warning" messages you see about "does not have an epub/mobi or a goodreads id) the emphasis should be on the "or". I will think about how to reword the message. Basically the book either doesn't have a goodreads id, or or doesnt have a valid format to scan for word count.

The final comment you make with the error stack, that looks interesting. Can you give me the goodreads id for that book? Also, what do you mean by "Page number is also written", I thought from the error it failed?

You also should delete two deprecated plugins that are polluting your error messages - "Goodreads Metadata.zip" and "Goodreads Covers.zip". Delete those specific zip files from your calibre plugins directory. Those were the predecessors to the "Goodreads.zip" plugin which replaced them and should be removed as they are not valid for modern calibre.

Oh yeah forgot to add - the plugin does *not* scan editions, it scans only the page for the book you have linked. You have to decide which edition you want if you want a different one.

Thanks for the feedback, appreciate it and from any others trying it.

[Noughty]

I have a column for page numbers.

And yes I understood that the warning was with "or" and the meaning of it. I just didn't knew that it still had to be epub/mobi format. That explains it always giving error with pdf.

As for page number written - I meant on goodreads. As sometimes there is no page number and I thought that might be it.

Book ID doesn't matter - I get the same error with any book. I will try deleting the bad zip files and try again. Maybe error is because of that.

Testing again.

I think that I created column for pages long time before and choose floating numbers instead of integrals. Creating new column with integrals and see how it goes. Before this plug in worked with this column (counting from file contents).

Test gives the same results. Deleted the old zip files, created new column. The same for every book. What other settings might interfere with this?

[DuskyRose]

Originally Posted by Noughty

I have a column for page numbers.

And yes I understood that the warning was with "or" and the meaning of it. I just didn't knew that it still had to be epub/mobi format. That explains it always giving error with pdf.
?

I'm getting the exact same error, and my formats are both epub and mobi.

So it's not picking up that I have either one.

On the "details" on the error report I just get the single word of the title of the book. It's the same on all the epubs and mobi's I've tried.

[kiwidude]

Hmmm, I don't understand that, it is working without a problem for me.

If you go to the Jobs page in calibre, post what you see from the Job details window.

[tilia]

If I set the plugin to estimate from content, choose only books with epub/mobi, I get the message that no epub/mobi can be found. No job even started.

Estimate from Goodreads gives me pagenumbers from Goodreads.

[theducks]

Originally Posted by tilia

If I set the plugin to estimate from content, choose only books with epub/mobi, I get the message that no epub/mobi can be found. No job even started.

Estimate from Goodreads gives me pagenumbers from Goodreads.

From 'Content' has to open each book and count. The PI can only do that with Mobi and Epub.
Goodreads is simply a database lookup (someone else did the counting).

[kiwidude]

@theducks - actually tillia is right, there is a bug with that beta for someone trying the old "estimate" approach (I had only tried the Download option after my changes). I think that is perhaps what DuskyRose was doing as well.

I will add a new beta up shortly, I'm adding support for the case I discussed with noughty above of downloading page counts where formats are not epub/mobi.

[kiwidude]

Ok, new beta version attached to the post above. As I just mentioned, this fixes me breaking the existing "Estimation" logic, and also supports downloading a page count for a book regardless of what formats it has (if any at all), provided you haven't also enabled word counts.

@noughty/DuskyRose/tilia - thx for the invaluable feedback, please let me know how you get on with this version.

[Noughty]

Great. It's finally working for me. 7 out of 10 books had page numbers downloaded. (other didn't have that information)

From my little test I can see that it also ignores if number count is 0, which is nice. No book can have 0 pages.

Counting from contents works too.