[GUI Plugin] Count Pages

[del.libro]

I just migrated my laptop to a new disk. My home folder (I'm on Linux) was copied from the old to the new -- so the calibre folder is the same. Right after I did this, there was an update to Calibre (to version 6.12).

After this, I noticed the plugin was no longer storing the readability states. Books that had the stats already still display it. I can manually add numbers to the columns and they will appear in the book details pane.

Log below. Apparently the "copy_reg" module is missing, or has been removed, or something.

For book 'Purity: A Novel - Jonathan Franzen', using format azw3
Worker Launch took: 0.00 seconds
Job: 0 Count Page/Word Statistics finished
Count Page/Word Statistics
do_count_statistics - book_path=/tmp/calibre_6.12.0_tmp_zs3p3yjk/zrmyv7w6_count_pages/15454.azw3, pages_algorithm=3, page_count_mode=Estimate, statistics_to_run=['PageCount', 'WordCount', 'FleschReading', 'FleschGrade', 'GunningFog'], custom_chars_per_page=1880, icu_wordcount=True
do_count_statistics - job started for file book_path=/tmp/calibre_6.12.0_tmp_zs3p3yjk/zrmyv7w6_count_pages/15454.azw3
-------------------------------
Logfile for book ID 15454 (Purity: A Novel - Jonathan Franzen)
Method of counting _page_count_mode=Estimate _download_sources=[]
results= {'PageCount': 733, 'WordCount': 243505}
Found 733 pages
Found 243505 words
15454
Traceback (most recent call last):
File "calibre_plugins.count_pages.jobs", line 211, in do_statistics_for_book
iterator, text_analysis = get_text_analysis(iterator, book_path, nltk_pickle)
File "calibre_plugins.count_pages.statistics", line 324, in get_text_analysis
t = TextAnalyzer(nltk_pickle)
File "calibre_plugins.count_pages.nltk_lite.textanalyze r", line 22, in __init__
self.eng_tokenizer = pickle.loads(eng_tokenizer_pickle)
ModuleNotFoundError: No module named 'copy_reg\r'
do_statistics_for_book: /tmp/calibre_6.12.0_tmp_zs3p3yjk/zrmyv7w6_count_pages/15454.azw3 3 Estimate [] ['PageCount', 'WordCount', 'FleschReading', 'FleschGrade', 'GunningFog'] 1880 True
Page count: 733
Word count using icu_wordcount - trying to count_words
Word count - used count_words: 243505
Word count: 243505

[theducks]

did you also copy the configuration folder? That has the settings thet the PI uses to know where to put stuff

[del.libro]

Originally Posted by theducks

did you also copy the configuration folder? That has the settings thet the PI uses to know where to put stuff

Yes, as it is in my home folder, it was copied as well. In any event, a missing config wouldn't explain the error being thrown. Plus, CP's config was the first thing I checked, and it was unchanged. Page and word count are properly stored; there's only an issue with the readability stats.

I also did the obvious experiment of creating a new column and telling CP to use that, and I got the same behavior and error.

If I disable the readability stats (by setting the columns for storing them nothing), no error is thrown, so it seems fairly clear the that missing/mis-referenced copy_reg is the problem.

[RelUnrelated]

I'm encountering the same problem since the most recent update to the plugin.

ModuleNotFoundError: No module named 'copy_reg\r'

Count Pages (1.12.1)
Calibre (6.12)
Ubuntu 22.10

Originally Posted by del.libro

After this, I noticed the plugin was no longer storing the readability states. Books that had the stats already still display it. I can manually add numbers to the columns and they will appear in the book details pane.

Log below. Apparently the "copy_reg" module is missing, or has been removed, or something.

[arcchancellor]

Since the last update from 02-03-2023 the readability statistics #fleshease (Flesh-Grad) no longer works and leaves an empty field. Word/page count work as always.

Calibre is 6.12.
OS is Ubuntu 22.04.

[theducks]

Originally Posted by arcchancellor

Since the last update from 02-03-2023 the readability statistics #fleshease (Flesh-Grad) no longer works and leaves an empty field. Word/page count work as always.

Calibre is 6.12.
OS is Ubuntu 22.04.

Same on Win 10 x64

[dewd]

Originally Posted by arcchancellor

Since the last update from 02-03-2023 the readability statistics #fleshease (Flesh-Grad) no longer works and leaves an empty field. Word/page count work as always.

Calibre is 6.12.
OS is Ubuntu 22.04.

Same here on Mac OS 12.6.1, Calibre 6.9.0 and Count Pages 1.12.1, using Gunning Fox Index.

[kiwidude]

I am aware of this Gunning Fog issue, someone raised it on Github too. As I posted in another thread recently I’m not able to do much work on plugins as much as I might like right at the moment with other real world priorities of relocating across the planet. I’m still at least 6 weeks away from my computer allegedly arriving and clearing customs assuming it is in one piece, and just got keys to my new house this week. So my life is chaos at the moment.

I have however tried to make sure that any breaking issues get fixed when I can and this one would fall into that category. It looks like it is a Python compatibility issue from v2 to v3 related to the calibre upgrade. At best I may get an hour or two to look at it this weekend if lucky, no promises at this point however.

[del.libro]

Originally Posted by kiwidude

I am aware of this Gunning Fog issue, someone raised it on Github too. As I posted in another thread recently I’m not able to do much work on plugins as much as I might like right at the moment with other real world priorities of relocating across the planet. I’m still at least 6 weeks away from my computer allegedly arriving and clearing customs assuming it is in one piece, and just got keys to my new house this week. So my life is chaos at the moment.

I have however tried to make sure that any breaking issues get fixed when I can and this one would fall into that category. It looks like it is a Python compatibility issue from v2 to v3 related to the calibre upgrade. At best I may get an hour or two to look at it this weekend if lucky, no promises at this point however.

If it helps, the only place copy_reg (which is the P2 version of P3's copyreg) is in nltk/english.pickle. So, it seems you need to upgrade to a recent version of NLTK. I looked at NLTK on Github, but don't see the 'lite' version. So, my guess is this won't be a quick fix, but I haven't done any serious development in almost a decade, so this is only a barely educated guess.

[kiwidude]

Originally Posted by del.libro

If it helps, the only place copy_reg (which is the P2 version of P3's copyreg) is in nltk/english.pickle. So, it seems you need to upgrade to a recent version of NLTK. I looked at NLTK on Github, but don't see the 'lite' version. So, my guess is this won't be a quick fix, but I haven't done any serious development in almost a decade, so this is only a barely educated guess.

Thanks for taking a look. My quick look myself in response found that NLTK Lite became NLTK at version 2.0 and along the way Python 3 migrated. I cannot remember what other dependencies etc there were in that plugin (10 years or so I wrote it) to know if that is a straightforward upgrade or if there are other libraries in there depending on it which will make this all a big thing or not.

All will no doubt be revealed when I find time to peek at the code again…

[kiwidude]

Ok quick update on this. That couple of hours I thought I might have didn't happen. I've taken a small peek at it now, and it is definitely not looking like a 5 minute job for me to fix which is all I have to spare right now.

The way I see it there are three options:
1. Try to make the existing NLTK Lite code Python 3 compatible - or at least the subset of it bundled with this plugin. There is only a handful of files in there, so it might not be too hard a job to do. Contributions welcome
2. Alternatively I put the whole latest NLTK library (which is huge) as part of the plugin. Not keen on doing that, am sure Kovid wouldn't want me too either.
3. Another alternative is to grab just the bits out of the latest NLTK I need. Which would require much analysis and investigation.

Then on top of that for options 2 and 3 I may also have to replace the actual analysis code, if there are compatibility issues between the old NLTK Lite and NLTK current. I found what looks to be an interesting wrapper library here for instance:
https://pypi.org/project/py-readability-metrics/

TLDR - not going to happen in the next month or so at least until my computer arrives and is operational.

[Nutzer0815]

I'm currently having a problem. I installed the Count Pages extension and activated it in FanFicFare. So far no problems. After loading a text with FanFicFare, Count Pages was started automatically and all 5 values were calculated. Currently only pages and words are calculated. Fleshkincaid, Gunnigfog, and Readability remain blank lately. All available updates are installed.

[JSWolf]

Originally Posted by Nutzer0815

I'm currently having a problem. I installed the Count Pages extension and activated it in FanFicFare. So far no problems. After loading a text with FanFicFare, Count Pages was started automatically and all 5 values were calculated. Currently only pages and words are calculated. Fleshkincaid, Gunnigfog, and Readability remain blank lately. All available updates are installed.

If you actually read the last few pages of this thread, you'd see that the analytics are not current working and will take some time to be fixed.

[Nutzer0815]

Thanks for the hint.
Unfortunately, my English is bad. I would be grateful for a link to get started.

[theducks]

Originally Posted by Nutzer0815

Thanks for the hint.
Unfortunately, my English is bad. I would be grateful for a link to get started.

Known problem.
Author moving so fix delayed.