[GUI Plugin] Count Pages

[valdor27]

Originally Posted by JimmXinu

I cheat and include other info in the title using metadata plugins. See attached.

I actually managed to do something nice by adding the pages count and the Gunning fog next to the author name and it is working like a charm.

Thank you very much !

[Dirk-71]

Since Calibre Version 2.76 (Mac Version) Count Pages is not working anymore when using ADE algorithm. Windows Version of Calibre 2.76 is working perfectly.

The result is always zero.

[davidfor]

Originally Posted by Dirk-71

Since Calibre Version 2.76 (Mac Version) Count Pages is not working anymore when using ADE algorithm. Windows Version of Calibre 2.76 is working perfectly.

I don't know of anything that has changed that would affect the Mac version. I don't have a Mac, so I cannot test this. But, the first thing to do is check the configuration is still correct. Then, after running a count pages job, look at the job details. You do this by clicking "Jobs" in the bottom right corner of the calibre screen to open the job list. Then select the count job and press the "Show job details" button. This should show the counts done and some other details from the job. Please post these.

[Dirk-71]

Originally Posted by davidfor

I don't know of anything that has changed that would affect the Mac version. I don't have a Mac, so I cannot test this. But, the first thing to do is check the configuration is still correct. Then, after running a count pages job, look at the job details. You do this by clicking "Jobs" in the bottom right corner of the calibre screen to open the job list. Then select the count job and press the "Show job details" button. This should show the counts done and some other details from the job. Please post these.

I checked the configuration. I uninstalled and reinstalled "Count Pages". I tried everything but it seems to be a bug. I reported to Kovid Goyal but he answered, he doesn't support third party plugins.

Here is the result on Mac Version 2.76:

Count Page/Word Statistics
-------------------------------
Logfile for book ID 2 (Die Stille vor dem Tod)
2
Page count: 0.0

And here is the result on Windows Version 2.76:

Count Page/Word Statistics
-------------------------------
Logfile for book ID 2 (Die Stille vor dem Tod)
Found 42 pages
2
C:\Program Files\Calibre2\pylib.zip\dateutil\parser.py:339: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
Page count: 42.0

Here is the result on Mac Version 2.75.1:

Count Page/Word Statistics
-------------------------------
Logfile for book ID 2 (Die Stille vor dem Tod)
Found 42 pages
2
Page count: 42.0

Here is the result on Windows Version 2.75.1:

Count Page/Word Statistics
-------------------------------
Logfile for book ID 2 (Die Stille vor dem Tod)
Found 42 pages
2
C:\Program Files\Calibre2\pylib.zip\dateutil\parser.py:339: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
Page count: 42.0

[davidfor]

@Dirk-71: OK, that's pretty conclusive that it isn't working.

Do you know if you are using temporary directory that is a symlink? Looking at the changes between 2.75.1 and 2.76.0, I found this. It is to fix a problem with the viewer when running on Mac where the temp directory is a symlink. And the count pages plugin does use this. It is used to open the book to get the parts in it and then run the statistics over it. If this is broken for the plugin for you, I would think it would also break the viewer.

So:

- What do you have you the temporary directory set to? Is this the system temp dir or one for calibre?
- Does the viewer work with calibre 2.76.0? Did it work for 2.75.1?

[kovidgoyal]

@davidfor: According to the logs the OP posted it does not work for 2.76 but does for 2.75.1 -- the reason is almost certainly the call to os.path.realpath() Presumably the plugin somewhere does something like os.path.relpath() or checks if the filename startswith the root directory, using the symlinked form of the root directory instead of EbookIterator.base (changing that should fix it).

[kovidgoyal]

You can probably simulate the problem on windows by changing the self.base at line 142 in iterator/book.py to point to some other directory than os.path.realpath(self._tdir.__enter__())

[Dirk-71]

@davidfor: The viewer is working in all version. I think Kovid has already given the answer to fix the bug. Thank you for your support.

[davidfor]

@Kovid: Yes, that's what is happening. There is a call to "os.path.relpath()" using "EbookIterator.base". This is being done when calculating the page count via the Adobe method. All it is doing is finding what the name of the files would be so that the compressed file size can be found from the epub file

The code involved is below. The appropriate line is in red. I assume with the change in the way the EbookIterator.base is done, the "private" is no longer in the path. As I don't have access to a Mac to test this on, I have no idea what should be here. Or maybe it would be simpler to go back to the OPF and get the names from there as they should match the names in the file.

def _get_page_count_adobe(iterator, book_path):
    '''
    This algorithm uses the proper adobe count. We look at the compressed size in the
    zip of every file in the spine, and apply the 1024 bytes calculation to that...
    '''
    import math
    from calibre.utils.zipfile import ZipFile

    with ZipFile(book_path, 'r') as zf:
        size_map = dict({(ci.filename, ci.compress_size) for ci in zf.infolist()})

        pages = 0.0
        for i in iterator.opf.spine:
            spath = i.path
            ppath = spath.partition('#')[0]
            if not os.path.exists(spath) and os.path.exists(ppath):
                spath = ppath

            if isosx:
                internal_path = os.path.relpath(spath.replace('/private',''), iterator.base).replace('\\','/')
            else:
                internal_path = os.path.relpath(spath, iterator.base).replace('\\','/')
            if internal_path in size_map:
                pages += math.ceil(size_map[internal_path] / 1024.0)
        zf.close()

    return pages

[kovidgoyal]

The robust way to do it would be something like:

base = iterator.base
csizes = {os.path.abspath(os.path.join(base, ci.filename)): ci.compress_size for ci in zf.infolist()}

for path in iterator.spine:
   sz = csizes.get(os.path.abspath(path))
   if sz is not None:
      ...

[davidfor]

@Kovid: Yes, that's much better. I'll have a beta version up later, or maybe tomorrow.

[davidfor]

Attached is a beta of the Count Pages plugin. The trigger for the release is the change in calibre for handling the temporary directory on Macs. The fix for this should work and be backwardly compatible with all calibre versions and OSes. But, there are a lot more changes than this.

The other changes come largely from discussion here early last year about how to count words. At the time, I implemented a word count using the ICU method that had been added to calibre and released this as a beta. There were complaints and I never released this properly. Other changes were because of other discussions or because I was fiddling. Some have been in beta's, some haven't and I've forgotten which have. I think the German Flesch Reading Ease was, but I'm not sure. The full change list from the current released version is:

• Fix Adobe Page count on Mac machines with calibre 2.76 or later.
• Changed word count to use ICU BreakIterator. This accepts the book language, so it should be more accurate for non-English as well. Will fall back to old method if the ICU BreakIterator method cannot be loaded.
• Add option to choose between ICU word count and the old one. The default is ICU word count.
• Add callback for other plugins calling the count pages. See method call_plugin_callback in common_utils.py.
• Add language awareness for ICU word count. Uses language in the book, otherwise defaults to English.
• Added German version of Flesch Reading Ease.
• Changed tooltip in configuration dialog to show on field as well as label.
• Added localization support for dialogs.
• Change way statistics were written the metadata to reduce side effects.

I have been default word count will be the ICU method. That's right, it isn't backwardly compatible, but that's the way I am doing it. If you don't like that, uncheck the option in the configuration before doing a count.

I have also add made the plugin localizable. If anyone wants to do a translation, you can use POEdit. This is an easy to use, free tool for translating. You need to extract the file "default.po" from the plugin file and open it POEdit, set the language and enter the translated text for each source string. POEdit shows what has and hasn't been translated. When you have done the translations, save the file and post or send it to me. And if you see something I missed, tell me.

I'll give this a few days for testing and hopefully some translations. If there are any problems, please report them here with as much detail as possible. Suggestions for improvements are also welcome, but I'll probably not do anything with them until after releasing these changes.

[Dirk-71]

@davidfor: Thank you. I just checked the beta and it's working now on my Mac. If you need support to translate your plugin in German I can help but need to know how to do it.

[Terisa de morgan]

@davidfor, I've checked the beta and the callback function works smoothly. Here I send you the translation in Spanish. I've found only a mistake "Updating" and "Updating ".

[Dirk-71]

Oh, I see, it's a po-File. I know how to do it :-).
I will elaborate on the German translation.