Beta testers for calibre 9 needed

[kovidgoyal]

Hi all,

I am excited to announce betas for calibre 9. The major new feature is a "bookshelf view" that can be used in calibre in addition to the existing book list and cover grid views. To try it, click the layout button in the bottom right corner and switch to Bookshelf view. See the screenshot below for what it looks like. Betas are available at: https://download.calibre-ebook.com/betas/

You can hover over any book spine and it will show the full cover. Keyboard and mouse controls all work as expected. Double click to view a book, just as in the cover grid view. Right click and you can choose to group the books on the Shelves by author, publication year, rating, etc.

You can configure the shelf view in Preferences->Look & feel. The hover animation can be changed or disables. And you can choose whether and how you want the cover thumbnail displayed on the spine.

Additionally, the version of Python and Qt and various other dependencies have all been updated, which may be a source of regressions, so testing is appreciated.

Note that the width of the spines is based on an estimation of the number of pages in the book. For this there is a new Pages column in calibre that is maintained automatically and updated when you add new books/formats to calibre. You can view the pages column by adding it to the book list view via Preferences->Add your own columns. When you add the pages column, it will initially say "calculating". The page counts for all books will be updated gradually in the background at the rate of about a 1000 books an hour. Once all existing books have been counted, new books added will get their updated pages value in a second or two. You can also configure the bookshelf to instead use an existing custom column via a template.

Because of the addition of the Pages column if you install the beta it will update the database schema to add the new column. The updated database will still work with older calibre versions, but the pages column will get out of sync since older calibre will not update it. You can ask calibre to update the pages column by going to Preferences->Look & feel->Bookshelf and clicking "Recalculate page counts".

Thanks and Happy New Year!

P. S: The bookshelf view was developed largely by MR member un_pogaz

[Karellen]

Hi @kovidgoyal
Thanks for the update, which I have just installed.
In the Editor while editing an epub, I get this error when trying to save my changes...

calibre, version 8.99.0
ERROR: Could not save: Saving of the book failed. Click "Show details" for more information. You can try to save a copy to a different location, via File->Save a copy

Traceback (most recent call last):
File "calibre\gui2\tweak_book\save.py", line 195, in process_save
File "calibre\gui2\tweak_book\save.py", line 207, in do_save
File "calibre\gui2\tweak_book\save.py", line 62, in save_container
AttributeError: module 'os' has no attribute 'fchown'

[kovidgoyal]

Fixed by:
https://github.com/kovidgoyal/calibr...e08ff2403122f9

[Karellen]

Thanks
When will you release an update? I guess you will want a few more bugs fixed.
(damn, shouldn't have installed it on my working library )

[kovidgoyal]

In an hour or so. These are betas I have no problems making frequent updates In fact I'll leave it to build now while I go off to have a leisurely lunch.

[Karellen]

Thanks. Enjoy lunch!!

[un_pogaz]

Originally Posted by kovidgoyal

P. S: The bookshelf view was developed largely by MR member un_pogaz

And I myself must give a lot of credit to the GitHub user Andy Chuong who made the initial PR, where I took over the project somewhat by chance, and without whom I would not have had a foundation to work upon.

Damn, it was something to make, but truly rewarding at every stage. Including to see my code go to the limbo as Kovid work on it 🥲, but it was to be expected: even if fonctionaly and a good prototype, it was poorly optimized.
Thanks to you too to have make it what it's now.

[Terisa de morgan]

Enjoy your lunch, I'll use part of the vacation time these days for checking the beta

[rantanplan]

For the new number of pages field, how does it calculate? Is it similar to the way how the Count Pages plugin works?

Also, is the number of pages used when exporting an epub to the e-reader?

[Comfy.n]

Originally Posted by rantanplan

For the new number of pages field, how does it calculate? Is it similar to the way how the Count Pages plugin works?

Also, is the number of pages used when exporting an epub to the e-reader?

I was wondering that too. My Count Pages setup uses ebook viewer algorithm.

Then I tried the below templates (PTM should be faster, as @chaley used to say):

GPM:

PHP Code:

program:
    pages_val = field('pages');
    pp_val = field('#pp');
    if pages_val == pp_val then 'Equal' else 'Not Equal' fi 


PTM:

PHP Code:

python:
def evaluate(book, context):
    pages_val = book.get('pages', 0)
    pp_val = book.get('#pp', 0)
    if pages_val == pp_val:
        return 'Yes'
    else:
        return 'No' 


In the examples, #pp is my Count Pages custom column. The templates are meant to be used in a custom column built from other columns and provide a yes/no, equal/not equal, comparing the CP column against calibre's builtin Pages.

Calibre is still calculating my library page counts, but from what I saw so far, it very rarely matches the Count Pages. So rarely that I plan to research what do the identical page count books have in common...

I'm getting errors on counts, mostly on PDF files. I am planning to report a bug when i have more info, but here's some examples:

Spoiler:

Failed to count pages in book: 86415 PDF with error:
Traceback (most recent call last):
File "calibre\library\page_count.py", line 251, in serve_requests
File "calibre\library\page_count.py", line 187, in count_pages
File "calibre\library\page_count.py", line 42, in count_pages_pdf
File "calibre\utils\podofo\__init__.py", line 183, in get_page_count
podofo.Error: PdfErrorCode::InvalidDataType
Callstack:t#0 Error Source: main\PdfTokenizer.cpp(457), Information: Could not read variant
File: main\PdfTokenizer.cpp Line: 457 Could not read variant

Failed to count pages in book: 96826 PDF with error:
Traceback (most recent call last):
File "calibre\library\page_count.py", line 251, in serve_requests
File "calibre\library\page_count.py", line 187, in count_pages
File "calibre\library\page_count.py", line 42, in count_pages_pdf
File "calibre\utils\podofo\__init__.py", line 183, in get_page_count
podofo.Error: PdfErrorCode::NoEOFToken, No EOF Marker was found in the PDF file.
Callstack:t#0 Error Source: main\PdfParser.cpp(97), Information: Unable to load objects from file
t#1 Error Source: main\PdfParser.cpp(143), Information: Unable to load xref entries
t#2 Error Source: main\PdfParser.cpp(356)
File: main\PdfParser.cpp Line: 97 Unable to load objects from file
File: main\PdfParser.cpp Line: 143 Unable to load xref entries
File: main\PdfParser.cpp Line: 356

Failed to count pages in book: 96859 PDF with error:
Traceback (most recent call last):
File "calibre\library\page_count.py", line 251, in serve_requests
File "calibre\library\page_count.py", line 187, in count_pages
File "calibre\library\page_count.py", line 42, in count_pages_pdf
File "calibre\utils\podofo\__init__.py", line 183, in get_page_count
podofo.Error: PdfErrorCode::InvalidDataType
Callstack:t#0 Error Source: main\PdfTokenizer.cpp(457), Information: Could not read variant
File: main\PdfTokenizer.cpp Line: 457 Could not read variant


Failed to count pages in book: 96812 EPUB with error:
Traceback (most recent call last):
File "calibre\utils\formatter.py", line 2093, in safe_format
File "calibre\utils\formatter.py", line 1973, in evaluate
File "string\__init__.py", line 209, in vformat
File "string\__init__.py", line 263, in _vformat
File "calibre\utils\formatter.py", line 1913, in format_field
File "calibre\utils\formatter.py", line 1810, in _eval_program
File "calibre\utils\formatter.py", line 1034, in program
File "calibre\utils\formatter.py", line 1048, in expression_list
File "calibre\utils\formatter.py", line 1684, in expr
File "calibre\utils\formatter.py", line 1682, in expr
File "calibre\utils\formatter.py", line 1193, in do_node_func
File "calibre\utils\formatter_functions.py", line 268, in eval_
File "calibre\utils\formatter_functions.py", line 1275, in evaluate
File "calibre\utils\formatter_functions.py", line 277, in only_in_gui_error
File "calibre\utils\formatter_functions.py", line 197, in only_in_gui_error
ValueError: The function approximate_formats can be used only in the GUI

Anyway, everything works fine overall. The custom icons for my libraries aren't showing anymore in the Windows taskbar; I get the default calibre icon.

Updating Qt and Python is surely a source of trouble, but this might fix other issues, like the font subtypes thing!

Happy New Year!

[un_pogaz]

Originally Posted by rantanplan

For the new number of pages field, how does it calculate? Is it similar to the way how the Count Pages plugin works?

Also, is the number of pages used when exporting an epub to the e-reader?

Quite different to Count Pages plugin.
If the ePub is aPub3 and that a page number metadata is set, Calibre will use it, else it use a similiar methods in spirit (emulate lines rendering and calc pages from that), but the Calibre implementation is different. The divergence toward PC could certainly higly vary.

Else is important to note that the accuraty of the pages columns it to take with caution because of the reflow nature of ebook, the various details of formating and images; there is no "correct" value. Especialy toward a physical edition. It just average value to easly compare the book side to side.

(ComicsBook archives and PDF should be identical because static)

And no, the page is not exported to devices (I didn't see any commits on that).
My guess is since each devices has his own method to store this value, this need a specific implementation for each ones (which is a work), but that is to Kovid to see.

[Comfy.n]

Originally Posted by un_pogaz

Quite different to Count Pages plugin.
If the ePub is aPub3 and that a page number metadata is set, Calibre will use it

That helps explain the difference I noticed.

there is no "correct" value. Especialy toward a physical edition. It just average value to easly compare the book side to side.

yes, that's how I use CP, just as a comparison tool.

Calibre is very fast at the counting phase; has already counted double the books I ran CP on in my largest library.

[BetterRed]

Page counts for DOCX are incorrect, 28 page document is reported as 38 pages, 226 page document is reported as 410 pages.

I only have a word count column for the Count Pages plugin, so I can't easily compare

But afaik, the DOCX metadata includes Page count, i.e. there's no need to calculate.

BR

[kovidgoyal]

No DOCX metadata has only a page count if word bothers to update it, it doesnt always, sometimes leaving it wildly off and other software does not set it. Thus I chose not to use it.

Page counts are done using a modified version of the original calibre algorithm to yield numbers closer to paper book values.

Page counts are not stored in book metadata and not sent to devices. Indeed the page count number is rather arbitrary and is suitable only to get a sense of the different sizes of books in comparison.

[kovidgoyal]

@Comfy.n: The PDF errors are PDF files that the PODOFO library cannot parse. The page count plugin falls back to another PDF library calibre's code does not. The EPUB error is a strange one and I would like access to that epub file to try to replicate the issue.