Beta testers for calibre 9 needed

[JSWolf]

Originally Posted by kacir

I have asked for a tweak or something here:
https://www.mobileread.com/forums/sh...15#post4564615

It was explained to me that: the #pages column is a dynamicaly generated column and as such is not backed up in metadata.opf file (which was the main concern of mine) and that the existing books are not scanned unless you activate a bookshelf view (or somehow display the #pages column value). The newly added books are processed to determine the number of pages but that adds very little overhead. You can see for yourself: create a small library and activate the bookshelf view. You can see how quickly the books are processed as the bookshelf is displayed with proper book widths.

Currently I am looking at compiling my own version of Calibre with a bookshelf view option disabled in the layout menu/toolbar, so I do not click on it by mistake when switching between a list view and cover grid but it is not a high priority task. I use Linux at home, if I was a Mac or Windows user I would not even contemplate that, you MUST compile Calibre on Linux and it is a highly non-trivial task. Even just making sure that you have everything needed to compile the "vanilla" sources is a lot of work.

It's not #pages. It's {pages}.

Will you be editing the code for every single release of calibre going forward? Is this going to be as (un)supported as the version that supposedly removed AI?

[un_pogaz]

Originally Posted by kacir

Currently I am looking at compiling my own version of Calibre with a bookshelf view option disabled in the layout menu/toolbar, so I do not click on it by mistake when switching between a list view and cover grid but it is not a high priority task. I use Linux at home, if I was a Mac or Windows user I would not even contemplate that, you MUST compile Calibre on Linux and it is a highly non-trivial task. Even just making sure that you have everything needed to compile the "vanilla" sources is a lot of work.

You don't need to go that far.
Yes, scanning the entire libray will take a bit of time if you accidentaly click on the bookshelf button, but it is a process perform in backroung that like when you add new book, adds very little overhead.

That infortunate, but is a complex mixte of compromise to ensure that the build-in Calibre Pages count contain valid value in the case the user want to use it later
A choice that was made easier by the fact that Calibre Pages count is pretty fast (3 book by second, 1000 by hour), especialy compared to Pages Count Plugin, so once your library is completely done, your completely fine. Just be a little patient.

The only point that matter is IF you encounter freezing and stuttering of main interface when the library scan is perform, please thanks to report it if so.

[un_pogaz]

Also, is not a "dynamicaly generated column": the Calibre Page is well stored into the library database, making that once a book is scanned, is not scanned again. But inded, this value is not stored into metadata.opf

[un_pogaz]

@Kovid, I honestly hadn't noticed that the Calibre Page was not stored in the metadata.opf backup in the library. It is logical given the particular nature of this field, however, I think it would still be relevant to at least export this value in OPF file when using the actions "Save as...", even if it means ignoring this when importing a book in the library.

I can try to work on that, but make sens to you?

[kovidgoyal]

Why? What purpose would it serve there? What's the use case for it?

[JSWolf]

Originally Posted by kovidgoyal

Why? What purpose would it serve there? What's the use case for it?

IMHO, the reason to store the calibre generated page number is if anyone has to restore the database, calibre won't have to recompute all the page numbers again. Anyone using a mechanical hard drive could find this process a bit slow if they have enough books.

[JSWolf]

If the database is restored and then the bookshelf button is not clicked, will the page number compute?

[un_pogaz]

Originally Posted by kovidgoyal

Why? What purpose would it serve there? What's the use case for it?

Well, is a value that exist and can be possibly used, and make it "unreachable" outside of the library doesn't seem particularly good to me. There are internal values in the library that really make no sense to be export, but the pages count not realy fall in this category either.

But it right I don't have use case for that, so certainly don't bother that much. Just a passing idea.

[un_pogaz]

Originally Posted by JSWolf

IMHO, the reason to store the calibre generated page number is if anyone has to restore the database, calibre won't have to recompute all the page numbers again. Anyone using a mechanical hard drive could find this process a bit slow if they have enough books.

With efficiency of the scan is, the speed gain would not be relevant when restoring a library. Especially since the value stored in the database is more complexe that just a numeric value, so it's much easier and safe to rescan the book from scratch.

[JSWolf]

Originally Posted by un_pogaz

With efficiency of the scan is, the speed gain would not be relevant when restoring a library. Especially since the value stored in the database is more complexe that just a numeric value, so it's much easier and safe to rescan the book from scratch.

What's stored in the database besides the page number?

[un_pogaz]

• "book" the book id
• "pages" the value itself
• "algorithm" numeric value of what version of count page algorithm was used for this entry
• "format" format used to retrive the value
• "format_size" size of the format used to retrive the value
• "timestamp" horodatage of the scan
• "needs_scan" bool value to determine if this entry is in the pending queu or not

The only value that deserve to be exported is "pages", so the one that everyone see. Expoted the others that are internal values make no sense, and in consequently, it is unreliable to reimport the "pages" value.

[kacir]

Originally Posted by JSWolf

What's stored in the database besides the page number?

Install a "DB browser for SQLite", or some equivalent, make a copy of your Calibre database file and have a look. I was surprised how complex the database is and how many tables it has. It has 38 tables, 27 indices, 11 views, 40 triggers.

[kacir]

Originally Posted by un_pogaz

The only value that deserve to be exported is "pages" ...

I respectfully disagree. metadata.opf is there in case of disaster, when a damaged/incomplete Calibre collection needs to be salvaged, or when the main database file is corrupted. In such case a recompute of page size of the book is a negligible expense in comparison with the rest of the work.

[un_pogaz]

Originally Posted by kacir

I respectfully disagree. metadata.opf is there in case of disaster, when a damaged/incomplete Calibre collection needs to be salvaged, or when the main database file is corrupted. In such case a recompute of page size of the book is a negligible expense in comparison with the rest of the work.

So, in fact we are on the same opinion. That exactly with a say ealry in separated comments (but less clearly).

I talk of "The only value that deserve to be exported is "pages"..." in the case of separated saved OPF when using the action "Save as...", but the metadata.opf backup is a whole other story and I agree with you.

[theducks]

Originally Posted by kacir

I have asked for a tweak or something here:
https://www.mobileread.com/forums/sh...15#post4564615

It was explained to me that: the #pages column is a dynamicaly generated column and as such is not backed up in metadata.opf file (which was the main concern of mine) and that the existing books are not scanned unless you activate a bookshelf view (or somehow display the #pages column value). The newly added books are processed to determine the number of pages but that adds very little overhead. You can see for yourself: create a small library and activate the bookshelf view. You can see how quickly the books are processed as the bookshelf is displayed with proper book widths.

Currently I am looking at compiling my own version of Calibre with a bookshelf view option disabled in the layout menu/toolbar, so I do not click on it by mistake when switching between a list view and cover grid but it is not a high priority task. I use Linux at home, if I was a Mac or Windows user I would not even contemplate that, you MUST compile Calibre on Linux and it is a highly non-trivial task. Even just making sure that you have everything needed to compile the "vanilla" sources is a lot of work.

I think you are confusing which is which. #pages is from the PI
and metadata.opf (the backup file) DOES get an entry.
snipped from one of my book entries:

  <meta name="calibre:user_metadata:#pages" content="{&quot;table&quot;: &quot;custom_column_4&quot;, &quot;column&quot;: &quot;value&quot;, &quot;datatype&quot;: &quot;int&quot;, &quot;is_multiple&quot;: null, &quot;kind&quot;: &quot;field&quot;, &quot;name&quot;: &quot;PagesP&quot;, &quot;search_terms&quot;: [&quot;#pages&quot;], &quot;label&quot;: &quot;pages&quot;, &quot;colnum&quot;: 4, &quot;display&quot;: {&quot;number_format&quot;: &quot;{:,}&quot;, &quot;description&quot;: &quot;&quot;, &quot;web_search_template&quot;: &quot;&quot;}, &quot;is_custom&quot;: true, &quot;is_category&quot;: false, &quot;link_column&quot;: &quot;value&quot;, &quot;category_sort&quot;: &quot;value&quot;, &quot;is_csp&quot;: false, &quot;is_editable&quot;: true, &quot;rec_index&quot;: 32, &quot;#value#&quot;: 351, &quot;#extra#&quot;: null, &quot;is_multiple2&quot;: {}}"/>