Function virtual_libraries()

[Divingduck]

Short question, have anything change in using the function?

I use it from time to time with a custom column to find out in what VL a book is available.

Setup:
Custom column: Column build from other columns, behaves like tags
Code: program:virtual_libraries()

Behavior:
First book will shown correctly and all following books showing "Template Error maximum recursion depth exceeded while calling a Python object" but if I click on the column of the book it shows the listed virtual libraries correct in the template editor.
Same behavior for code:
{0:'virtual_libraries()'}

[chaley]

There are many problems with that function, so many that I am considering removing it. Not only does it not work in some unknown situations, its performance can be so bad that calibre slows to a crawl.

Try using one or both of the plugins [GUI Plugin] Save Virtual Libraries To Column (GUI) or [Library Closed Plugin] Save Virtual Libraries To Column. These populate a custom column with the VLs for a book, and are much faster and more reliable than the function. The first is run on demand. The second runs whenever you close the library.

[Divingduck]

@chaley,
Thanks for your answer and alternative solution. I will check out your PI's. The performance issue was known and that's why I use the functionality/column only if I need it.

If you want, I can send you the DB with 7 books for testing. Maybe this helps you and Kovid locating the problem. When I switch between libraries, sometimes the error changed to "These function can only use in the GUI" and in this case I became a template error on other columns build from other columns (e.g. {author_sort}). When I open the VL-column in the template editor I became the old error message back. Curious, it looks like there is running something out of boundary.

[chaley]

Originally Posted by Divingduck

@chaley,
If you want, I can send you the DB with 7 books for testing. Maybe this helps you and Kovid locating the problem. When I switch between libraries, sometimes the error changed to "These function can only use in the GUI" and in this case I became a template error on other columns build from other columns (e.g. {author_sort}). When I open the VL-column in the template editor I became the old error message back. Curious, it looks like there is running something out of boundary.

Sure, send it to calibre_bugs at charles dot haleys dot org.

[chaley]

I received the library. Of course, I cannot make it fail.

I am running Windows 10/64 and calibre 64. I tried both the calibre 3.6 binary and running from source.

Things I tried, with no failures, no exceptions, no error messages:

• Start calibre with your library as the current library.
• Start calibre with a different library then switch to your library.
• Switch back and forth between libraries.
• Open a template tester and run "program:virtual_libraries()"
• Bring up the template editor on your Vitruelle Bibliothek column.

Some thoughts:

• Is there any chance that your machine is running out of memory?
• If you are running on Linux or iOS, do you have a windows machine around where you can try it? I don't have any other OS.
• Are you running the 32-bit version of calibre?

[Divingduck]

I forgot to add the system data. The machine is a Windows10/64bit latest official build including all updates and 8GB RAM. So RAM should not be an issue on this machine. I run calibre 64bit.

I did test it as well with --ignore-plugins to take care of influences from this side. No change. I will setup a VL and make a clean test environment and look what happen. Maybe there is something else going on as this machine have some extras installed. I will come back as soon I am ready.

[Divingduck]

Hi chaley,
I setup a new Windows 10/64bit system with only calibre 64bit installed. I got the same error behavior.
Pls see attached pic.

[Divingduck]

@chaley, I maybe find it...

I was wondering why I got the error immediately in a clean installation too. For me it have something to do how calibre is managing the searches. I use saved searches in VL's as well as searches defined by the VL-dialog.

I delete from the test library the files metadata.db and metadata_db_prefs_backup.json to get rid of all database setup, open calibre and told it to restore the database from scratch using its own metadata.opf files. After that the errors gone away, what do not really surprise me (as there are no searches available to provide the function with values. So the field was empty for each book.
Then I start to build up the VL's via create VL's manually again as they was listed in the dialogue for editing VL's. After I put in the forth VL search string the error came back again. (see pic)

I then go back and delete the last VL definition and the third after that too. and the error go away. After checking the search terms I think what's going on here.

I have two searches, one for books in German and one for books in English with following definitions in the test library:
languages:"=Deutsch" or "=German", languages:"=Englisch" or "=English"
After I changed this to
languages:"=Deutsch" or languages:"=German", languages:"=Englisch" or languages:"=English"
the test library works.

What makes me wonder is, why on one system this makes no difference and work fine and on a other systems the function crash and is able to influence other columns with definition "build from other columns" like I had it for the authors sort or calculation columns. In addition I know the old definitions was working for long time good. It seems like there is a little glitch in the chain. Maybe this will help to find out what happen with some installations to find the root cause.

@Kovid,
by doing this test I recognize, it would be nice having the possibility to export/import all or single a definition of Virtual Libraries and saved searches. This will help to use them selective for libraries (e.g. like we have it in the editor). Maybe something you can consider to implement some when in the future. At last for me it is a benefit that I was missing in the past and just remember back again by this action.

[chaley]

Originally Posted by Divingduck

Hi chaley,
I setup a new Windows 10/64bit system with only calibre 64bit installed. I got the same error behavior.
Pls see attached pic.

I found the problem. I created a totally clean calibre configuration and the errors started appearing. The problem is that some of your virtual library search expressions use "naked" searches; searches with no column prefix. Example: Bücher Deutsch has the expression

languages:"=Deutsch" or "=German"

Because the second term, "=German", has no prefix, calibre will search in every calibre column, including the composite column computing the virtual libraries. Because that column hasn't yet been computed, the searches for the virtual libraries are evaluated again. This causes recursion because the search will hit the composite column and start over again.

There are several cases in your VL definitions where you have a naked term. If I remember correctly they are all of the form col:value1 or value2. I am sure that you meant col:value1 or col:value2.

The reason I didn't see the error at first is that I have checked the box in Preferences / Searching that limits the fields searched by naked terms.

[chaley]

Originally Posted by Divingduck

@chaley, I maybe find it...

You found exactly what I found and reported a few seconds ago.

I suspect that on the other machine you also have checked the box in Preferences / Searching.

[Divingduck]

Edit: Yes I set the same check box.
Guess, we find out at the same time

[Divingduck]

@chaley,

Short feedback. I change all involved searches in my main library and everything looks good now. Now I only need to check it for all other libraries and I'm fine.

Thanks for your help and time.

Best regards,
DivingDuck

[chaley]

Originally Posted by Divingduck

@chaley,

Short feedback. I change all involved searches in my main library and everything looks good now. Now I only need to check it for all other libraries and I'm fine.

Thanks for your help and time.

Best regards,
DivingDuck

Thanks for the feedback!