[Bug?] Field(s) in prefix/suffix

[Backi]

Could you confirm the malfunction I encountered using fields in prefix/suffix?

This template (just an example) works on books with and without series/series_index:

{series:|{isbn}|}{title}

These surprisingly don't work on books without series/series_index:

{series_index:|{isbn}|}{title}
{series_index:||{isbn}}{title}

In both cases I get this error message (here for the first one):

calibre, version 0.8.42
WARNING: Could not save some books: Could not save some books, Click the show details button to see which ones.

Traceback (most recent call last):
File "site-packages\calibre\library\save_to_disk.py", line 437, in save_serialized_to_disk
File "site-packages\calibre\library\save_to_disk.py", line 305, in do_save_book_to_disk
ValueError: Failed to calculate path for save to disk. Template: {series_index:|{isbn}|}{title}
Error: Single '}' encountered in format string

The only difference could be that series is a string whereas series_index is a number?
I wonder, why the field series_index is considered at all for books where it's empty. The section "Advanced formatting" states:

If field has no value, then the result will be the empty string (nothing); the prefix and suffix are ignored.

[theducks]

I don't think inserting a field was ever envisioned in this manner.
I believe those positions expect 'literals' (characters).
If you don't have a Series, you can't have a series index

I believe you will need to us Program mode (And, Please don't ask me for help with that)

[Backi]

Originally Posted by theducks

I don't think inserting a field was ever envisioned in this manner.
I believe those positions expect 'literals' (characters).

I don't think so, because otherwise I would get something like "_isbn_" instead of the value for {isbn} in prefix/suffix.

Originally Posted by theducks

If you don't have a Series, you can't have a series index

But this one without fields in prefix and suffix works:

{series_index:|test1|test2}{title}

[chaley]

Originally Posted by Backi

Could you confirm the malfunction I encountered using fields in prefix/suffix?

This template (just an example) works on books with and without series/series_index:

{series:|{isbn}|}{title}

These surprisingly don't work on books without series/series_index:

{series_index:|{isbn}|}{title}
{series_index:||{isbn}}{title}

It says in the single-function section of the manual: "All arguments must be constants; there is no sub-evaluation." Your template is using sub-evaluation because "{isbn}" is inside the series_index template expression.

The use of template expressions within template expressions is highly unreliable. I consider it a bug that it works at all, and would take it out except that doing so would cause the people for whom it works to yell very loudly. You have hit one of the problems -- expecting { ... } to nest. There are cases where they do not, one of which is references to series_index in save_to_disk templates. In the failure case a { is paired with the nearest }, not with the matching nested }. In other words, the { in front of "series_index" is matched against the } following "isbn", not against the one following the '|' character. The other common failure case occurs when the value being inserted contains a template special character such as '|', '{', '}', or sometimes ','.

You can express the equivalent of {series_index:|{isbn}|}{title} without nesting by using the following "template program mode" template.

{series_index:'test($, strcat(field('isbn'), $), '')'}{title}

For completeness, an equivalent template in general program mode is

program: 
	s = field('series_index');
	strcat(test(s, strcat(field('isbn'), s), ''), field('title'))

[Backi]

Originally Posted by chaley

It says in the single-function section of the manual: "All arguments must be constants; there is no sub-evaluation." Your template is using sub-evaluation because "{isbn}" is inside the series_index template expression.

Is it really sub-evaluation? I thought there is no sub-evaluation, because there is no function after ":". All I have is a prefix/suffix which is a field, not an argument of any function.

Originally Posted by chaley

The use of template expressions within template expressions is highly unreliable. I consider it a bug that it works at all, and would take it out except that doing so would cause the people for whom it works to yell very loudly. You have hit one of the problems -- expecting { ... } to nest. There are cases where they do not, one of which is references to series_index in save_to_disk templates. In the failure case a { is paired with the nearest }, not with the matching nested }. In other words, the { in front of "series_index" is matched against the } following "isbn", not against the one following the '|' character.

This is a weird behavior. I think it should be either taken completely out or implemented completely, or at least documented in the manual.

Originally Posted by chaley

You can express the equivalent of {series_index:|{isbn}|}{title} without nesting by using the following "template program mode" template.

{series_index:'test($, strcat(field('isbn'), $), '')'}{title}

For completeness, an equivalent template in general program mode is

program: 
	s = field('series_index');
	strcat(test(s, strcat(field('isbn'), s), ''), field('title'))

Thanks! I think, I will use the general program mode.

[chaley]

Originally Posted by Backi

Is it really sub-evaluation? I thought there is no sub-evaluation, because there is no function after ":". All I have is a prefix/suffix which is a field, not an argument of any function.

Yes, it is sub-evaluation. There is a non-constant in the template -- a template expression in a template expression.

This is a weird behavior. I think it should be either taken completely out or implemented completely, or at least documented in the manual.

I can't implement it completely/correctly without reimplementing python's text formatting language, which isn't something I am willing to do.

You are right that the documentation can be improved, which is a never-ending task. In this case I copied the bold "Do not use subtemplates" warning from the single-function-mode section to the section discussing prefixes and suffixes.