Plugboard, template, and custom composite column recipes

[Sweetpea]

Originally Posted by chaley

Yes, it would be possible to create a text column type that is actually an enumeration. There are, of course, complexities. The biggest is ensuring that all the places that can change the contents respect the enumeration values. This will touch metadata editing, drag & drop, the CLI (I think), and the library GUI.

Aren't there always

Originally Posted by chaley

Nope. T'is all or nothing.

I figured (but I might have overlooked something!)

Originally Posted by chaley

Thinking about it, you can get a poor-man's enumeration type using a normal and a composite column. The normal column would be used to enter the information. The composite column would use a switch statement to limit the text. For example, if the first column is integer, then the switch would test for 1, 2, etc. If the value is not one of those, it takes the default.

Hmm, that might work... Let me play with that!

[chaley]

Originally Posted by Sweetpea

Would it be possible to make a custom composite column that limits the text you can add?

Would an enumeration type that forces inclusion of the empty string be useful? In other words, would having a column that allows [one, two, three] and the empty string be of use?

[Sweetpea]

Originally Posted by chaley

Would an enumeration type that forces inclusion of the empty string be useful? In other words, would having a column that allows [one, two, three] and the empty string be of use?

Yes.

I tried the two-column layout. It works, but it's not practical. So, I'll mis-use the boolean till there's a true enumeration field

[chaley]

Originally Posted by Sweetpea

Would it be possible to make a custom composite column that limits the text you can add?

Example: I now use the "tri-state boolean" to keep track of read/to-be-read/reading books. But, I'd like to use the normal boolean for other columns. But I can't have both (or can I?).

An enumeration column type will be in the next release.

When you create the column, you specify the comma-separated list of valid values. The true list of valid values is None + the supplied list, and for this reason you cannot provide an empty value in the list. The default value for the column is None.

[Sweetpea]

Originally Posted by chaley

An enumeration column type will be in the next release.

When you create the column, you specify the comma-separated list of valid values. The true list of valid values is None + the supplied list, and for this reason you cannot provide an empty value in the list. The default value for the column is None.

Thanks!

[chaley]

I have a partial implementation of an 'advanced' template language. Before I submit it to kovid for review, I want a sanity check. I might be making something that no one else can use.

The overarching goal is to make the template language into a 'real' programming language, at least up to a point. In the current language, all items are constants. For example, the replacement in a switch statement is a constant; you can't replace the source with something computed, such as another field.

Subgoals:
1) use as much of the existing template stuff as I can
2) make it easy to parse (efficiency)
3) support sequences of expressions (aka statements)

What I have is a language where everything is a function. A constant is a function that returns itself. A name not followed by a '(' is a reference to a local variable, otherwise it is a call of a function.

An example might make the above clearer. Consider the following (which actually works but is rather silly):

{#enum:0>15s:'
  assign(a, field('title'));
  assign(b, switch(a, '[DE]', lowercase(a), $));
  b
'}

The first line (line breaks are optional) is the same as today's language. It names the field that you are working with and defines the format to apply after the template function(s) are run. The difference is that the trailing colon is followed by a single quote. This tells the template processor that the 'new style' language is being used.

The second line assigns the value of the metadata field 'title' to the local variable 'a'.

The call to 'switch' in the third line shows one difference. The first variable in a function call specifies the value that the function is to operate upon. The special local variable '$' means the initial field, in this case #enum. The second parameter is the regexp that switch is to apply. The third parameter is the value that switch will return if the regexp matches. This third parameter is a call of the function lowercase(a). The default value of the switch (no regexps match) is $ (#enum). The result of the switch will be the lowercase value in the variable 'a' if it contains either a D or an E, otherwise the value in $ (#enum). The result of switch is assigned to the variable b.

The last line simply names 'b', so that is its value. As it is the last line in the program, that is also the value of the program. Had this line been left out, the value of the assign (value of b) would be the result of the template.

Note the trailing single quote before the } character. This quote is required as a check that the expression is in the new template language.

As you can see in the example, constant strings are entered in quotes. Either single or double quotes can be used. Constants can also be numbers.

In addition to the standard functions, the extended language has:

• cmp(x, y, lt, eq, gt) -- treating x and y as numbers, if x < y then return lt else if x == y then return eq else return gt
• concat(x, y) -- return a string x + y (x . y in PHP)
• field(x) -- lookup x in metadata and return that value.
• strcmp(x, y, lt, eq, gt) -- like cmp, only treat x and y as strings
• substr(x, start, end) -- return the start'th through the end'th characters of x. If end == 0, then use length(x).

I know that I don't have looping constructs, but I don't see any need for them.

Have I missed something important? Any comments?

[kovidgoyal]

It would be nice to support negative indices for substr for "length from the end"

[chaley]

Originally Posted by kovidgoyal

It would be nice to support negative indices for substr for "length from the end"

It does.

The result is x[start:end]. As it makes no sense for end to be zero, I used zero to indicate x[start:]. There is no ambiguity with the start field, so no hack was required.

[chaley]

The new language will be available in the next release.

[Starson17]

Originally Posted by chaley

The new language will be available in the next release.

Thanks. (I don't have an immediate use, but it's nice to know the feature is there.)

[chaley]

As an example and a test of the new language, I have implemented the last recipe in the first post in the new template language. Results are under the spoiler.

In the process I 'improved' the language a bit. The possibility to add comments, the 'program:' keyword, use of '=' for assignment, and the print function will be available in the next release (0.7.36)

Metadata: Put series into the title, using either initials or a shortened form. Strip leading articles from the series name (any) (inspired by Gary_M_Mugford)

Spoiler:

The following program produces the same results as the original recipe, using only one custom column to hold the results of a program that computes the special title value.

    Custom column: 
    Name: #special_title
    Template: (the following with all leading spaces removed)
        program:
        #	compute the equivalent of the composite fields and store them in local variables
            stripped = re(field('series'), '^(A|The|An)\s+', '');
            shortened = shorten(stripped, 4, '-' ,4);
            initials = re(stripped, '[^\w]*(\w?)[^\s]+(\s|$)', '\1');

        #	Format the series index. Ends up as empty if there is no series index.
        #	Note that leading and trailing spaces will be removed by the formatter,
        # 	so we cannot add them here. We will do that in the strcat below.
        #	Also note that because we are in 'program' mode, we can freely use
        #	curly brackets in strings, something we cannot do in template mode.
            s_index = template('{series_index:0>2.0f}');

        #	print(stripped, shortened, initials, s_index);

        #	Now concatenate all the bits together. The switch picks between 
        # 	initials and shortened, depending on whether there is a space
        #	in stripped. We then add the brackets around s_index if it is
        #	not empty. Finally, add the title. As this is the last function in
        # 	the program, its value will be returned.
            strcat(
                switch(	stripped, 
                        '.\s', initials, 
                        '.', shortened,
                        field('series')),
                test(s_index, strcat(' [', s_index, '] '), ''),
                field('title'));

    Plugboard expression:
    Template:{#special_title}
    Destination field: title

It would be possible to do the above with no custom columns by putting the program into the template box of the plugboard. However, to do so, all comments must be removed because the plugboard text box does not support multi-line editing. It is debatable whether the gain of not having the custom column is worth the vast increase in difficulty caused by the program being one giant line.

[Maggie-Me]

[QUOTE=chaley;1141585]Several people are doing interesting and useful things with templates, plugboards, and composite fields. This thread is both a request that they share what they have done and a place to put the information.

Suggestions/recipes are summarized below. Don't hesitate to post corrections and useful variants.


Sorry if this has been covered, I take it you would use plugboards to write the title of the book to device (sony in my case) where would you enter this information to say add series number to begining of book title?

Thanks

[chaley]

Originally Posted by Maggie-Me

Sorry if this has been covered, I take it you would use plugboards to write the title of the book to device (sony in my case) where would you enter this information to say add series number to begining of book title?

There are several recipes on the first post that do variants of what you are asking for. The one that seems closest is

Metadata: Show series - series index - title, with series as initials (Sony) (inspired by speakingtohe)

If you want the full series name instead of the series as initials, then use the following in the template:

{series}{series_index:0>2s| - | - }{title}

[Maggie-Me]

Originally Posted by chaley

There are several recipes on the first post that do variants of what you are asking for. The one that seems closest is

Metadata: Show series - series index - title, with series as initials (Sony) (inspired by speakingtohe)

If you want the full series name instead of the series as initials, then use the following in the template:

{series}{series_index:0>2s| - | - }{title}

thanks for quick response, I entered that in two places... Under "Send to Device" in preference and under "Device Plug In" for Sony, but doesn't add the series number to the reader. Am I entering to the right place?

thanks

[chaley]

Originally Posted by Maggie-Me

thanks for quick response, I entered that in two places... Under "Send to Device" in preference and under "Device Plug In" for Sony, but doesn't add the series number to the reader. Am I entering to the right place?

No.

You need to set up a plugboard (preferences -> metadata plugboards). Choose the format 'device_db', then choose the device 'PRS505' (which covers all sonys). Fill in one of the template boxes with either the line from the original recipe or the line from my post. Set the destination field to 'title'. Save the plugboard.

If metadata management is set to automatic (preferences, send to device, metadata management), then the titles will be updated the next time you connect your Sony. If metadata management is set to something else, then you must resend your books.