Template language changes

[chaley]

Originally Posted by ownedbycats

so for a custom column, it's just #whatever?

Yes, but in quotes. It is an expression returning a lookup name. You could even use

('t' &'ags')

if for some reason you wanted to do that.

[chaley]

Originally Posted by ownedbycats

EDIT: I pulled the latest changes but I am not seeing inlist_field.

It is an operator, not a function. There isn't anywhere for you to "see" it other than by using it.

Another example:

program:
	if '^science' inlist_field '#genre' then 'yes' else 'no' fi

NB: I forgot to add the new operator to the syntax highlighting in the template tester so it doesn't yet show as blue. I submitted the required change.

[ownedbycats]

Ah, I was looking in the function reference. It works when I tried it.

[ownedbycats]

Does inlist_field only work for strings? e.g.:

program:

	readgoal = strcat('readinggoal:', format_date(today(), 'yyyy'));

	if
		readgoal inlist_field '#admintags'

[chaley]

Originally Posted by ownedbycats

Does inlist_field only work for strings? e.g.:

program:

	readgoal = strcat('readinggoal:', format_date(today(), 'yyyy'));

	if
		readgoal inlist_field '#admintags'

Yes, because there isn't a calibre multiple-value field that doesn't contain strings. Note that in your example, the string generated by strcat() must exactly match some item in #admintags', including blanks. That said, if it works with inlist then it will work with inlist_field.

EDIT 2: the changes to the syntax highlighter are now in calibre source.

[chaley]

28 Oct 2024 (in calibre 7.21)

• Improvement to template function has_notes. It now has two variants.
  1. has_note(field_name, field_value) -- if field_value is not '' (the empty string) , return '1' if the value field_value in the field field_name has an attached note, otherwise ''. Example:
     

     has_note('tags', 'Fiction')

     returns '1' if the tag "fiction" has a note, otherwise ''.
     
  2. (NEW) has_note(field_name, '') -- if the second parameter is '' then return a list of values in field_name that have a note. If no item in the field has a note, return ''. Example:
     

     has_note('authors', '')

     returns a list of authors that have notes, or '' if no author has a note.
  For example, the second variant is useful for showing column icons if any value in the field has a note rather than a specific value.
  
  You can also test if all the values have a note by comparing the list length of this function's return value against the list length of the values in field_name. Example:
  

  list_count(has_note('authors', ''), '&') ==# list_count_field('authors')

[thiago.eec]

Nice. Even simpler. Now I can find all authors without a note like this:

not template:"""program:
	if $authors in has_note('authors', '') then
		'1'
	else
		 ''
	fi;#@#:t:1"""

[chaley]

Originally Posted by thiago.eec

Nice. Even simpler. Now I can find all authors without a note like this:

not template:"""program:
	if $authors in has_note('authors', '') then
		'1'
	else
		 ''
	fi;#@#:t:1"""

That search finds books where all authors don't have notes, not any author that doesn't have a note. It compares two lists as strings, not as lists, so the "in" is true if the has_note() result string equals the authors string, which means all authors have notes.

It will also be at least 10 times slower than the template I posted in the other thread. Perhaps as much as 50 times slower. Of course, this only matters if you have lots of books.

This template uses the new has_note() to find books where 1 or more authors don't have a note. It works by returning '1' if all authors for a book have notes, otherwise ''. It will be slightly faster than your template, but nowhere near as fast as the python template.

program:
	if list_count(has_note('authors', ''), '&') ==# list_count_field('authors') then
		return '1'
	fi;
	''

You could eliminate the outer "not" in the search by making changing the return '1' to return '' and changing the last line to '1'.

[thiago.eec]

Oh, I see my mistake now. I'll leave the python template, then.

When using my template, I do think the first time was slow, then other times were fast. I guess it's the work of the cache in action.
________________________

EDIT: I decided to check the difference between the two templates (mine and yours, but not the python one). It seems to find the same results, even if only one author has a note. Then, looking again at the code, I notice it does work:

Example: If i have a book with two authors (AuthorA and AuthorB), and only have notes for AuthorA.

$authors = 'AuthorA & AuthorB'
has_note('authors', '') = 'AuthorA'

Now, the comparison if $authors in has_note('authors', '') will return False, as intended.

But I'm definitely staying with the python template.

[chaley]

12 Feb 2025 - (in calibre 7.26)

• Book details search URLs for text, enumeration, series, and composite custom columns. This isn't a template language change but here is a good place to mention it. You can now define a "search template" for a custom column. It is used to generate a search URL for the column, replacing the standard calibre 'search'. You enter the template in the "Search template:" box in the column definition dialog (Preferences / Add your own columns). The documentation is in the tool tip. Worth mentioning here: there are three new metadata variables available to these search templates:
  • item_value: the value of the item clicked in Book details.
  • item_value_quoted: the value of the item URL-encoded with spaces replaced by plus signs.
  • item_value_no_plus: the value of the item URL-encoded with spaces replaced by '%20'.
• New template functions:
  • make_url(path, [query_name, query_value]+) -- this function is the easiest way to construct a query URL. It uses a path, the web site and page you want to query, and query_name, query_value pairs from which the query is built. In general, the query_value must be URL-encoded. With this function it is always encoded and spaces are always replaced with '+' signs.
    
    At least one query_name, query_value pair must be provided.
    
    Example: constructing a Wikipedia search URL for the author Niccolò Machiavelli:
    

    make_url('https://en.wikipedia.org/w/index.php', 'search', 'Niccolò Machiavelli')

    returns
    

    https://en.wikipedia.org/w/index.php?search=Niccol%C3%B2+Machiavelli

    If you are writing a custom column book details URL template then use $item_name or field('item_name') to obtain the value of the field that was clicked on. Example: if Niccolò Machiavelli was clicked then you can construct the URL using:
    

    make_url('https://en.wikipedia.org/w/index.php', 'search', $item_name)

    See also the functions make_url_extended, query_string and encode_for_url.
  • make_url_extended(...) -- this function is similar to make_url but gives you more control over the URL components. The components of a URL are
    
    scheme:://authority/path?query string.
    
    See Uniform Resource Locater on Wikipedia for more detail.
    
    The function has two variants:
    

    make_url_extended(scheme, authority, path, [query_name, query_value]+)

    and
    

    make_url_extended(scheme, authority, path, query_string)

    This function returns a URL constructed from the scheme, authority, path, and either the query_string or a query string constructed from the query argument pairs. The authority can be empty, which is the case for calibre scheme URLs. You must supply either a query_string or at least one query_name, query_value pair. If you supply query_string and it is empty then the resulting URL will not have a query string section.
    
    Example 1: constructing a Wikipedia search URL for the author Niccolò Machiavelli:
    

    make_url_extended('https', 'en.wikipedia.org', '/w/index.php', 'search', 'Niccolò Machiavelli')

    returns
    

    https://en.wikipedia.org/w/index.php?search=Niccol%C3%B2+Machiavelli

    See the query_string() function for an example using make_url_extended() with a query_string.
    
    If you are writing a custom column book details URL template then use $item_name or field('item_name') to obtain the value of the field that was clicked on. Example: if Niccolò Machiavelli was clicked on then you can construct the URL using :
    

    make_url_extended('https', 'en.wikipedia.org', '/w/index.php', 'search', $item_name')

    See also the functions make_url, query_string and encode_for_url.
  • query_string([query_name, query_value, how_to_encode]+)-- returns a URL query string constructed from the (query_name, query_value, how_to_encode) triads. A query string is a series of items where each item looks like query_name=query_value where query_value is URL-encoded as instructed. The query items are separated by '&' (ampersand) characters.
    
    If how_to_encode is 0 then query_value is encoded and spaces are replaced with '+' (plus) signs. If how_to_encode is 1 then query_value is encoded with spaces replaced by %20. If how_to_encode is 2 then query_value is returned unchanged; no encoding is done and spaces are not replaced. If you want query_value not to be encoded but spaces to be replaced then use the re function, as in re($series, ' ', '%20')
    
    You use this function if you need specific control over how the parts of the query string are constructed. You could then use the resultingquery string in make_url_extended, as in
    

    make_url_extended(
           'https', 'your_host', 'your_path',
           query_string('encoded', 'Hendrik Bäßler', 0, 'unencoded', 'Hendrik Bäßler', 2))

    giving you the the URL that is invalid because of the space after "Hendrik"
    

    https://your_host/your_path?encoded=Hendrik+B%C3%A4%C3%9Fler&unencoded=Hendrik Bäßler

    You must have at least one query_name, query_value, how_to_encode triad, but can have as many as you wish.
    
    The returned value is a URL query string with all the specified items, for example: name1=val1[&nameN=valN]*. Note that the '?' path / query string separator is not included in the returned result.
    
    If you are writing a custom column book details URL template then use $item_name or field('item_name') to obtain the unencoded value of the field that was clicked. You also have item_value_quoted where the value is already encoded with plus signs replacing spaces, and item_value_no_plus where the value is already encoded with %20 replacing spaces.
    
    See also the functions make_url, make_url_extended and encode_for_url.
  • encode_for_url(value, use_plus) -- returns the value encoded for use in a URL as specified by use_plus. The value is first URL-encoded. Next, if use_plus is 0 then spaces are replaced by '+' (plus) signs. If it is 1 then spaces are replaced by %20.
    
    If you do not want the value to be encoding but to have spaces replaced then use the re function, as in re($series, ' ', '%20')
    
    See also the functions make_url, make_url_extended and query_string.

[Comfy.n]

Interesting updates, thank you! I noticed that running:

program: make_url_extended('https', 'en.wikipedia.org', '/w/index.php', 'search', field('author'))

returns slightly different URLs on Template Tester and L&F -> BD -> Custom search URL -> Template dialog:

on Template tester (good link):

https://en.wikipedia.org/w/index.php...B2+Machiavelli

on new Template dialog for custom search:

https://en.wikipedia.org/w/index.php...%2BMachiavelli

[chaley]

Originally Posted by Comfy.n

Interesting updates, thank you! I noticed that running:

program: make_url_extended('https', 'en.wikipedia.org', '/w/index.php', 'search', field('author'))

returns slightly different URLs on Template Tester and L&F -> BD -> Custom search URL -> Template dialog:

on Template tester (good link):

https://en.wikipedia.org/w/index.php...B2+Machiavelli

on new Template dialog for custom search:

https://en.wikipedia.org/w/index.php...%2BMachiavelli

To be clear, you aren't testing the new book details custom column URL feature. Instead you are testing the long-existing L&F / Book details custom authors rule.

The problem is that the L&F / Book details dialog *knows* it is for authors only, and in fact doesn't even have the rest of the metadata. Historically, because L&F / Book details knows it is for authors, to simplify things for the expected use case the author is URL-encoded before it gets to the template. Thas is why the template

https://www.wikipedia.org/w/index.php?search={author}

works there; the interpolated 'author' is already URL-encoded. This can't be changed without breaking existing custom author links. Note also that you can't make decisions in the template such as "if author == 'John Doe' then" because the author will already be encoded as 'John+Doe'.

If you want to use make_url_extended() there then you must use the variant where you control whether the query value is encoded (again) during URL construction.

program:
	qs = query_string('search', $author, 2);
	make_url_extended('https', 'wikepedia.com', '/w/index.php', qs)

You will see in the L&F / Book details template tester result that the author, Niccolò Machiavelli, is URL-encoded. In the normal template tester, where the original metadata is being used, it won't be URL-encoded.

This discrepency is why the new custom column link generation feature offers the item value unencoded, URL-encoded with '+' for spaces, and URL-encoded with %20 for spaces.

Bottom line: you can't use the same template in L&F/ BD as you use in the new custom column web link feature.

[ownedbycats]

Originally Posted by Comfy.n

https://en.wikipedia.org/w/index.php...%2BMachiavelli

%25 is encoding for... percent sign?

EDIT: Explain above.

[chaley]

Originally Posted by ownedbycats

%25 is encoding for... percent sign. I feel like this is a messup.

No, it isn't. See my previous response for why.

[Comfy.n]

At the moment I'm trying to figure how a deepseek query could be made using just a url. Not sure it's possible. :\

This add-on works fine on browser context menu:

https://addons.mozilla.org/en-US/fir.../ask-deepseek/

Spoiler:

content.js:

function waitForElement(selector) {
  return new Promise(resolve => {
    const interval = setInterval(() => {
      const element = document.querySelector(selector);
      if (element) {
        clearInterval(interval);
        resolve(element);
      }
    }, 500);
  });
}

(async () => {
  try {
    const input = await waitForElement("textarea");
    const text = await navigator.clipboard.readText();
    
    input.value = text;
    input.dispatchEvent(new Event("input", { bubbles: true }));
    
    setTimeout(() => {
      input.dispatchEvent(new KeyboardEvent("keydown", {
        key: "Enter",
        code: "Enter",
        bubbles: true
      }));

      waitForElement(".assistant-message").then(response => {
        const memeBtn = document.createElement("button");
        memeBtn.textContent = " 🖼️ Meme";
        memeBtn.onclick = () => {
          window.open(`https://imgflip.com/memegenerator?text=${encodeURIComponent(response.innerText)}`);
        };
        
        const speakerBtn = document.createElement("button");
        speakerBtn.textContent = " 🔊 Read";
        speakerBtn.onclick = () => {
          const speech = new SpeechSynthesisUtterance(response.innerText);
          window.speechSynthesis.speak(speech);
        };
        
        response.prepend(memeBtn, speakerBtn);
        response.scrollIntoView({ behavior: "smooth" });
      });
    }, 1000);
    
  } catch (error) {
    console.error("Error:", error);
  }
})();

background.js:


// Parent menus
browser.contextMenus.create({
  id: "ask-menu",
  title: "Ask DeepSeek",
  contexts: ["selection"]
});

browser.contextMenus.create({
  id: "translate-menu",
  title: "Translate with DeepSeek",
  contexts: ["selection"]
});

browser.contextMenus.create({
  id: "support-menu",
  title: "☕ Buy Me a Coffee",
  contexts: ["all"] // Shows even without text selection
});

// Ask DeepSeek submenu
const askOptions = [
  { id: "ask-opinion", title: "🤔 What do you think of '%s'?" },
  { id: "ask-factcheck", title: "🔍 Fact check: '%s'" },
  { id: "ask-eli5", title: "🧒 ELI5: '%s'" },
  { id: "ask-debate", title: "⚖️ Debate: '%s'" },
  { id: "ask-tldr", title: "📌 TL;DR: '%s'" },
  { id: "ask-istrue", title: "✅ Is this true?: '%s'" },
  { id: "ask-questions", title: "❓ Question Me: '%s'" } // New feature
];

// Translate submenu
const languages = [
  { id: "en", title: "🇬🇧 English" },
  { id: "zh", title: "🇨🇳 Chinese" },
  { id: "hi", title: "🇮🇳 Hindi" },
  { id: "es", title: "🇪🇸 Spanish" },
  { id: "fr", title: "🇫🇷 French" },
  { id: "ar", title: "🇸🇦 Arabic" },
  { id: "bn", title: "🇧🇩 Bengali" },
  { id: "ru", title: "🇷🇺 Russian" },
  { id: "pt", title: "🇵🇹 Portuguese" },
  { id: "id", title: "🇮🇩 Indonesian" },
  { id: "ur", title: "🇵🇰 Urdu" },
  { id: "de", title: "🇩🇪 German" },
  { id: "ja", title: "🇯🇵 Japanese" },
  { id: "sw", title: "🇹🇿 Swahili" },
  { id: "mr", title: "🇮🇳 Marathi" },
  { id: "it", title: "🇮🇹 Italian" }
];

// Build menus
askOptions.forEach(option => {
  browser.contextMenus.create({
    id: option.id,
    title: option.title,
    contexts: ["selection"],
    parentId: "ask-menu"
  });
});

languages.forEach(lang => {
  browser.contextMenus.create({
    id: `translate-${lang.id}`,
    title: lang.title,
    contexts: ["selection"],
    parentId: "translate-menu"
  });
});

// Click handler
browser.contextMenus.onClicked.addListener(async (info, tab) => {
  // Handle "Buy Me a Coffee"
  if (info.menuItemId === "support-menu") {
    browser.tabs.create({ url: "https://buymeacoffee.com/technologicalpixel" });
    return;
  }

  const text = info.selectionText;
  let question;

  // Ask DeepSeek options
  if (info.menuItemId.startsWith("ask-")) {
    switch(info.menuItemId) {
      case "ask-opinion": question = `What do you think of "${text}"?`; break;
      case "ask-factcheck": question = `Fact-check this: "${text}". Provide sources.`; break;
      case "ask-eli5": question = `Explain this like I'm 5: "${text}". Use simple words.`; break;
      case "ask-debate": question = `Argue both FOR and AGAINST: "${text}".`; break;
      case "ask-tldr": question = `Summarize this in 3 bullet points: "${text}".`; break;
      case "ask-istrue": question = `Is this true? Explain why: "${text}".`; break;
      case "ask-questions": question = `Generate 10 thought-provoking questions about: "${text}".`; break;
    }
  }

  // Translate options
  if (info.menuItemId.startsWith("translate-")) {
    const lang = languages.find(l => `translate-${l.id}` === info.menuItemId);
    question = `Translate this to ${lang.title}: "${text}"`;
  }

  // Copy & open DeepSeek
  if (question) {
    await navigator.clipboard.writeText(question);
    const deepseekTab = await browser.tabs.create({
      url: "https://chat.deepseek.com/",
      active: true
    });

    setTimeout(() => {
      browser.tabs.executeScript(deepseekTab.id, { file: "content.js" });
    }, 1000);
  }
});