Path separator not working in CC templates?

DavidTC · 08-28-2016, 12:05 AM

So, here's a bug that I figured out how to get around while typing it up, (I was doing things the hard way, and now just use the Calibre path.), but it still seems like a bug and I feel I should report.

So, I have a field called #savepath that contains '{#genre}/{title} - {id}'.

If I put {#savepath} as the save template in Calibre, a book, for example, gets saved correctly to 'Science Fiction/Douglas Adams' Starship Titanic - 3209.epub' pretty much everywhere, including CC if it's using the Calibre file path. It works fine.

However, putting {#savepath} as the *CC* template, and using that, results in a file in the top CC directory named 'Science Fiction_Douglas Adams' Starship Titanic - 3209.pub'.

It doesn't notice I'm trying to make a subdirectory. And then, I presume, is forced to change the / it cannot put in a filename to a _.

This behavior, I guess, would be reasonable if CC just didn't want to put books in subdirectories, but it will happily put them there if *Calibre* tells it to. So I suspect it's a bug.

The 'Remove accents' option is off, just in case anyone is wondering if that's relevant.

I only had time to test this using the wireless device connection.

DoctorOhh · 08-28-2016, 05:20 AM

Quote:

Originally Posted by DavidTC

So, here's a bug that I figured out how to get around while typing it up, (I was doing things the hard way, and now just use the Calibre path.), but it still seems like a bug and I feel I should report.

So, I have a field called #savepath that contains '{#genre}/{title} - {id}'.

I may be reading this CC FAQ entry wrong, but I don't believe this is a bug. CC's template does not have the full feature set of calibre and I don't think what you're attempting is supported in the manner your trying to accomplish it.

chaley · 08-28-2016, 05:31 AM

Actually, the fact that it works in a calibre template is probably a bug.

Calibre removes slashes in template items, with the exception of a leading slash in a tag. The probable bug is that composite columns (columns built from other columns) don't have that processing (slash removal) applied to them. There isn't a good reason why composites should be treated differently from custom text or standard columns. However, I am not going to fix that probable bug because it has been there too long and people (like you) might depend upon it.

I am not willing to change CC's template processor to allow slashes in fields by default because such a change could break things. Consider a title like "My Magazine 2011/10". That slash should not (must not?) indicate a new directory level. Neither should a series like "Coffee/Tea". However, I am willing to add a format specifier that tells CC to not remove slashes for that template item, something like

Code:

{#mycolumn:allowslashes}

If #mycolumn contains "aaa/bbb", title = "Foo" and authors = "Blogs, Joe" then the following template

Code:

{#mycolumn:allowslashes}/{title} - {author}

would produce

Code:

aaa/bbb/Foo - Blogs, Joe

Leaving out the allowslashes format specifier would produce

Code:

aaa_bbb/Foo - Blogs, Joe

if #mycolumn is empty then the template would produce

Code:

Foo - Blogs, Joe

Do note that using calibre's template will probably create problems if you use CC's content server or cloud connections. See How do I get the same file names in CC no matter which connection I use? for more information.

EDIT: also note that if you use a composite column in a CC template and also use the cloud connection then you almost certainly want to use the calibre plugin that stores the composite column values for CC's use. See When I look at book details using the cloud connection my custom columns "built from other columns" are missing. Why? for some explanation.

DavidTC · 08-28-2016, 07:41 PM

Calibre removes slashes in template items, with the exception of a leading slash in a tag.

I've been testing this since you posted, and I am in shock.

The behavior I was expecting, in fact, the behavior that I bet everyone was suspecting, that when you fill in a template, that template should come up with the same result everywhere. And then, *at the end*, if it's saving a file, it parse the directories and, at the end, right before it saves, it replaces invalid characters.

And I think most of us assume that if the result of the template evaluation is A/B, it would treat that the same if it got that from {#ab} or {#a}/{#b} or some program.

But apparently getting a metadata field, either with {} or field(), has some magical string processing during it that behaves differently if you're calculating a save template, and only a save template.

Not only is this an extremely inconsistent thing, it sorta breaks the entire concept of the language, if completely identical pieces of code can give different results depending on where they are.

It also appears to not be documented anywhere. The documentation says, literally, 'field(name) – returns the metadata field named by name.', This documentation is wrong.

Instead, what field() does is 'returns the metadata field named be name, unless it's called inside a save template, at which point it munges the metadata field to be more like a file name before returning it.' And that's just field(), there's probably others non-documented...how does lookup() behave? Does this affect first_non_empty()? author_links()?

Also, does raw_field() get the value *without* doing that? I probably should check, but I don't have the time, and having to constantly restart Calibre to change the save template is annoying. (Which is why I put that in a field in the first place.) Is there even a *way* to get the original field?

Seriously, the solution to 'newbies might have a / in their tags and accidentally make directories' was to change how some part of the language worked in a certain place. (And then not mention it anywhere.) Instead of just adding a strip_nonfile_chars() function and mentioning people should use {#fieldname:strip_nonfile_chars()} in the intro template documentation if that happens?

And at this point...yes, I know some people probably have existing stuff that assumes it, and oh noes, it can't ever be changed....but wow, this is such a huge and weird behavioral inconsistency that I argue it really should be dealt with. Give people a checkbox on the save template screen if they want the old way.

But that's not really you, I need to talk to kovidgoyal about that, I guess?

chaley · 08-29-2016, 03:07 AM

Moderator Notice
This is no longer a thread about CC. I am copying it to the calibre forum where I will respond.

08-28-2016, 12:05 AM	#1
DavidTC Connoisseur Posts: 77 Karma: 10 Join Date: Sep 2011 Device: Nook, Boox C67ML	Path separator not working in CC templates? So, here's a bug that I figured out how to get around while typing it up, (I was doing things the hard way, and now just use the Calibre path.), but it still seems like a bug and I feel I should report. So, I have a field called #savepath that contains '{#genre}/{title} - {id}'. If I put {#savepath} as the save template in Calibre, a book, for example, gets saved correctly to 'Science Fiction/Douglas Adams' Starship Titanic - 3209.epub' pretty much everywhere, including CC if it's using the Calibre file path. It works fine. However, putting {#savepath} as the CC template, and using that, results in a file in the top CC directory named 'Science Fiction_Douglas Adams' Starship Titanic - 3209.pub'. It doesn't notice I'm trying to make a subdirectory. And then, I presume, is forced to change the / it cannot put in a filename to a _. This behavior, I guess, would be reasonable if CC just didn't want to put books in subdirectories, but it will happily put them there if Calibre tells it to. So I suspect it's a bug. The 'Remove accents' option is off, just in case anyone is wondering if that's relevant. I only had time to test this using the wireless device connection.

08-28-2016, 05:31 AM	#3
chaley Grand Sorcerer Posts: 12,423 Karma: 8012664 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Actually, the fact that it works in a calibre template is probably a bug. Calibre removes slashes in template items, with the exception of a leading slash in a tag. The probable bug is that composite columns (columns built from other columns) don't have that processing (slash removal) applied to them. There isn't a good reason why composites should be treated differently from custom text or standard columns. However, I am not going to fix that probable bug because it has been there too long and people (like you) might depend upon it. I am not willing to change CC's template processor to allow slashes in fields by default because such a change could break things. Consider a title like "My Magazine 2011/10". That slash should not (must not?) indicate a new directory level. Neither should a series like "Coffee/Tea". However, I am willing to add a format specifier that tells CC to not remove slashes for that template item, something like Code: {#mycolumn:allowslashes} If #mycolumn contains "aaa/bbb", title = "Foo" and authors = "Blogs, Joe" then the following template Code: {#mycolumn:allowslashes}/{title} - {author} would produce Code: aaa/bbb/Foo - Blogs, Joe Leaving out the allowslashes format specifier would produce Code: aaa_bbb/Foo - Blogs, Joe if #mycolumn is empty then the template would produce Code: Foo - Blogs, Joe Do note that using calibre's template will probably create problems if you use CC's content server or cloud connections. See How do I get the same file names in CC no matter which connection I use? for more information. EDIT: also note that if you use a composite column in a CC template and also use the cloud connection then you almost certainly want to use the calibre plugin that stores the composite column values for CC's use. See When I look at book details using the cloud connection my custom columns "built from other columns" are missing. Why? for some explanation. Last edited by chaley; 08-28-2016 at 05:37 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Importing tags with ; as separator	Daigomi	Library Management	2	05-13-2015 09:33 AM
Paragraph separator	roger64	Editor	3	04-26-2014 03:17 AM
PRS-350 About the epub custom font path And how to know the path?	52manhua	Sony Reader Dev Corner	6	12-20-2013 01:28 AM
<hr> as section separator?	orange!	ePub	13	09-17-2013 03:31 PM
Subgroup Separator	ilovejedd	Library Management	8	03-03-2012 10:55 AM

08-28-2016, 07:41 PM	#4
DavidTC Connoisseur Posts: 77 Karma: 10 Join Date: Sep 2011 Device: Nook, Boox C67ML	Calibre removes slashes in template items, with the exception of a leading slash in a tag. I've been testing this since you posted, and I am in shock. The behavior I was expecting, in fact, the behavior that I bet everyone was suspecting, that when you fill in a template, that template should come up with the same result everywhere. And then, at the end, if it's saving a file, it parse the directories and, at the end, right before it saves, it replaces invalid characters. And I think most of us assume that if the result of the template evaluation is A/B, it would treat that the same if it got that from {#ab} or {#a}/{#b} or some program. But apparently getting a metadata field, either with {} or field(), has some magical string processing during it that behaves differently if you're calculating a save template, and only a save template. Not only is this an extremely inconsistent thing, it sorta breaks the entire concept of the language, if completely identical pieces of code can give different results depending on where they are. It also appears to not be documented anywhere. The documentation says, literally, 'field(name) – returns the metadata field named by name.', This documentation is wrong. Instead, what field() does is 'returns the metadata field named be name, unless it's called inside a save template, at which point it munges the metadata field to be more like a file name before returning it.' And that's just field(), there's probably others non-documented...how does lookup() behave? Does this affect first_non_empty()? author_links()? Also, does raw_field() get the value without doing that? I probably should check, but I don't have the time, and having to constantly restart Calibre to change the save template is annoying. (Which is why I put that in a field in the first place.) Is there even a way to get the original field? Seriously, the solution to 'newbies might have a / in their tags and accidentally make directories' was to change how some part of the language worked in a certain place. (And then not mention it anywhere.) Instead of just adding a strip_nonfile_chars() function and mentioning people should use {#fieldname:strip_nonfile_chars()} in the intro template documentation if that happens? And at this point...yes, I know some people probably have existing stuff that assumes it, and oh noes, it can't ever be changed....but wow, this is such a huge and weird behavioral inconsistency that I argue it really should be dealt with. Give people a checkbox on the save template screen if they want the old way. But that's not really you, I need to talk to kovidgoyal about that, I guess?

08-29-2016, 03:07 AM	#5
chaley Grand Sorcerer Posts: 12,423 Karma: 8012664 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Moderator Notice This is no longer a thread about CC. I am copying it to the calibre forum where I will respond.

Advert

Advert