Why calibre transforms " : " and " .. " into just " _ "??. File name

[Latios]

Hi,

I am using the tool save to disc. And everytime theres a colon ( : ) or two dots together (..), calibre puts an underline. Let me give you two examples:

In both examples, the template for name filing is: {author_sort}. {title}

1. A book with title name "Grammar test: Level 4" and author "John Smith".
The file's name of this saved-to-disc ebook becomes:
Smith, John. Grammar test_Level 4

Why that underline instead of the colon???


2. A book with the title name "The greatest" and author "Tim R. Swan".
The file's name of this saved-to-disc ebook becomes:
Swan, Tim R_ The greatest

Why that underline instead of to following dots???


Can anyone help me to fix it?

[davidfor]

A colon is an illegal character in a file name under Windows. So, calibre can't create a file with a colon in its name and replaces it with an underscore. And it does this for any invalid character. For compatibility across all the OSes that calibre runs, it will probably use the same rules.

The trailing dot is related. For the examples I have seen, it is the last character before the dot separating the extension. Windows seems to be OK with this, so it might be another OS. Or maybe it is because it would be very confusing and cause all sorts of problems coding around having the dot at the end of a file name. In the example you gave, it isn't strictly necessary, but it probably is following a standard rule for file names.

[Latios]

Originally Posted by davidfor

A colon is an illegal character in a file name under Windows. So, calibre can't create a file with a colon in its name and replaces it with an underscore. And it does this for any invalid character. For compatibility across all the OSes that calibre runs, it will probably use the same rules.

The trailing dot is related. For the examples I have seen, it is the last character before the dot separating the extension. Windows seems to be OK with this, so it might be another OS. Or maybe it is because it would be very confusing and cause all sorts of problems coding around having the dot at the end of a file name. In the example you gave, it isn't strictly necessary, but it probably is following a standard rule for file names.

I use windows.

Ok, about the colon. I understand now. There is no file in the whole windows that can have a colon in its name. Got it.

But about the two dots or trailing dot " .. " , why calibre changes it for an underscore??
It is actually possible to create a file with " .. " in the file name in windows.

[BetterRed]

Originally Posted by Latios

It is actually possible to create a file with " .. " in the file name in windows.

Calibre can run on Windows, MacOS, and Linux; each of which support several file systems, and they in turn have varying file/folder naming conventions, some of which are user configurable. One of those file systems does not allow '..' in file and folder names - I can't recall which one.

Another consideration is that calibre libraries should be portable between different Operating System and File System combinations.

BR

[Latios]

I just found out something that does not make sense :S

About when calibre changes ".." for "_".

In the calibre library I have this ebook with this metadata:
Author: Jens R. Hope
Year: 2000
Title: Water.
(please, pay special attention that the title ends up with a dot ".")

Then I apply this code:

{author_sort}. {title}{:'format_date (raw_field ('pubdate'),'yyyy')'|. |}

And I get this name:
Hope, Jens R_ Water.. 2000

As before, with {author_sort}, calibre transforms ".." into "_".
BUT, AND HERE IT COMES THE CONTROVERSY: after {title}, calibre leaves ".." untouched.

So it looks that IT IS possible for calibre to filename something with ".." in it.

However, I've been playing with the code using many variations with for example "|. |" .. and I can't find a way to get this:
Hope, Jens R.. Water.. 2000

How can I make calibre to leave untouched ".." also after {author_sort}??

[DNSB]

Originally Posted by Latios

So it looks that IT IS possible for calibre to filename something with ".." in it.

However, I've been playing with the code using many variations with for example "|. |" .. and I can't find a way to get this:
Hope, Jens R.. Water.. 2000

How can I make calibre to leave untouched ".." also after {author_sort}??

Sorry but that's not a '..' in the sense you are meaning. It is a single '.' with the '.' the operating system (going back to CP/M and inherited by MS-DOS)* uses to distinguish the two parts of the filename (filename.ext) appearing as the second '.'. The filename was stored as 8 bytes of a filename and 3 bytes for the extension.

* Actually, going back even further since Gary Kildall of CP/M borrowed his filenaming conventions.

[theducks]

..\ tells the OS to go up 1 level closer to the root (Win and Linux)
.\ tells the OS to include the current directory name. a dot also separates the files name from the extension. and directories CAN have a dot part (just not common usage in windows)
oh
.filename in Linux are hidden

[Latios]

Originally Posted by DNSB

Sorry but that's not a '..' in the sense you are meaning. It is a single '.' with the '.' the operating system (going back to CP/M and inherited by MS-DOS)* uses to distinguish the two parts of the filename (filename.ext) appearing as the second '.'. The filename was stored as 8 bytes of a filename and 3 bytes for the extension.

* Actually, going back even further since Gary Kildall of CP/M borrowed his filenaming conventions.

Originally Posted by theducks

..\ tells the OS to go up 1 level closer to the root (Win and Linux)
.\ tells the OS to include the current directory name. a dot also separates the files name from the extension. and directories CAN have a dot part (just not common usage in windows)
oh
.filename in Linux are hidden

Thanks, but with that explanation, it looks that calibre should never allowed ".." to be in a filename. But it does.

So, I still dont understand how calibre DOES allow this (in blue) :
Hope, Jens R_ Water..2000
(this is a real example)

But it DOES NOT allows this (in red) :
Hope, Jens R.. Water.. 2000


when using this template code:

{author_sort}. {title}{:'format_date (raw_field ('pubdate'),'yyyy')'|. |}

with this metadata:

Author: Jens R. Hope
Year: 2000
Title: Water.
(please, pay special attention that the title ends up with a dot ".")

[theducks]

..\

note the slash

so you .. in the middle must be OK (Kovid would no mess up on things like this)

there are other special characters that mean stuff to different OS.
Kovid does KISS: one code to rule them all (even if it might be allowed buy x)

[BetterRed]

@Latios would seem to have a point. If an Author's name ends in a dot (.) it seems to get a different treatment to a Title that ends in a dot (.),

On Windows/NTFS I created this book



Then with these StD basic settings



... and this template

{title}. {author_sort}. {:'format_date (raw_field ('pubdate'),'yyyy')'|. |}

I get this file name

Hello_ Better Red_ . 2000.epub

And with this template (the one Latios posted)

{author_sort}. {title}{:'format_date (raw_field ('pubdate'),'yyyy')'|. |}

I get this file name

Better Red_ Hello.. 2000.epub

Curious : @Latios - why do you want dots in the file names?

BTW, in my Test library (which is the one I was using) I don't swap first and last names to create Author Sort.

BR

[AnotherCat]

Originally Posted by Latios

...So, I still dont understand how calibre DOES allow this (in blue) :
Hope, Jens R_ Water..2000
(this is a real example)

But it DOES NOT allows this (in red) :
Hope, Jens R.. Water.. 2000...

What you see as 2 periods in your blue example is interpreted as the first being a period as part of the filename and the second being the extension separator - so interpreted in the filename as being one period.

While two periods are allowed within filenames (as in your red example) by Windows (and I think by Linux too) I have never seen them so used and I suspect most people would do a double take if seen used by others; that if only because use of the second period normally adds nothing to the clarity of a filename (and, in Windows, it uses up one of the 260 path length characters total path and filename length for no purpose). A single period (and spaces) may add to the clarity of the filename (such as an author's name e.g. R. L. Stevenson)

Many people regard it as bad practice to use any periods (or spaces) at all other than for the extension separator. As far as Windows users go this is a likely mainly a hangover from original DOS file naming days. Also as theducks has pointed out single and double periods have a specific use when preceded by a \ (or/).

For myself I use the gamut of Windows allowed characters when naming files but I would not use double periods. When using spaces I normally consider whether their use may cause problems (e.g. a human typing a filename can accidently strike two spaces and not notice it - two spaces are allowed in Windows) in which case I use a Low Line instead.

[Latios]

Originally Posted by BetterRed

@Latios would seem to have a point. If an Author's name ends in a dot (.) it seems to get a different treatment to a Title that ends in a dot (.),

Thanks!

Originally Posted by BetterRed

Curious : @Latios - why do you want dots in the file names?

BTW, in my Test library (which is the one I was using) I don't swap first and last names to create Author Sort.

BR

Well.. it is not that I specially want two dots in the file name....
It's just that I do not want a "_"

basically because I have all my library superclean with all file names. and it bothers me like a lot to see that everytime I have an author with just the initial as second name (e.g: "R.", or "T."... etc)...
calibre decides to add a "_" at the end of the author_sort...
consequently making my library the opposite of clean.

More visually, I mean that by using this template code:

{author_sort}. {title}{:'format_date (raw_field ('pubdate'),'yyyy')'|. |}

I get lists of filenames like this:

Cox, Jen. Green world. 1948
Smith, Jogh. Crystal water. 2010
Tim, Sam R_ Glass. 1999
Murphy, Melissa. Manual. 2008

And that fuc%&! underscore is freaking me out!!!! LOL

So i just wanted to find a solution to create a template code to fix that, while at the same time leaving the rest as clean as it is....

call it OCD, yes.