Inconsistent Folder Names, Can't Figure it out

[Trel]

I'm trying to figure out why Calibre is inconsistantly naming folders in my library.

For example, I have a series of 21 books. I bulk edited the Author and Author Sort name to ensure they were ALL identical.

I exported everything to files, and then nuked my library folder (to ensure that there was nothing residual).

I then re-imported all the books.

What I got was

Two folders in my library
"Frank; Brian Herbert; Kevin J. Ande"
and
"Frank; Brian Herbert; Kevin J. Ander"

The first has the first three books, and the other has the other eighteen.

All metadata within Calibre shows it the same still. I can't figure out why it's doing this. I considered overall filename length, but it can't be that, because there are some longer titles in the second folder.

Anyone have any idea?

[eschwartz]

Why, oh why, are you using a semicolon to separate authors, when calibre uses "&".
Why, oh why, are you using a semicolon to ALSO separate two author first names when they have the same last name?


And calibre is certainly trimming the filenames, since I assume the author (singular) "Frank; Brian Herbert; Kevin J. Ander" is actually "Frank; Brian Herbert; Kevin J. Anderson".
(But it should be "Frank Herbert & Brian Herbert & Kevin J. Anderson".)

Now, I would expect, if the metadata as definitely the same, that calibe would rename the author folder. It is possible that when shortening the path, it keeps old folders (I doubt it), but in any even, if you temporarily rename the author, it will recalculate the values and put them all together.

EDIT: well, chaley seems to be implying that calibre does, indeed, create the entire path independently, and split it arbitrarily.
Can't say I'm too terribly surprised though.

[Trel]

Originally Posted by eschwartz

Why, oh why, are you using a semicolon to separate authors, when calibre uses "&".
Why, oh why, are you using a semicolon to ALSO separate two author first names when they have the same last name?

I didn't originally name these. I can change them to "&" and make the way they're worded more sane, but doesn't explain what's happening here.

(But for the record, on the actual cover, it's listed as "Frank Herbert, Brian Herbert, And Kevin J. Anderson")

1. Bulk rename all 21 books, setting both Author and Author Sort (there should be no other author field, correct?)

2. Export all the books. (Result here is a single folder for the author)

3. Completely remove the Calibre library (the entire folder). There should be no residual metadata anywhere.

4. Import the books from what I did in #2.

5. That's how I ended up where I am with the two folders. (Originally it was just a single book in a different folder, now it's three)

And calibre is certainly trimming the filenames, since I assume the author (singular) "Frank; Brian Herbert; Kevin J. Ander" is actually "Frank; Brian Herbert; Kevin J. Anderson".
(But it should be "Frank Herbert & Brian Herbert & Kevin J. Anderson".)

Agreed, however, it's not doing it consistently. For example, there's a title that's ~5 characters longer than any of the other titles. It's listed under the folder with the longer name.

Now, I would expect, if the metadata as definitely the same, that calibe would rename the author folder. It is possible that when shortening the path, it keeps old folders (I doubt it), but in any even, if you temporarily rename the author, it will recalculate the values and put them all together.

That's why I exported, nuked the library, and reimported. I still get the two folders, so I don't know what's going on here.

EDIT: I renamed them using "&" and it looks like it just uses the first author for the folder name now, though, it is using a single folder. I can live with that.

[BetterRed]

Originally Posted by Trel

EDIT: I renamed them using "&" and it looks like it just uses the first author for the folder name now, though, it is using a single folder. I can live with that.

Correct - the other authors are in the database, but they don't need their own folder. If you added a book where "Kevin J. Anderson" was the sole author you would have a folder by that name with that one book in it.

BR

[Trel]

I'm still curious how (and why) it was splitting the books between the two folders I mentioned.

It gets weirder still.

Without the author fix, it wasn't consistently splitting the books between the folders the same way each time.

[theducks]

Originally Posted by Trel

I'm still curious how (and why) it was splitting the books between the two folders I mentioned.

It gets weirder still.

Without the author fix, it wasn't consistently splitting the books between the folders the same way each time.

"Frank; Brian Herbert; Kevin J. Ande" is one Author as far as Calibre is concerned
& tells Calibre to parse these
There is a tweak that sets the additional names to parse DO NOT try and force your ; in there

# Splitting multiple author names

authors_split_regex = '(?i),?\\s+(and|with)\\s+'

[chaley]

Originally Posted by Trel

I'm still curious how (and why) it was splitting the books between the two folders I mentioned.

It gets weirder still.

Without the author fix, it wasn't consistently splitting the books between the folders the same way each time.

The length of the path to a book is limited, especially in windows. If the "standard" path is too long, calibre will shorten it by removing characters from each path segment (author, title, book). Because the number of characters to remove depends on the length of the path, the resulting segments will vary from one to another.

[Trel]

Originally Posted by chaley

The length of the path to a book is limited, especially in windows. If the "standard" path is too long, calibre will shorten it by removing characters from each path segment (author, title, book). Because the number of characters to remove depends on the length of the path, the resulting segments will vary from one to another.

That's the logic I was expecting, but what I got ended up with final paths like this

c:\longer author name\book (-----) 1\book (-----) 1.epub
c:\longer author name\book (---) 2\book (---) 2.epub
c:\longer author name\book (-----------) 3\book (-----------) 3.epub
and
c:\author name\book (---) 4\book (---) 4.epub
c:\author name\book (---------) 5\book (---------) 5.epub


Essentially the book with the longest title (and as such with the longest path) was put into the folder with the author's name truncated to MORE characters.

That's why I posted to ask to begin with. If it was simply that the longer titles were in the folder with the author's name truncated smaller, that would have made sense, so I'd have understood what was happening.

[chaley]

The code calibre uses to shorten a path is under the spoiler. You will see that calibre shortens each component by a % of the amount needed, thus the longest components are shortened more.

Spoiler:

def shorten_component(s, by_what):
    l = len(s)
    if l < by_what:
        return s
    l = (l - by_what)//2
    if l <= 0:
        return s
    return s[:l] + s[-l:]

def shorten_components_to(length, components, more_to_take=0, last_has_extension=True):
    filepath = os.sep.join(components)
    extra = len(filepath) - (length - more_to_take)
    if extra < 1:
        return components
    deltas = []
    for x in components:
        pct = len(x)/float(len(filepath))
        deltas.append(int(ceil(pct*extra)))
    ans = []

    for i, x in enumerate(components):
        delta = deltas[i]
        if delta > len(x):
            r = x[0] if x is components[-1] else ''
        else:
            if last_has_extension and x is components[-1]:
                b, e = os.path.splitext(x)
                if e == '.':
                    e = ''
                r = shorten_component(b, delta)+e
                if r.startswith('.'):
                    r = x[0]+r
            else:
                r = shorten_component(x, delta)
            r = r.strip()
            if not r:
                r = x.strip()[0] if x.strip() else 'x'
        ans.append(r)
    if len(os.sep.join(ans)) > length:
        return shorten_components_to(length, components, more_to_take+2)
    return ans

[Trel]

And that fully explains it.

Mystery solved, as well as problem solved (by correcting the author field to begin with)

[jecilop]

I just skipped the tech talk on this one to answer...based on what you said..you exported the files in a group and not the entire folders individually.

It sounds as if you didn't update the metadata in the files. If you are just importing individual files, it will pull from the metadata in them. If it was never updated, then you won't see your fixes. Calibre doesn't do this automatically. You have to update the metadata for it to be "sealed" in the file.

I hope this helps.