Send to device puts most books in the wrong place, consistently

[Colonel Cathcart]

I've set things up to send books to: Books/{author_sort}/{title}
I'd like all book transfered to be placed under A Books sub-foder. Call it a books jail. I don't like to see multiple folders in the drive of the root, keep things tidy.
It's been like that since, practically, day one.

Now, after a while I haven't look at the folder structure, I found some books in the wrong place, for example:
E:\Bs\Rece\Cormen, Thomas H. & Leiserson, Cest, Ronald L. & Stein, Clifford\Introduction to Algorithms, Third Edition - Thoma E. Leiserson & Ronald L. Rivest & Clifford Stein.pdf
I have no idea where it got that "Rece" part. I have a custom column I used with a shelves value, and the value for this book is "Reference". Is it b/c it's very long so it truncates?

E:\Books\Fiction\Roth, Philip\plot against America, The - Philip Roth.kepub.epub
Most of the book are located in Books/{shelf}/{sort author}, although I'm not using this folder structure.
Also, If I delete the books, disconnect/refresh the Libra/Connect and transfer the books again they end up in the same place they were, ignoring my settings, it seems.

The plugboard only have a couple of default KoboTouchExtended entries to handle authors and authors_sort. Not something I made or changed.
I'm using KoboTouchExtended, the latest Calibre (5.35), everything is up to date. I tried reinstalling and the repair option - that didn't seem to change anything.

This is driving me crazy.

[Moonbase59]

I once fell for that and went (almost) crazy, too. Seems to happen with long filenames only, since apparently Kovid decided to "snip out" text in various places in order to be sure not to exceed filesystem name length restrictions.

Unfortunately, the logic behind it isn’t too clear (for me, at least).

I finally went for shorter filenames, like omitting subtitles and reducing the number of authors to one only (in the filenames; I actually have an anthology with 84 (!) authors, guess what the filename would look like…).

[davidfor]

Yes, it is the total length of the file name. The driver limits the total path length to 185 characters. This is because of the limitations in FAT32 for the path length and that some of it is used by by the device. Calibre uses the template to generate the file name and then fiddles with it if it is to long. That fiddling is by removing characters from the middle of different parts of the path. So, "Books" become "Bks" and "Reference" becomes "Rece".

Are you sure about the save template? You seem to be saying you are using:

Books/{author_sort}/{title}

But that doesn't match the examples you have given. You mention "Books/{shelf}/{sort author}" as where the books are. From that and the examples, it looks like you are using:

Books/{shelf}/{author_sort}/{title} - {author}

And with multiple authors, that will give produce a long path.

Are you looking at this in the driver configuration? If you change the save template on the "Sending books to devices" page, it doesn't change what the driver uses.

I don't know how to fix this, except to ask why does it matter? Kobo devices don't care. They use the path internally, but, don't really show it. They work off the metadata in the book. If you want to shorten the path, and you are using the shelf, then drop the "Books". That reduces the entries in the root directory on the device. Otherwise, you could do something like only use the first author instead of the full author_sort. Or drop the author from the filename part.

[JSWolf]

This is what I use.

books/{author} - {title}

I don't use subtitles under book.

[Colonel Cathcart]

Thanks, guys, for your support.
Yes, I'm sure I'm using "Books/{author_sort}/{title}" as the send-to. I copy-and-pasted it directly to the post from Calibre.
Your explanation makes (some) sense regarding the shortening of the file path. Why hasn't Calibre upped the limit of the file path length with the times and sticks to the ol' FAT32 is a question in itself.
But that doesn't explain what's going on with that other book I gave as example. In fact, most of the tens of books are stored under some shelf name: Fiction, Non-Fiction, Reference, Chess, Comics.
I have no idea why or where that came from, and as I wrote above, deleting the book and re-adding it sends it right back to the wrong place.

Edit: As a side question/observation. I've looked at the documentation, and it seems I should be able to use shorten to control how shortening the full path works so that it won't chomp off the "Books" folder.

[Moonbase59]

Thanks for the idea of using shorten! Sounds like I should rework my save templates…

I only ever let Calibre check the Books and Downloads folders on my devices, and my Books is divided into Fiction, Non-Fiction, News and Tests. Fiction and Non-Fiction also have additional series subfolders, so my Tolino will create series "collections" from these.

Guess how upset I was when Calibre insisted storing some books into "Ficion" instead of "Fiction". ;-) Thought I’d suddenly beamed to Spain… (Well, okay, it would have been "Ficción" but what the heck, hee hee.)

Also, cutting off from the base Books folder would be horrible, because the readers on my devices only start looking from Books downwards.

[Colonel Cathcart]

Originally Posted by Moonbase59

Thanks for the idea of using shorten! Sounds like I should rework my save templates…

While you're at it, there's also strlen, so you might want to do something even smarter, with cmps, ifs, and what nots.

Still scratching my head here regarding why these shelves folder is created.

[Moonbase59]

Some Kobo expert should chime in here, I think. Maybe the Kobo behaves somehow like my Tolino, auto-creating "collections" (or "shelves") for each folder beneath "Books"?

[Colonel Cathcart]

Sigh, I think I found the culprit. The KoboTouchExtended plugin has an option of its own for file name template. I must've set it up months ago and forgot about it completely.

[ownedbycats]

I notice you have multiple authors listed that pushes it past the char limit. Try using {author_sort:sublist(0,1,&)} in your path template to list only the first author.

[Colonel Cathcart]

Thank you for your suggestion. I'll need to do some experimenting.
It's a bit of a shame there's no sticky thread about file naming templates people use.

[davidfor]

Originally Posted by Colonel Cathcart

Sigh, I think I found the culprit. The KoboTouchExtended plugin has an option of its own for file name template. I must've set it up months ago and forgot about it completely.

Which I did state. The settings on the "Sending to devices" page of the preferences is a default. If you make any changes to a driver configuration, it copies the preference and you must maintain it in the driver configuration from then on.

[davidfor]

Originally Posted by Colonel Cathcart

Your explanation makes (some) sense regarding the shortening of the file path. Why hasn't Calibre upped the limit of the file path length with the times and sticks to the ol' FAT32 is a question in itself.

Because it can't. The limitation here is with the device, not calibre. Kobo ereaders use FAT32 for their book partition. Any book you put on it has to respect those limitations, if you want the device to work. And the device creates other files using the path which means the full FAT32 path length cannot be used. The path length used has been worked out to work with the device.

And calibre is cross-platform and tries to work with all of them in the same way. That means it does sometimes use the lowest-common denominator for these sort of things. That will mean it works for most people.

[DNSB]

For what it's worth, the template I use for send to device is:

ebooks/{author_sort:sublist(0,1,&)}/{title} - {authors:sublist(0,1,&)}

Both the author_sort used for the directory name and the authors appended to the file name are shortened to the first name in the list (the sublist bit). The author names are visible to the Kobo GUI.

I ran into the shortened path issue years back with an anthology that listed 42 authors in the authors list. It made for a rather interesting entry in my library.

[Colonel Cathcart]

Thank you, davidfor. Indeed, you did give me the right answer.

Is the 185 character limit to the path the only restriction? No additional restriction regarding the file name length? (obviously there are other restrictions regarding illegal characters).
And regarding to path, what, exactly, does that include? Take for example:

C:/Books/Hastie, Trevor & Tibshirani, Robert & Friedman, Jerome/The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition

Is the "C:/" part counted towards that 185 character limit?

The reason I ask is that I'd like to write something a bit more robust and need a clear understanding what's going on under the hood.