Help with replicating dir structure

[maxpall]

Dear all, new at the forum... great work the SW and the community too!

My question is the following. I use my Sony PRS T1 to read and to note a lot of PDF docs. So, I need these to be synced back to my laptop, where I keep my full bibliography.

I do this via dropsync, but I would like to sync the Calibre biblio as it is, and to be able to manage the metadata via Calibre (titles, tags, etc). So:
-if I sync my biblio as it is, and then I connect my device, Calibre is not able to recognize the presence of the documents on it as Calibre bibliotheque.
- but if I first sent my Calibre biblio to the device (with the aim of keeping it synced later on), it would reproduce a dir structure which is not exactly that of Calibre. And therefore I will not be able to sync it.

The solution is to play with the template, and to 'send to device' through a template that reproduces the Calibre dir structure. I've done several tries, but no success...

Can anyone help me, by posting the template that reproduces exactly the Calibre internal dir structure?

THanks!

Max

[Shai-Hulud]

Originally Posted by maxpall

Can anyone help me, by posting the template that reproduces exactly the Calibre internal dir structure?

Actually, I'd be interested in replicating the exact calibre library structure too if anyone can help. I'm obliged to circumvent a calibre <--> kobo touch sync issue by doing a direct folder to folder copy in Windows (using the add/remove book features in calibre cause the usb connection to fail with depressingly predictable regularity. Drag-and-drop just works). Keeping the directory structure consistent between the two would make syncing and management straightforward.

The specific naming issue I have is how to pass the number that gets appended to the folder when it's created. e.g.

Let's say Fred adds "The Three Musketeers". calibre creates a folder "Alexandre Dumas\The Three Musketeers (103)\The Three Musketeers - Alexandre Dumas.epub". Whereas when I add it to my library, it creates "Alexandre Dumas\The Three Musketeers (41)\The Three Musketeers - Alexandre Dumas.epub". For Fred the number variable is 103, for me 41.

The closest I've got so far following the calibre help is

{authors}/{title} (number)/{title} - {authors}

but as a calibre newcomer I don't yet know the correct syntax to express (number) in a template. Can any calibre afficionados throw some light on this please?

[theducks]

Originally Posted by Shai-Hulud

Actually, I'd be interested in replicating the exact calibre library structure too if anyone can help. I'm obliged to circumvent a calibre <--> kobo touch sync issue by doing a direct folder to folder copy in Windows (using the add/remove book features in calibre cause the usb connection to fail with depressingly predictable regularity. Drag-and-drop just works). Keeping the directory structure consistent between the two would make syncing and management straightforward.

The specific naming issue I have is how to pass the number that gets appended to the folder when it's created. e.g.

Let's say Fred adds "The Three Musketeers". calibre creates a folder "Alexandre Dumas\The Three Musketeers (103)\The Three Musketeers - Alexandre Dumas.epub". Whereas when I add it to my library, it creates "Alexandre Dumas\The Three Musketeers (41)\The Three Musketeers - Alexandre Dumas.epub". For Fred the number variable is 103, for me 41.

The closest I've got so far following the calibre help is

but as a calibre newcomer I don't yet know the correct syntax to express (number) in a template. Can any calibre afficionados throw some light on this please?

{id}

[Shai-Hulud]

Originally Posted by theducks

{id}

Believe it or not I had just sussed that out by reading "Preferences, Sending Books To Devices" and was coming back to post my findings when I see you pipped me to it. But thanks very much

For Max & everyone else, the template to replicate calibre's library on your device is:

{authors}/{title} ({id})/{title} - {authors}

[maxpall]

Shai-Hulud, thanks for the attempt but.... it doesn't work...

I believe the problem is with complicated authors/titles. This book, in the PC is


\Calibre\Dematteis G_\Il territorio nello sviluppo locale. Il (378)\Il territorio nello sviluppo locale. Il - Dematteis G_.pdf


In the reader becomes

\books\Dematteis G. & Governa F_\territorio nello sviluppo locale. Il contributo del modello SLoT, Il (378)\territorio nello sviluppo locale. Il contributo del modello SLoT, Il - Dematteis G. & Governa F_.pdf


There are two issues: a long title becomes truncated, but it is not easy to find out the rule; and then only the first_author is used in the calibre original structure

Anyone has a hint...?


Max

[JSWolf]

do away with the directory structure on the device as when you delete the eBooks, you are left with empty directories.

Go to preferences and under Sending books to devices, put in {authors) - {title}.

[Shai-Hulud]

Originally Posted by maxpall

I believe the problem is with complicated authors/titles. This book, in the PC is

\Calibre\Dematteis G_\Il territorio nello sviluppo locale. Il (378)\Il territorio nello sviluppo locale. Il - Dematteis G_.pdf

In the reader becomes

\books\Dematteis G. & Governa F_\territorio nello sviluppo locale. Il contributo del modello SLoT, Il (378)\territorio nello sviluppo locale. Il contributo del modello SLoT, Il - Dematteis G. & Governa F_.pdf

First off I better declare that I am a new user of calibre, so I am still finding my way. But my results are determined empirically; that is by actual experimentation. I originally found this thread after searching Mobileread for how to faithfully replicate the calibre library structure. As such, I didn't fully read your use case (i.e. the inclusion of dropsync into the equation) but for the purposes of clarity, I'm going to concentrate on the interaction between calibre and a usb mass storage device (assuming that is how your Sony presents itself).

I have tested the template I posted previously and it correctly builds the default calibre library folder/file structure on the root of a usb device when you use calibre to initiate the sync.
e.g. [Author]\[Book Title] [ID]\[Book Title] - [Author].pdf

The examples you have posted above ring alarm bells for me because neither adhere to the default "Calibre Library\Author\Book Name\Book Name - Author.pdf" structure used by calibre. I suspect that from a combination of using dropsync, direct copying to calibre and a misconfigured Sony plug-in, your calibre library is no longer wholly in the standard format which is why the template doesn't work for you. So do you try to correct it manually or start again from scratch? Obviously I can't decide for you since it depends on personal factors such as available time and library size. All I can offer is how I would set up calibre to ensure a consistent result:

1. Choose your canonical source of PDFs (I strongly suggest your calibre library) and make all future changes with respect to that.
2. Assuming you have a clean install of calibre and have freshly re-imported your PDFs, edit the metadata for each book paying careful attention that the author and book title values are in the correct order and that the sort values match them.
3. Edit your device plugin (Preferences\Plug-ins\Device Interface Plug-ins\Sony PRST1 and newer) and replace the template with the one I suggested. I would also suggest unticking the "author sort for author" option and "Use Sony Author format".
4. Having previously reset (or cleared all books from) your Sony, try a test sync for one book from calibre to your Sony. Hopefully the file/folder structure will now match your calibre library.

Alternatively, consider if you really need to adhere to calibre's folder/file structure. If not, follow JSWolf's advice and keep it simple.
Good luck.

[JSWolf]

Originally Posted by Shai-Hulud

Alternatively, consider if you really need to adhere to calibre's folder/file structure. If not, follow JSWolf's advice and keep it simple.

You don't actually need and in some cases, don't want the directory structure as when you go to delete the eBook, the directory is left behind. A lot of readers will not delete the empty directory when you delete the eBooks using the device's delete function. If you use Calibre (I think) you may be deleting the directory when you delete the content.

So if you want to risk cluttering up your reader with lots of empty directories, feel free to create them. But you don't really need them.

[Shai-Hulud]

Originally Posted by JSWolf

You don't actually need and in some cases, don't want the directory structure as when you go to delete the eBook, the directory is left behind.

Well I can't speak for Max but in my case I do (if you read my previous posts). It's a trade off for getting the bulk of the calibre library onto my KT then using calibre to manage them seamlessly.

[Pepin33]

You can try using this template to replicate Calibre's folder structure:

program: strcat(sublist(field('author'),0,1,'&'), '/', substr(raw_field('title'),0,40), ' (', field('id'), ')','/', substr(raw_field('title'),0,40), ' - ', sublist(field('author'),0,1,'&'))

[Shai-Hulud]

Originally Posted by Pepin33

You can try using this template to replicate Calibre's folder structure:

program: strcat(sublist(field('author'),0,1,'&'), '/', substr(raw_field('title'),0,40), ' (', field('id'), ')','/', substr(raw_field('title'),0,40), ' - ', sublist(field('author'),0,1,'&'))

But if calibre correctly interprets the eminently (human) readable form

{authors}/{title} ({id})/{title} - {authors}

is there honestly any advantage in expressing it your way?

[Pepin33]

Originally Posted by Shai-Hulud

But if calibre correctly interprets the eminently (human) readable form

is there honestly any advantage in expressing it your way?

Yes. If you want to replicate exactly Calibre's folders structure, there is some problems with: {authors}/{title} ({id})/{title} - {authors}

1.- If the book has more than one author, {auhors} puts all them. In Calibre's folders, only the first author is used.
2.- If the title is too long, in Calibre's folders only first 40 characters are used. {title} uses all them.
3.- Also, {title} uses save_template_title_series_sorting variable to set the title's sorting. Calibre's folders use always strictrly alphabetic order. And when I tried the template, and I put save_template_title_series_sorting = 'strictrly_alphabetic', Calibre ignores it, and continues using library_order (a bug?).

For example, a book titled "The long way" form "Peter Jones & Carl Black": Using the template {authors}/{title} ({id})/{title} - {authors}, the directory structure is:
Peter Jones & Carl Black/long way, The (xxx)/long way, The - Peter Jones & Carl Black
And using the template 'program', the result is:
Peter Jones/The long way (xxx)/The long way - Peter Jones
And this is the same directory structure Calibre uses for internal book storage (the question MaxPall asks in first post)

Can anyone help me, by posting the template that reproduces exactly the Calibre internal dir structure?

[Shai-Hulud]

Originally Posted by Pepin33

Yes. If you want to replicate exactly Calibre's folders structure, there is some problems with: {authors}/{title} ({id})/{title} - {authors}.... SNIP

Yes, you're quite right. I obviously tested with some straightforward author/book names because I didn't encounter any issues. Having just read your reply, I tried a few more complicated cases (multiple authors, some titles prepended with A/The) and the simplified template doesn't create the same directory structure after all.

Thanks Pepin for your explanation and template - it does indeed replicate the calibre library structure (which is what both Maxpall and I wanted). Cheers

[maxpall]

Sorry for very late reply: I was out of town for sometimes, and wanted to test the solution proposed before answering.

Thanks, Pepin, for the code you posted. It works fine, a part of some issues (so far), concerning the substr of 40 chr from the title:


First case: In case the last of the 40 characters is a ' ' (empty space), it is cut in the generation of the subdir, and also the title of the file.

EXAMPLE:

the following directory is for Calibre internal structure

"After 2015_ progress and challenges for[two spaces](215)"

but it is exported to device as

"After 2015_ progress and challenges for (215)"


Second case: if the book's title has a two subsequent empty spaces, it is reduced to one in exporting

EXAMPLE:

the following directory is for Calibre internal structure

"After 2015 -[two spaces]Contexts, politics and pro (225)"

but it is exported to device as

"After 2015 - Contexts, politics and pro (225)"


Third case: ? In this third case I0m not able to understand: an underscore at the place of an empty space

EXAMPLE:

the following directory is for Calibre internal structure

"Post-2015 policymaking What is being pl (222)"

but it is exported to device as

"Post-2015 policymaking_What is being pla (222)"


Thanks again!


Max

[JSWolf]

Originally Posted by maxpall

Sorry for very late reply: I was out of town for sometimes, and wanted to test the solution proposed before answering.

Thanks, Pepin, for the code you posted. It works fine, a part of some issues (so far), concerning the substr of 40 chr from the title:


First case: In case the last of the 40 characters is a ' ' (empty space), it is cut in the generation of the subdir, and also the title of the file.

EXAMPLE:

the following directory is for Calibre internal structure

"After 2015_ progress and challenges for[two spaces](215)"

but it is exported to device as

"After 2015_ progress and challenges for (215)"

You don't want the eBooks exported in subdirectories created by Calibre. The problem being that if you delete a book from the T1 using the T1, the subdirectory isn't deleted and you will end up with a bunch of empty directories.

{title} - {authors}

The above is what you want to use when sending to the T1. So it's not going to be an issue about duplicating the directory structure as you don't want it and you don't need it.