Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 06-24-2012, 07:54 PM   #1
maxpall
Junior Member
maxpall began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jun 2012
Device: Sony TS1
Help with replicating dir structure

Dear all, new at the forum... great work the SW and the community too!

My question is the following. I use my Sony PRS T1 to read and to note a lot of PDF docs. So, I need these to be synced back to my laptop, where I keep my full bibliography.

I do this via dropsync, but I would like to sync the Calibre biblio as it is, and to be able to manage the metadata via Calibre (titles, tags, etc). So:
-if I sync my biblio as it is, and then I connect my device, Calibre is not able to recognize the presence of the documents on it as Calibre bibliotheque.
- but if I first sent my Calibre biblio to the device (with the aim of keeping it synced later on), it would reproduce a dir structure which is not exactly that of Calibre. And therefore I will not be able to sync it.

The solution is to play with the template, and to 'send to device' through a template that reproduces the Calibre dir structure. I've done several tries, but no success...

Can anyone help me, by posting the template that reproduces exactly the Calibre internal dir structure?

THanks!

Max

Last edited by maxpall; 06-26-2012 at 08:54 AM. Reason: Late night mistake in the name of my beloved reader...
maxpall is offline   Reply With Quote
Old 07-22-2012, 12:42 PM   #2
Shai-Hulud
Silicon Book Worm
Shai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud of
 
Shai-Hulud's Avatar
 
Posts: 129
Karma: 27430
Join Date: Jul 2012
Location: England
Device: Kobo Touch
Quote:
Originally Posted by maxpall View Post
Can anyone help me, by posting the template that reproduces exactly the Calibre internal dir structure?
Actually, I'd be interested in replicating the exact calibre library structure too if anyone can help. I'm obliged to circumvent a calibre <--> kobo touch sync issue by doing a direct folder to folder copy in Windows (using the add/remove book features in calibre cause the usb connection to fail with depressingly predictable regularity. Drag-and-drop just works). Keeping the directory structure consistent between the two would make syncing and management straightforward.

The specific naming issue I have is how to pass the number that gets appended to the folder when it's created. e.g.

Let's say Fred adds "The Three Musketeers". calibre creates a folder "Alexandre Dumas\The Three Musketeers (103)\The Three Musketeers - Alexandre Dumas.epub". Whereas when I add it to my library, it creates "Alexandre Dumas\The Three Musketeers (41)\The Three Musketeers - Alexandre Dumas.epub". For Fred the number variable is 103, for me 41.

The closest I've got so far following the calibre help is
Quote:
{authors}/{title} (number)/{title} - {authors}
but as a calibre newcomer I don't yet know the correct syntax to express (number) in a template. Can any calibre afficionados throw some light on this please?
Shai-Hulud is offline   Reply With Quote
Old 07-22-2012, 12:52 PM   #3
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,870
Karma: 5654321
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by Shai-Hulud View Post
Actually, I'd be interested in replicating the exact calibre library structure too if anyone can help. I'm obliged to circumvent a calibre <--> kobo touch sync issue by doing a direct folder to folder copy in Windows (using the add/remove book features in calibre cause the usb connection to fail with depressingly predictable regularity. Drag-and-drop just works). Keeping the directory structure consistent between the two would make syncing and management straightforward.

The specific naming issue I have is how to pass the number that gets appended to the folder when it's created. e.g.

Let's say Fred adds "The Three Musketeers". calibre creates a folder "Alexandre Dumas\The Three Musketeers (103)\The Three Musketeers - Alexandre Dumas.epub". Whereas when I add it to my library, it creates "Alexandre Dumas\The Three Musketeers (41)\The Three Musketeers - Alexandre Dumas.epub". For Fred the number variable is 103, for me 41.

The closest I've got so far following the calibre help is

but as a calibre newcomer I don't yet know the correct syntax to express (number) in a template. Can any calibre afficionados throw some light on this please?
{id}
theducks is offline   Reply With Quote
Old 07-22-2012, 01:17 PM   #4
Shai-Hulud
Silicon Book Worm
Shai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud of
 
Shai-Hulud's Avatar
 
Posts: 129
Karma: 27430
Join Date: Jul 2012
Location: England
Device: Kobo Touch
Quote:
Originally Posted by theducks View Post
{id}
Believe it or not I had just sussed that out by reading "Preferences, Sending Books To Devices" and was coming back to post my findings when I see you pipped me to it. But thanks very much

For Max & everyone else, the template to replicate calibre's library on your device is:

{authors}/{title} ({id})/{title} - {authors}
Shai-Hulud is offline   Reply With Quote
Old 07-27-2012, 07:01 PM   #5
maxpall
Junior Member
maxpall began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jun 2012
Device: Sony TS1
Shai-Hulud, thanks for the attempt but.... it doesn't work...

I believe the problem is with complicated authors/titles. This book, in the PC is


\Calibre\Dematteis G_\Il territorio nello sviluppo locale. Il (378)\Il territorio nello sviluppo locale. Il - Dematteis G_.pdf


In the reader becomes

\books\Dematteis G. & Governa F_\territorio nello sviluppo locale. Il contributo del modello SLoT, Il (378)\territorio nello sviluppo locale. Il contributo del modello SLoT, Il - Dematteis G. & Governa F_.pdf


There are two issues: a long title becomes truncated, but it is not easy to find out the rule; and then only the first_author is used in the calibre original structure

Anyone has a hint...?


Max
maxpall is offline   Reply With Quote
Old 07-27-2012, 07:27 PM   #6
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 37,707
Karma: 18475502
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Sony Reader PRS-650, iPad, nook STR
do away with the directory structure on the device as when you delete the eBooks, you are left with empty directories.

Go to preferences and under Sending books to devices, put in {authors) - {title}.
JSWolf is offline   Reply With Quote
Old 07-28-2012, 09:18 AM   #7
Shai-Hulud
Silicon Book Worm
Shai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud of
 
Shai-Hulud's Avatar
 
Posts: 129
Karma: 27430
Join Date: Jul 2012
Location: England
Device: Kobo Touch
Quote:
Originally Posted by maxpall View Post
I believe the problem is with complicated authors/titles. This book, in the PC is

\Calibre\Dematteis G_\Il territorio nello sviluppo locale. Il (378)\Il territorio nello sviluppo locale. Il - Dematteis G_.pdf

In the reader becomes

\books\Dematteis G. & Governa F_\territorio nello sviluppo locale. Il contributo del modello SLoT, Il (378)\territorio nello sviluppo locale. Il contributo del modello SLoT, Il - Dematteis G. & Governa F_.pdf
First off I better declare that I am a new user of calibre, so I am still finding my way. But my results are determined empirically; that is by actual experimentation. I originally found this thread after searching Mobileread for how to faithfully replicate the calibre library structure. As such, I didn't fully read your use case (i.e. the inclusion of dropsync into the equation) but for the purposes of clarity, I'm going to concentrate on the interaction between calibre and a usb mass storage device (assuming that is how your Sony presents itself).

I have tested the template I posted previously and it correctly builds the default calibre library folder/file structure on the root of a usb device when you use calibre to initiate the sync.
e.g. [Author]\[Book Title] [ID]\[Book Title] - [Author].pdf

The examples you have posted above ring alarm bells for me because neither adhere to the default "Calibre Library\Author\Book Name\Book Name - Author.pdf" structure used by calibre. I suspect that from a combination of using dropsync, direct copying to calibre and a misconfigured Sony plug-in, your calibre library is no longer wholly in the standard format which is why the template doesn't work for you. So do you try to correct it manually or start again from scratch? Obviously I can't decide for you since it depends on personal factors such as available time and library size. All I can offer is how I would set up calibre to ensure a consistent result:

1. Choose your canonical source of PDFs (I strongly suggest your calibre library) and make all future changes with respect to that.
2. Assuming you have a clean install of calibre and have freshly re-imported your PDFs, edit the metadata for each book paying careful attention that the author and book title values are in the correct order and that the sort values match them.
3. Edit your device plugin (Preferences\Plug-ins\Device Interface Plug-ins\Sony PRST1 and newer) and replace the template with the one I suggested. I would also suggest unticking the "author sort for author" option and "Use Sony Author format".
4. Having previously reset (or cleared all books from) your Sony, try a test sync for one book from calibre to your Sony. Hopefully the file/folder structure will now match your calibre library.

Alternatively, consider if you really need to adhere to calibre's folder/file structure. If not, follow JSWolf's advice and keep it simple.
Good luck.
Shai-Hulud is offline   Reply With Quote
Old 07-28-2012, 12:18 PM   #8
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 37,707
Karma: 18475502
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Sony Reader PRS-650, iPad, nook STR
Quote:
Originally Posted by Shai-Hulud View Post
Alternatively, consider if you really need to adhere to calibre's folder/file structure. If not, follow JSWolf's advice and keep it simple.
You don't actually need and in some cases, don't want the directory structure as when you go to delete the eBook, the directory is left behind. A lot of readers will not delete the empty directory when you delete the eBooks using the device's delete function. If you use Calibre (I think) you may be deleting the directory when you delete the content.

So if you want to risk cluttering up your reader with lots of empty directories, feel free to create them. But you don't really need them.
JSWolf is offline   Reply With Quote
Old 07-28-2012, 04:37 PM   #9
Shai-Hulud
Silicon Book Worm
Shai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud of
 
Shai-Hulud's Avatar
 
Posts: 129
Karma: 27430
Join Date: Jul 2012
Location: England
Device: Kobo Touch
Quote:
Originally Posted by JSWolf View Post
You don't actually need and in some cases, don't want the directory structure as when you go to delete the eBook, the directory is left behind.
Well I can't speak for Max but in my case I do (if you read my previous posts). It's a trade off for getting the bulk of the calibre library onto my KT then using calibre to manage them seamlessly.
Shai-Hulud is offline   Reply With Quote
Old 08-01-2012, 05:50 AM   #10
Pepin33
Zealot
Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.
 
Posts: 109
Karma: 419
Join Date: Aug 2012
Location: Spain
Device: Kindle Touch
You can try using this template to replicate Calibre's folder structure:

program: strcat(sublist(field('author'),0,1,'&'), '/', substr(raw_field('title'),0,40), ' (', field('id'), ')','/', substr(raw_field('title'),0,40), ' - ', sublist(field('author'),0,1,'&'))
Pepin33 is offline   Reply With Quote
Old 08-01-2012, 06:48 AM   #11
Shai-Hulud
Silicon Book Worm
Shai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud of
 
Shai-Hulud's Avatar
 
Posts: 129
Karma: 27430
Join Date: Jul 2012
Location: England
Device: Kobo Touch
Quote:
Originally Posted by Pepin33 View Post
You can try using this template to replicate Calibre's folder structure:

program: strcat(sublist(field('author'),0,1,'&'), '/', substr(raw_field('title'),0,40), ' (', field('id'), ')','/', substr(raw_field('title'),0,40), ' - ', sublist(field('author'),0,1,'&'))
But if calibre correctly interprets the eminently (human) readable form
Quote:
{authors}/{title} ({id})/{title} - {authors}
is there honestly any advantage in expressing it your way?
Shai-Hulud is offline   Reply With Quote
Old 08-01-2012, 07:50 AM   #12
Pepin33
Zealot
Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.Pepin33 has a complete set of Star Wars action figures.
 
Posts: 109
Karma: 419
Join Date: Aug 2012
Location: Spain
Device: Kindle Touch
Quote:
Originally Posted by Shai-Hulud View Post
But if calibre correctly interprets the eminently (human) readable form

is there honestly any advantage in expressing it your way?
Yes. If you want to replicate exactly Calibre's folders structure, there is some problems with: {authors}/{title} ({id})/{title} - {authors}

1.- If the book has more than one author, {auhors} puts all them. In Calibre's folders, only the first author is used.
2.- If the title is too long, in Calibre's folders only first 40 characters are used. {title} uses all them.
3.- Also, {title} uses save_template_title_series_sorting variable to set the title's sorting. Calibre's folders use always strictrly alphabetic order. And when I tried the template, and I put save_template_title_series_sorting = 'strictrly_alphabetic', Calibre ignores it, and continues using library_order (a bug?).

For example, a book titled "The long way" form "Peter Jones & Carl Black": Using the template {authors}/{title} ({id})/{title} - {authors}, the directory structure is:
Peter Jones & Carl Black/long way, The (xxx)/long way, The - Peter Jones & Carl Black
And using the template 'program', the result is:
Peter Jones/The long way (xxx)/The long way - Peter Jones
And this is the same directory structure Calibre uses for internal book storage (the question MaxPall asks in first post)
Quote:
Can anyone help me, by posting the template that reproduces exactly the Calibre internal dir structure?

Last edited by Pepin33; 08-01-2012 at 07:56 AM.
Pepin33 is offline   Reply With Quote
Old 08-05-2012, 10:05 AM   #13
Shai-Hulud
Silicon Book Worm
Shai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud ofShai-Hulud has much to be proud of
 
Shai-Hulud's Avatar
 
Posts: 129
Karma: 27430
Join Date: Jul 2012
Location: England
Device: Kobo Touch
Quote:
Originally Posted by Pepin33 View Post
Yes. If you want to replicate exactly Calibre's folders structure, there is some problems with: {authors}/{title} ({id})/{title} - {authors}.... SNIP
Yes, you're quite right. I obviously tested with some straightforward author/book names because I didn't encounter any issues. Having just read your reply, I tried a few more complicated cases (multiple authors, some titles prepended with A/The) and the simplified template doesn't create the same directory structure after all.

Thanks Pepin for your explanation and template - it does indeed replicate the calibre library structure (which is what both Maxpall and I wanted). Cheers
Shai-Hulud is offline   Reply With Quote
Old 09-01-2012, 06:17 PM   #14
maxpall
Junior Member
maxpall began at the beginning.
 
Posts: 8
Karma: 10
Join Date: Jun 2012
Device: Sony TS1
Sorry for very late reply: I was out of town for sometimes, and wanted to test the solution proposed before answering.

Thanks, Pepin, for the code you posted. It works fine, a part of some issues (so far), concerning the substr of 40 chr from the title:


First case: In case the last of the 40 characters is a ' ' (empty space), it is cut in the generation of the subdir, and also the title of the file.

EXAMPLE:

the following directory is for Calibre internal structure

"After 2015_ progress and challenges for[two spaces](215)"

but it is exported to device as

"After 2015_ progress and challenges for (215)"


Second case: if the book's title has a two subsequent empty spaces, it is reduced to one in exporting

EXAMPLE:

the following directory is for Calibre internal structure

"After 2015 -[two spaces]Contexts, politics and pro (225)"

but it is exported to device as

"After 2015 - Contexts, politics and pro (225)"


Third case: ? In this third case I0m not able to understand: an underscore at the place of an empty space

EXAMPLE:

the following directory is for Calibre internal structure

"Post-2015 policymaking What is being pl (222)"

but it is exported to device as

"Post-2015 policymaking_What is being pla (222)"


Thanks again!


Max

Last edited by maxpall; 09-01-2012 at 06:21 PM.
maxpall is offline   Reply With Quote
Old 09-02-2012, 02:45 PM   #15
JSWolf
Resident Curmudgeon
JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.JSWolf ought to be getting tired of karma fortunes by now.
 
JSWolf's Avatar
 
Posts: 37,707
Karma: 18475502
Join Date: Nov 2006
Location: Roslindale, Massachusetts
Device: Sony Reader PRS-650, iPad, nook STR
Quote:
Originally Posted by maxpall View Post
Sorry for very late reply: I was out of town for sometimes, and wanted to test the solution proposed before answering.

Thanks, Pepin, for the code you posted. It works fine, a part of some issues (so far), concerning the substr of 40 chr from the title:


First case: In case the last of the 40 characters is a ' ' (empty space), it is cut in the generation of the subdir, and also the title of the file.

EXAMPLE:

the following directory is for Calibre internal structure

"After 2015_ progress and challenges for[two spaces](215)"

but it is exported to device as

"After 2015_ progress and challenges for (215)"
You don't want the eBooks exported in subdirectories created by Calibre. The problem being that if you delete a book from the T1 using the T1, the subdirectory isn't deleted and you will end up with a bunch of empty directories.

Code:
{title} - {authors}
The above is what you want to use when sending to the T1. So it's not going to be an issue about duplicating the directory structure as you don't want it and you don't need it.
JSWolf is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[old thread] filename and library structure /author and titel structure tscamera Library Management 4 05-31-2011 05:44 PM
Lost.Dir musicman2523 PocketBook 2 03-20-2011 11:29 AM
Updates and LOST.DIR contents drv Nook Color & Nook Tablet 2 01-28-2011 06:13 PM
Kaufst Du Dir einen Kindle? Alexander Turcic Amazon Kindle 223 06-01-2010 05:49 AM
Copyright issues in replicating you library in eBook format ZacWolf News 34 11-29-2009 04:51 PM


All times are GMT -4. The time now is 06:51 PM.


MobileRead.com is a privately owned, operated and funded community.