"Add Books" naming convention peeves - Page 2

louwin · 04-01-2012, 09:10 PM

Okay, thanks for the advice given

I have read ALL I can about Regular Expressions both from the tutorial AND the section in the Python manual. I have concluded that, even with my experience in programming, to acheive what I want with regex is beyond my capabilities.

So I have decided to play with the filenames of the eBooks I wish to add as I can't edit the "pdf" files themselves.

Could someone please point me to where in the manual I can find the full formatting of the file name?

I have done searches in both the Quick Start and the full Calibre user manuals but couldn't find the required info about "file name format".

I have worked out, by playing with a test library, many "Adds" and looking at the Calibre files in the Library, that to get Title and Author the filename has to be, for example, "A Book Title - A N Author.pdf"

I have started editing the filenames of the books I want to add to this format but it will be/is a HUGE task (over 2000 books).

Can I add "Series" information to this format of filename? And have it automatically extracted and stored?

Can I acheive this by formating/structuring the folders to be added? For instance:-

Author folder
-Series folder
--First Book folder
---Cover.jpg
---First Title.pdf
--Second Book folder
---Cover.jpg
---Second Title

I suspect not and that I will be back in the realms of extremely complex (impossible(?)) regexs?

A good start will be where to find the FULL filename specification

speakingtohe · 04-01-2012, 09:56 PM

I don't think folder names are used by calibre to assign metadata.

Under preferences>adding books you have several options and can test the regex against a file name.

I am not a regex person. I just import books and then download metadata and usually the default regex (9/10) works for me.

I have read data from the file contents checked and swap author firstname and lastname checked although I may on occasion uncheck one or both.

I actually pay little attention to the naming convention used on books before I import them as the metadata in my case is generally taken from the file contents.

But in a text file with no metadata embedded
Author LN, Author FN - series xx -title works for me with the way my preferences are set.

Smith,John - myseries 07 - title for book.txt

(?P<author>[^_-]+) -?\s*(?P<series>[^_0-9-]*)(?P<series_index>[0-9]*)\s*-\s*(?P<title>[^_].+) ?

I think it is easier to rename the books inside calibre after importing them. But that is me.
The bulk edit metadata dialogue has a swap author title function and the individual edit has one as well. In the individual edit metadata dialogue you can also view the book if the title is not apparent. and the download metadata is pretty configurable and works pretty good.

If you do want to rename all of your books first, and I can see reasons for wanting to do this, then make sure read data from the file contents is unchecked or the result may be unexpected.

Sorry to be so wordy

Helen

louwin · 04-01-2012, 10:50 PM

Hi Helen,

Thanks for your reply.

Sadly, my experience of extracting metadata from contents has been totally totally unsuccessful.

Editing after adding is also much much more work IMO.

So I think I'm stuck with playing with the file names.

I just use whatever regex is in the standard setup.

From my reading I couldn't see any way to use the directory structure in regex either.

speakingtohe · 04-01-2012, 11:17 PM

well I will tell you what I found to be easiest when I was imported adding what seemed like a zillion books.

Added all of the books and there were a lot. Took me two days to do this in chunks of 1000.
Lots were dups of course.

Then I did bulk metadata fetches and the metadata fetching was a lot less adequate, the series metadata was not downloaded at that time and a lot of it was butt ugly. The metadata itself is much better at the place it is fetched from these days

Then I sorted by title and went down the list and used the bulk edit screen to swap author title where it was easy to spot.

Then I sorted by author and did the same thing.

Then in batches I bulk downloaded the metadata with the option set not to change the title or author.

Then I checked for duplicates, author sort etc.

Took me quite a while. As I was reaching the end along came the quality check and the duplicate check plugins. I would have killed to have these earlier, but they still were useful.

Prior to biting the bullet and taking the just get it done approach I had spent weeks and weeks trying different methods of inputting books and data, saving to disk with different filenames, and generally getting nowhere.

I hope you are at least getting mostly good results. Me I would be happy one day and a week later wonder what was I thinking.

Now my books are titled and seriesed in a pretty consistent manner, but they still aren't perfect. Darn close though

Helen

Dopedangel · 04-01-2012, 11:20 PM

Quote:

Originally Posted by louwin

Hi Helen,

Thanks for your reply.

Sadly, my experience of extracting metadata from contents has been totally totally unsuccessful.

Editing after adding is also much much more work IMO.

So I think I'm stuck with playing with the file names.

I just use whatever regex is in the standard setup.

From my reading I couldn't see any way to use the directory structure in regex either.

You can use a batch renamer to replace by in all the names to - then once the title and author are separated by - in the filename you can easily import them into calibre with correct regex

louwin · 04-02-2012, 12:32 AM

I "Added" about 7000 books to an empty library as the first thing I did. Result - about 40% garbage!

I had hundreds of books titled "Prologue", hundreds of books by "harry", hundreds of books titled "Chapter 1" and "1", hundreds books with title appended with " by author" (hence this thread) and "Unknown" author, thousands of books with authors with names like "." "-" "--" "111111" "-----------------" and every combination of garbage. Funny thing is that I would have done much better using file names.

I just threw it away and started again which is where I am at the moment.

Not being in a position to edit the contents of "pdf" "rtf" "mobi" "epub" etc files leaves me with file names. I use a batch renamer with simple regex. The files are in a good structure which is why I am interested in "Adding" structured files.

I suspected, and it has been confirmed, that regexs can't use directory sttructures so file names are all I have

Nice talking to you. It is good to know that I not the only one

speakingtohe · 04-02-2012, 12:41 AM

Allrighty then. I will say (*hopefully) one last thing.

After adding books that have garbage (and yes it still happens ) I use the viewer from within the edit individual metadata dialogue and copy and paste the title/author.

Helen

louwin · 04-02-2012, 01:53 AM

Not a problem....

Last response and the thread can go to sleep.

I know, I have done it.

It wasn't worth trying to individually correct 3000 garbage entries.

I will do it through file names

Pity no one knows/said where to find the full file name specification

I will have to be happy with title and author

theducks · 04-02-2012, 02:15 AM

Quote:

Originally Posted by louwin

Not a problem....

Last response and the thread can go to sleep.

I know, I have done it.

It wasn't worth trying to individually correct 3000 garbage entries.

I will do it through file names

Pity no one knows/said where to find the full file name specification

I will have to be happy with title and author

There is no SPECIFICATION, because the filename can be anything you want that the file system accepts.
It is also your job to match the Import REGEX to your naming convention. This include matching all dashes, paren or brackets

Calibre ONLY looks at the filename, NOT any part of the PATHNAME

louwin · 04-02-2012, 03:20 AM

That probably explains why I couldn't find any....

JFI I reran that "Add to empty Library" test using file names

9304 files took 21 minutes and, as far as I can see, produced 21 entries in error involving 7 invalid authors.

I don't remember how long the "contents" test took but it was 6 to 7 hours.

Thanks and Goodnight (to the thread).

GeckoFriend · 04-02-2012, 08:34 AM

[QUOTE=speakingtohe;2026360]
After adding books that have garbage (and yes it still happens ) I use the viewer from within the edit individual metadata dialogue and copy and paste the title/author.

Helen[/QUOTE

This sounds useful but I can't find the viewer from within the edit metadata dialogue.

speakingtohe · 04-02-2012, 08:59 AM

[QUOTE=GeckoFriend;2026629]

Quote:

Originally Posted by speakingtohe

After adding books that have garbage (and yes it still happens ) I use the viewer from within the edit individual metadata dialogue and copy and paste the title/author.

Helen[/QUOTE

This sounds useful but I can't find the viewer from within the edit metadata dialogue.

double click on a format in the box in lower right corner
Helen

GeckoFriend · 04-02-2012, 10:44 AM

[QUOTE=speakingtohe;2026657]

Quote:

Originally Posted by GeckoFriend

double click on a format in the box in lower right corner
Helen

Thank you; that will definitely save me some time and effort.

irmar · 05-22-2012, 02:14 AM

I also have problems with the names of Calibre outputs. I've tried to play with the regex, but I'm not good at it.
I want all my books to be in this format:
Name Surname - Title (year)

Calibre thinks differently, and it will interpret the title as author name, and/or change my title to put the author at the end.
So I went do Adding books and made my regex like this:
(?P<author>[^_]+) - (?P<title>.+)
Still this wasn't enough.
Then I went to saving books and made the expression like this:
{author_sort}/{authors} - {title}

Still it didn't help. What am I doing wrong? Can someone please give a helping hand? I'm renaming them all one by one and going crazy!!!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
"The add books process seems to have hung"	joleta	Calibre	6	06-16-2012 02:26 AM
Calibre closes after clicking "Add Books" button	JeweledSunshine	Devices	4	09-25-2011 12:50 PM
Problem with "Add Books" regex	wygit	Library Management	9	07-24-2011 12:24 PM
Add books to "Empty Book"	SmiLynnLady	Library Management	8	02-10-2011 09:32 AM
[Enhancement suggestion] Folders when save books in "Add Books" function	simonbcn	Calibre	1	08-30-2009 12:59 PM

04-01-2012, 09:10 PM	#16
louwin Newbie Nerd Posts: 114 Karma: 1000354 Join Date: Feb 2012 Location: Perth, Western Australia Device: iPad 3 64Gb Black	Okay, thanks for the advice given I have read ALL I can about Regular Expressions both from the tutorial AND the section in the Python manual. I have concluded that, even with my experience in programming, to acheive what I want with regex is beyond my capabilities. So I have decided to play with the filenames of the eBooks I wish to add as I can't edit the "pdf" files themselves. Could someone please point me to where in the manual I can find the full formatting of the file name? I have done searches in both the Quick Start and the full Calibre user manuals but couldn't find the required info about "file name format". I have worked out, by playing with a test library, many "Adds" and looking at the Calibre files in the Library, that to get Title and Author the filename has to be, for example, "A Book Title - A N Author.pdf" I have started editing the filenames of the books I want to add to this format but it will be/is a HUGE task (over 2000 books). Can I add "Series" information to this format of filename? And have it automatically extracted and stored? Can I acheive this by formating/structuring the folders to be added? For instance:- Author folder -Series folder --First Book folder ---Cover.jpg ---First Title.pdf --Second Book folder ---Cover.jpg ---Second Title I suspect not and that I will be back in the realms of extremely complex (impossible(?)) regexs? A good start will be where to find the FULL filename specification

04-01-2012, 09:56 PM	#17
speakingtohe Wizard Posts: 4,812 Karma: 26912940 Join Date: Apr 2010 Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet	I don't think folder names are used by calibre to assign metadata. Under preferences>adding books you have several options and can test the regex against a file name. I am not a regex person. I just import books and then download metadata and usually the default regex (9/10) works for me. I have read data from the file contents checked and swap author firstname and lastname checked although I may on occasion uncheck one or both. I actually pay little attention to the naming convention used on books before I import them as the metadata in my case is generally taken from the file contents. But in a text file with no metadata embedded Author LN, Author FN - series xx -title works for me with the way my preferences are set. Smith,John - myseries 07 - title for book.txt (?P<author>[^_-]+) -?\s(?P<series>[^_0-9-])(?P<series_index>[0-9])\s-\s*(?P<title>[^_].+) ? I think it is easier to rename the books inside calibre after importing them. But that is me. The bulk edit metadata dialogue has a swap author title function and the individual edit has one as well. In the individual edit metadata dialogue you can also view the book if the title is not apparent. and the download metadata is pretty configurable and works pretty good. If you do want to rename all of your books first, and I can see reasons for wanting to do this, then make sure read data from the file contents is unchecked or the result may be unexpected. Sorry to be so wordy Helen

04-01-2012, 10:50 PM	#18
louwin Newbie Nerd Posts: 114 Karma: 1000354 Join Date: Feb 2012 Location: Perth, Western Australia Device: iPad 3 64Gb Black	Hi Helen, Thanks for your reply. Sadly, my experience of extracting metadata from contents has been totally totally unsuccessful. Editing after adding is also much much more work IMO. So I think I'm stuck with playing with the file names. I just use whatever regex is in the standard setup. From my reading I couldn't see any way to use the directory structure in regex either.

04-01-2012, 11:17 PM	#19
speakingtohe Wizard Posts: 4,812 Karma: 26912940 Join Date: Apr 2010 Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet	well I will tell you what I found to be easiest when I was imported adding what seemed like a zillion books. Added all of the books and there were a lot. Took me two days to do this in chunks of 1000. Lots were dups of course. Then I did bulk metadata fetches and the metadata fetching was a lot less adequate, the series metadata was not downloaded at that time and a lot of it was butt ugly. The metadata itself is much better at the place it is fetched from these days Then I sorted by title and went down the list and used the bulk edit screen to swap author title where it was easy to spot. Then I sorted by author and did the same thing. Then in batches I bulk downloaded the metadata with the option set not to change the title or author. Then I checked for duplicates, author sort etc. Took me quite a while. As I was reaching the end along came the quality check and the duplicate check plugins. I would have killed to have these earlier, but they still were useful. Prior to biting the bullet and taking the just get it done approach I had spent weeks and weeks trying different methods of inputting books and data, saving to disk with different filenames, and generally getting nowhere. I hope you are at least getting mostly good results. Me I would be happy one day and a week later wonder what was I thinking. Now my books are titled and seriesed in a pretty consistent manner, but they still aren't perfect. Darn close though Helen

04-02-2012, 12:32 AM	#21
louwin Newbie Nerd Posts: 114 Karma: 1000354 Join Date: Feb 2012 Location: Perth, Western Australia Device: iPad 3 64Gb Black	I "Added" about 7000 books to an empty library as the first thing I did. Result - about 40% garbage! I had hundreds of books titled "Prologue", hundreds of books by "harry", hundreds of books titled "Chapter 1" and "1", hundreds books with title appended with " by author" (hence this thread) and "Unknown" author, thousands of books with authors with names like "." "-" "--" "111111" "-----------------" and every combination of garbage. Funny thing is that I would have done much better using file names. I just threw it away and started again which is where I am at the moment. Not being in a position to edit the contents of "pdf" "rtf" "mobi" "epub" etc files leaves me with file names. I use a batch renamer with simple regex. The files are in a good structure which is why I am interested in "Adding" structured files. I suspected, and it has been confirmed, that regexs can't use directory sttructures so file names are all I have Nice talking to you. It is good to know that I not the only one

04-02-2012, 12:41 AM	#22
speakingtohe Wizard Posts: 4,812 Karma: 26912940 Join Date: Apr 2010 Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet	Allrighty then. I will say (*hopefully) one last thing. After adding books that have garbage (and yes it still happens ) I use the viewer from within the edit individual metadata dialogue and copy and paste the title/author. Helen

04-02-2012, 01:53 AM	#23
louwin Newbie Nerd Posts: 114 Karma: 1000354 Join Date: Feb 2012 Location: Perth, Western Australia Device: iPad 3 64Gb Black	Not a problem.... Last response and the thread can go to sleep. I know, I have done it. It wasn't worth trying to individually correct 3000 garbage entries. I will do it through file names Pity no one knows/said where to find the full file name specification I will have to be happy with title and author

04-02-2012, 03:20 AM	#25
louwin Newbie Nerd Posts: 114 Karma: 1000354 Join Date: Feb 2012 Location: Perth, Western Australia Device: iPad 3 64Gb Black	That probably explains why I couldn't find any.... JFI I reran that "Add to empty Library" test using file names 9304 files took 21 minutes and, as far as I can see, produced 21 entries in error involving 7 invalid authors. I don't remember how long the "contents" test took but it was 6 to 7 hours. Thanks and Goodnight (to the thread).

04-02-2012, 08:34 AM	#26
GeckoFriend Guru Posts: 614 Karma: 1164945 Join Date: Jan 2012 Location: SF Bay Area, California, USA Device: iPhone 7+, iPad Mini4, Kindle Voyage	[QUOTE=speakingtohe;2026360] After adding books that have garbage (and yes it still happens ) I use the viewer from within the edit individual metadata dialogue and copy and paste the title/author. Helen[/QUOTE This sounds useful but I can't find the viewer from within the edit metadata dialogue.

05-22-2012, 02:14 AM	#29
irmar Member Posts: 17 Karma: 10 Join Date: Nov 2010 Location: Athens, Greece Device: Pocketbook plus 360	I also have problems with the names of Calibre outputs. I've tried to play with the regex, but I'm not good at it. I want all my books to be in this format: Name Surname - Title (year) Calibre thinks differently, and it will interpret the title as author name, and/or change my title to put the author at the end. So I went do Adding books and made my regex like this: (?P<author>[^_]+) - (?P<title>.+) Still this wasn't enough. Then I went to saving books and made the expression like this: {author_sort}/{authors} - {title} Still it didn't help. What am I doing wrong? Can someone please give a helping hand? I'm renaming them all one by one and going crazy!!!

Advert

Advert