Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 03-30-2012, 03:35 AM   #1
louwin
Newbie Nerd
louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.
 
louwin's Avatar
 
Posts: 114
Karma: 1000354
Join Date: Feb 2012
Location: Perth, Western Australia
Device: iPad 3 64Gb Black
"Add Books" naming convention peeves

I'm sorry to complain (and with all due respect) since Calibre is free and all but why does it, when confronted with the text:-

Rendezvous with Rama by Arthur C Clarke

1 Spaceguard
2 Intruder
3 Rama and Sita
etc

conclude that the title of the book is "Rendezvous with Rama by Arthur C Clarke" and that the author is "1 Spaceguard"?

Well, I know why but WHY? Surely Calibre should recognise the "by" and conclude that the title is "Rendevous with Rama" and the author is "Arthur C Clarke"?

Must Calibre assume the first line is the title and the second line is the author?

I have dozens of books like this all incorectly titled by the "Add Books" option.

Maybe special significance should be given to " by " in the parsing logic?

I also have lots of books headed by:-

Book Title
Some Author

The way I assume Calibre likes things so setting an option somewhere is not the way to go

Also

What is it with the titles like:-

file:///D|/Documents%20and%20Settings/harry/Desktop/Anthony%20Piers%20-%20Up%20in%20a%20Heaval.html

which came from a pdf file with the following in the front of it

UP IN A
HEAVAL
TOR BOOKS BY PIERS ANTHONY

Calibre got the "Up in a Heaval" and the "Anthony Piers" why not guess this rather than the complex string with the interspaced "%20"s?

I'm sorry if this comes across a bit strong, no disrespect is intended
louwin is offline   Reply With Quote
Old 03-30-2012, 04:24 AM   #2
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@louwin - calibre has no "magic" to guess what is a title and what is an author it cant tell which is which, YOU have to tell it what part is the title and what part is the author.

Go to Preferences -> Adding books, and you will see the regular expression that calibre uses by default. If you search the forums you will find plenty of other examples by other users.

Or it might be that you have turned on the option to "read metadata from within the file" (which I think is the default). Which for certain file types means that calibre will look at some special fields stored in that file format that are used to specify thigns like title and author. Note this has *nothing* to do with what text you see on the page, it is special internal metadata that whoever created the book had responsibility for setting. As such it quite often contains nonsense, or fields the wrong way around, or a personal editors name instead of the actual author etc.

Calibre is garbage in, garbage out. if you want a perfectly named collection, sometimes you have to do a bit of work yourself to get it sorted. There are plugins that can help like extract ISBN in combination with metadata download.
kiwidude is offline   Reply With Quote
Old 03-30-2012, 05:45 AM   #3
louwin
Newbie Nerd
louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.
 
louwin's Avatar
 
Posts: 114
Karma: 1000354
Join Date: Feb 2012
Location: Perth, Western Australia
Device: iPad 3 64Gb Black
Response to response

Hi kiwidude,

Again with all due respect, I was a programmer of 40 years so I do know GIGO etc. I know there aren't any magic wands in programming.

Calibre is terrific but with what I suggest and a few minutes programming can be further improved.

When a string contains a " by " where a title and author are expected it would be a small modification to check the right hand side (of the " by ") against the table of author's names Calibre already has in memory and make a better guess.

With this minor mod you will improve the parsing of the available data. Some of the files, in my case, were "txt" so there isn't any meta data to worry about, just the strings in the front of the file. Even if there IS meta data to consider, don't take it at face value, further analyse it. Whatever logic that is already in place can be added to.... :?

In my case this would correctly interprete hundreds more books. This would mean many less books to manually "fix". I would imagine the same would apply to lots of others adding books to existing libraries
louwin is offline   Reply With Quote
Old 03-30-2012, 06:11 AM   #4
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
But you can cater for that "by" case yourself by modifying your regular expression in Preferences -> Adding books.

It is not worthy enough as a special case to try to build into calibre - imagine the confusion for users where you have a regular expression based approach for 99.9% of the books out there - but then when "by" happens to be somewhere in the title their regular expression gets discarded and other logic kicks in?

From years of downloading books from a wide range of sources I can say with confidence that a "Title by Author" file naming convention is by far in the minority - less than 0.1% of my collection at most. If it is a convention you have chosen for your books and is how you want to import them from, that is fine - just specify your regular expression to accommodate that. It is no different to people specifying a "Title - Author" or "Author - Title" or "xxx" convention.
kiwidude is offline   Reply With Quote
Old 03-30-2012, 08:27 AM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,442
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The House by the Lake
kovidgoyal is offline   Reply With Quote
Old 03-30-2012, 08:35 AM   #6
copyrite
Guru
copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.
 
copyrite's Avatar
 
Posts: 890
Karma: 1294851
Join Date: Sep 2010
Location: Maryland
Device: iTouch, Samsung Galaxy S4, K3K, PW, Kindle Fire 2, Nexus 7.2
I prefer Death by Chocolate

(don't mind me)
copyrite is online now   Reply With Quote
Old 03-30-2012, 09:30 AM   #7
transmitthis
Addict
transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.transmitthis ought to be getting tired of karma fortunes by now.
 
transmitthis's Avatar
 
Posts: 285
Karma: 1003542
Join Date: May 2011
Device: Google Nexus 7 16GB
Divine by Mistake

Death by the Book

(don't mind me)
transmitthis is offline   Reply With Quote
Old 03-30-2012, 04:27 PM   #8
speakingtohe
Wizard
speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.speakingtohe ought to be getting tired of karma fortunes by now.
 
Posts: 4,587
Karma: 25107878
Join Date: Apr 2010
Device: sony PRS-T1 and T3, Kobo Mini and Aura HD, Tablet
While I have seen quite a few 'edited by' prefixes, I don't see 'by' in the authors column that often. Maybe one in 1000?

I can see where this would be annoying if it was even 10% of your books, but in xx,000 ebooks this has not happened for me.

It may not be that difficult to parse, but some authors names also contain 'by' Byron, Kirby etc.

The calibre developers all maintain libraries using calibre and are way more nitpicky than most about things being as good as they can make them.

They are also extremely receptive to programmers such as yourself making actual code contributions.

Perhaps try search and replace if you are getting a lot per session.

And maybe the improvement you ask for will just magically show up one day. It has happened before.

Helen
speakingtohe is offline   Reply With Quote
Old 03-30-2012, 10:01 PM   #9
louwin
Newbie Nerd
louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.
 
louwin's Avatar
 
Posts: 114
Karma: 1000354
Join Date: Feb 2012
Location: Perth, Western Australia
Device: iPad 3 64Gb Black
further thoughts :(

I had considered

Suicide by Dark Chocolate

which is why I had suggested running "Dark Chocolate" against the list of authors already in memory(?).

If "Dark Chocolate" is an existing author then you have a correctly analysed the book, "Suicide" by "Dark Chocolate". If "Dark Chocolate" isn't an existing author then leave the book as "Suicide by Dark Chocolate" by "The Next Line". Or maybe "Dark Chocolate" IS an author but this is the first book you have of him/her.

Either way, this WILL be an improvement in the parsing logic.

My 2c worth....

I am talking 20,000 books adding 2,000 books, a herculean task that can only benefit by the improvement

120Gb main library and 10Gb of additions at a time so I'm not having to correct a "few" errors.

I experiment with complete setups on separate external hard drives using portable Calibre. If something works I apply it to my "live" library, if not I just abandon the effort and try something else or don't do the "Add" and wait for a better solution.

Failed attempts are just "copied over" or restored to "before".
louwin is offline   Reply With Quote
Old 03-30-2012, 10:19 PM   #10
louwin
Newbie Nerd
louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.
 
louwin's Avatar
 
Posts: 114
Karma: 1000354
Join Date: Feb 2012
Location: Perth, Western Australia
Device: iPad 3 64Gb Black
As to "Byron" or "Kirby", that is why I talk of " by " (or " By " or " BY ")

One space (or more spaces) before AND after the "by".

And, while we are on the subject of author names....

A bit more contentious but an author name (however it is arrived at) A N Author should be the same author as A. N. Author?
louwin is offline   Reply With Quote
Old 03-30-2012, 11:24 PM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,442
Karma: 4961459
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
As kiwidude suggested, simply use the regex to eliminate the by, it's not that hard. And as he also said, adding magic behavior to a widely used function to cater to your special case is not a good idea. Making something like parsing file names state dependant (i.e. dependent on the current state of the library) leads to unexpected and non-replicable behavior and is simply not good design.
kovidgoyal is offline   Reply With Quote
Old 03-31-2012, 08:30 AM   #12
louwin
Newbie Nerd
louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.
 
louwin's Avatar
 
Posts: 114
Karma: 1000354
Join Date: Feb 2012
Location: Perth, Western Australia
Device: iPad 3 64Gb Black
My last words on the thread

It is your program so what you say goes but ....

I created a file as below

Just a Book by A N Author

Contents

1 whenever
2 whatever

When this book was added, Calibre created an entry with title "Just a Book by A N Author" by author "Unknown", my suggestion would create an index with the title "Just a Book" by author "A N Author".

I know which I would choose.

This would benefit a few people with adding a few books. It would adversely effect nobody now or in the future.

It makes the best of a bad situation.

Your program, your decision
louwin is offline   Reply With Quote
Old 03-31-2012, 08:48 AM   #13
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,224
Karma: 1334002
Join Date: Oct 2010
Location: London, UK
Device: Kindle Paperwhite 3G, iPad 3, iPad Air
@louwin - you have repeatedly ignored everything which we have posted in response to your query.

As has been said multiple times you can have calibre import your titles in the way you want - by changing your regular expression. If you are importing titles under different naming conventions, you can use a plugin like Quick Preferences to quickly change between regex expressions before adding each batch of books.

And to perhaps make it more blunt, a naming convention of "Title by Author" is an *idiotic* file naming convention to use, with numerous examples given in this thread of titles which show why. Since Kovid agrees with my thoughts on this, it just is not going to happen - it is a *minor* edge case that is not worthy of consideration because of all the cases where workarounds would fall over, causing more confusion and pain to everyone else.
kiwidude is offline   Reply With Quote
Old 03-31-2012, 10:22 AM   #14
louwin
Newbie Nerd
louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.louwin ought to be getting tired of karma fortunes by now.
 
louwin's Avatar
 
Posts: 114
Karma: 1000354
Join Date: Feb 2012
Location: Perth, Western Australia
Device: iPad 3 64Gb Black
I didn't create the convention, I just happen to have a lot of books using it

It is a pity you have made it a personnal issue
louwin is offline   Reply With Quote
Old 03-31-2012, 10:29 AM   #15
copyrite
Guru
copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.copyrite ought to be getting tired of karma fortunes by now.
 
copyrite's Avatar
 
Posts: 890
Karma: 1294851
Join Date: Sep 2010
Location: Maryland
Device: iTouch, Samsung Galaxy S4, K3K, PW, Kindle Fire 2, Nexus 7.2
Errr these nice folks are giving you a great work around. You should try it.

Kiwidude is one sharp kiwi!
copyrite is online now   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
"The add books process seems to have hung" joleta Calibre 6 06-16-2012 02:26 AM
Calibre closes after clicking "Add Books" button JeweledSunshine Devices 4 09-25-2011 12:50 PM
Problem with "Add Books" regex wygit Library Management 9 07-24-2011 12:24 PM
Add books to "Empty Book" SmiLynnLady Library Management 8 02-10-2011 09:32 AM
[Enhancement suggestion] Folders when save books in "Add Books" function simonbcn Calibre 1 08-30-2009 12:59 PM


All times are GMT -4. The time now is 05:47 AM.


MobileRead.com is a privately owned, operated and funded community.