Commas in LRF metadata

kevin_boone · 02-10-2009, 04:05 AM

Hi

Sorry if this has been noticed before, but it seems that calibre doesn't like commas in the author field of metadata in LRF files it imports. So if the author is

`Blogs, Fred'

it appears in calibre as

`Blogs & Fred'

I can correct it easily enough, but when I change it back to a comma and export it to my PRS-505, the authors ends up as

`Blogs| Fred'

I'm not sure whether it's the commas or the spaces it doesn't like.

Best wishes
Kevin

itimpi · 02-10-2009, 06:10 AM

Calibre is treating the ampersand as specifying multiple authors. Somewhere as an optimisation in the handling of the dialog for changing metadata, it is converting the comma to an ampersand. Quite why the comma then ends up as a vertical bar I have no idea.

Internally calibre seems to hold authors in 'firstname lastname' format, with the sort field normally being set to 'lastname, firstname'. You may find it esier to simply live with this convention.

kevin_boone · 02-10-2009, 07:06 AM

Quote:

Originally Posted by itimpi

Internally calibre seems to hold authors in 'firstname lastname' format, with the sort field normally being set to 'lastname, firstname'. You may find it esier to simply live with this convention.

Thanks for the prompt reply, even though it's not the one I wanted

It's not irksome enough for me to stop using calibre, but it's a bit odd, nonetheless.

I think what you're saying (sorry if I'm misunderstanding this) is that when Calibre reads an author like this `Bloggs, Fred' it interprets it as two different authors -- one called (something) Bloggs and one called (something) Fred?

The problem is that that if I wanted to say that the book was by two different author, I would write the authors as `Bloggs and Scuggins' or whatever. I'm not sure why the software needs to make assumptions that the person who formatted the file meant something different from what he or she said. To complicate matters, the books I've downloaded from this site have so far all had the author in the form `Bloggs, Fred'.

And (so far as I can tell) the PRS-505 does not honour the `author sort' field that Calibre uses. I don't know if that's a Calibre issue or a PRS-505 one. But the PRS order authors by the displayed name field. Since I have books by Charles Dickens, Charles Darwin, Charles Grant, and Charles Williams, ordering by display name isn't terribly useful.

I can live with my PRS-505 displaying an authors as `Darwin|Charles' if the alternative is to be forced to put all the Charleses together in a bunch. But it would be nice if there was a way around this.

itimpi · 02-10-2009, 07:44 AM

Note that to separate two authors in calibre you must use the ampersand character and not the word 'and'.

Just for interest, how are you putting the comma back in? I am just wondering, because if it is via the metadata dialog, and you are using the option to look up metadata from the web it could be the server that is reversing the order?

The comma being converted to the vertical bar character might well be a bug. Guess we need to wait until Kovid is online to comment.

I would be very happy if at some time clibre has the option to store author names in the reverse order. I think this is on the list of things for Kovid to do in the future, but it is a non-trivial change and so is unlikely to appear in the near future.

kevin_boone · 02-10-2009, 09:44 AM

Well, if it's on the to-do list, then I guess I'll have to be content with that

It's not a huge big deal, certainly not something I can't live with.

It seems odd, all the same. I note that if I edit the metadata so the author is `Darwin, Charles' then I end up with a directory called `Darwin_ charles'. It seems the comma becomes an underscore there, instead of a | in the PRS505. I wonder if this is some freakiness that goes on to avoid writing illegal filenames? Perhaps on some platforms a comma is not allowed in filenames?

itimpi · 02-10-2009, 09:46 AM

There is certainly a sanitization of filenames to something that is allowed on all platforms and that seems to involve replacing awkward characters with underscores. However that is purely in the folder/file names and not in the metadata in the database.

gwynevans · 02-10-2009, 10:03 AM

Quote:

Originally Posted by kevin_boone

And (so far as I can tell) the PRS-505 does not honour the `author sort' field that Calibre uses. I don't know if that's a Calibre issue or a PRS-505 one.

I think that's the PRS-505 rather than Calibre, unfortunately!

kevin_boone · 02-10-2009, 11:21 AM

Quote:

Originally Posted by gwynevans

I think that's the PRS-505 rather than Calibre, unfortunately!

Well, fair enough; but it does mean that the `author' field in Calibre _must_ be set to the way I want the books both sorted and displayed, if I want to get sensible ordering on the PRS-505.

I suppose I could try `Darwin - Charles' or `Darwin ; Charles' or something, if it's just commas that are tricky. But for my sanity I can't really have the reader list authors in first-name order -- I'd never find anything :/

kovidgoyal · 02-10-2009, 05:22 PM

This is a bug, you can set the author field to a, b and it should show up as a, b on the reader. Will be fixed in the next release.

kevin_boone · 02-11-2009, 03:46 AM

Quote:

Originally Posted by kovidgoyal

This is a bug, you can set the author field to a, b and it should show up as a, b on the reader. Will be fixed in the next release.

You're a star. Will the same fix tackle the problem of commas coming out as `|'characters on the reader? Or is that something different altogether?

kovidgoyal · 02-11-2009, 04:19 AM

the fix is for the authors becomig a|b on the reader. You will still have to change the authors form a&b to a,b in the GUI

kevin_boone · 02-11-2009, 04:30 AM

OK, Thanks

I've just noticed that the isbndb server has authors in `Fred Bloggs' format. Yet the PRS505 _requires_ `Bloggs, Fred' to sort properly. And many of the LRF books posted on this forum have `Bloggs, Fred'. I presume that's because they've been posted by PRS owners who are aware of this restriction.

Grrrr! Why is there no consistency in this area? It wouldn't matter, except that I'm in the process of importing about 2000 books, and I want to minimse the amount of manual fiddling I have to do.

Xenophon · 02-11-2009, 10:41 AM

Part of the issue is that the LRF format has a second author field (named "reading as" I believe) that is intended to represent the authors name for sorting purposes. But none of Sony's readers pay any attention to the field!!!

So what's happening is that Calibre does its best to use the existing Author data to fill in a reasonable guess at the Author Sort field. But that guess is often wrong because author names are really complicated (check out the ALA's guidelines on this sometime -- it's really scary)! Then go file a BUG with Sony for ignoring part of their own format's meta-data.

Xenophon

kevin_boone · 02-11-2009, 02:06 PM

Quote:

Originally Posted by Xenophon

Part of the issue is that the LRF format has a second author field (named "reading as" I believe) that is intended to represent the authors name for sorting purposes. But none of Sony's readers pay any attention to the field!!!

Yeah, I guessed as much. That's really slack

And I guess it would make no sense to cripple Calibre so that it doesn't work properly with readers that do do the right thing, just because the PRS doesn't.

And, in fact, now I've used the isbndb database it bit more, I can see that there's little consistency in how author names are represented there either. I guess I can't expect any software to cope that.

Since the PRS/Calibre combination essentially requires that the author and author-sort fields are the same, if both are to work properly, I suppose what would help me when importing large numbers of books is a way to set this for the entire collection.

Is there some programmatic way to do that? I suppose at a pinch I could use the sqlite prompt, but I don't understand the data schema and I don't want to break it.

kovidgoyal · 02-11-2009, 02:20 PM

Not unless you're willing to a write python code

02-10-2009, 04:05 AM	#1
kevin_boone Enthusiast Posts: 35 Karma: 10 Join Date: Feb 2009 Device: prs-505	Commas in LRF metadata Hi Sorry if this has been noticed before, but it seems that calibre doesn't like commas in the author field of metadata in LRF files it imports. So if the author is `Blogs, Fred' it appears in calibre as `Blogs & Fred' I can correct it easily enough, but when I change it back to a comma and export it to my PRS-505, the authors ends up as `Blogs\| Fred' I'm not sure whether it's the commas or the spaces it doesn't like. Best wishes Kevin

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Seriously thoughtful What about commas?	GraceKrispy	Lounge	115	10-18-2010 10:19 PM
Regular Expression for filenames with commas	jomaweb	Calibre	2	02-04-2010 05:27 PM
Renaming LRF files, Editing Metadata?	brandoniain	LRF	9	06-18-2009 08:13 AM
mobi-meta changes commas to semicolons	GRiker	Calibre	7	04-30-2009 05:38 AM
Editing LRF metadata?	curtw	Sony Reader	11	02-01-2008 12:43 AM

02-10-2009, 06:10 AM	#2
itimpi Wizard Posts: 4,552 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	Calibre is treating the ampersand as specifying multiple authors. Somewhere as an optimisation in the handling of the dialog for changing metadata, it is converting the comma to an ampersand. Quite why the comma then ends up as a vertical bar I have no idea. Internally calibre seems to hold authors in 'firstname lastname' format, with the sort field normally being set to 'lastname, firstname'. You may find it esier to simply live with this convention.

02-10-2009, 07:44 AM	#4
itimpi Wizard Posts: 4,552 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	Note that to separate two authors in calibre you must use the ampersand character and not the word 'and'. Just for interest, how are you putting the comma back in? I am just wondering, because if it is via the metadata dialog, and you are using the option to look up metadata from the web it could be the server that is reversing the order? The comma being converted to the vertical bar character might well be a bug. Guess we need to wait until Kovid is online to comment. I would be very happy if at some time clibre has the option to store author names in the reverse order. I think this is on the list of things for Kovid to do in the future, but it is a non-trivial change and so is unlikely to appear in the near future.

02-10-2009, 09:44 AM	#5
kevin_boone Enthusiast Posts: 35 Karma: 10 Join Date: Feb 2009 Device: prs-505	Well, if it's on the to-do list, then I guess I'll have to be content with that It's not a huge big deal, certainly not something I can't live with. It seems odd, all the same. I note that if I edit the metadata so the author is `Darwin, Charles' then I end up with a directory called `Darwin_ charles'. It seems the comma becomes an underscore there, instead of a \| in the PRS505. I wonder if this is some freakiness that goes on to avoid writing illegal filenames? Perhaps on some platforms a comma is not allowed in filenames?

02-10-2009, 09:46 AM	#6
itimpi Wizard Posts: 4,552 Karma: 950151 Join Date: Nov 2008 Device: Sony PRS-950, iphone/ipad (Marvin/iBooks/QuickReader)	There is certainly a sanitization of filenames to something that is allowed on all platforms and that seems to involve replacing awkward characters with underscores. However that is purely in the folder/file names and not in the metadata in the database.

02-10-2009, 05:22 PM	#9
kovidgoyal creator of calibre Posts: 43,860 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	This is a bug, you can set the author field to a, b and it should show up as a, b on the reader. Will be fixed in the next release.

02-11-2009, 04:19 AM	#11
kovidgoyal creator of calibre Posts: 43,860 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	the fix is for the authors becomig a\|b on the reader. You will still have to change the authors form a&b to a,b in the GUI

02-11-2009, 04:30 AM	#12
kevin_boone Enthusiast Posts: 35 Karma: 10 Join Date: Feb 2009 Device: prs-505	OK, Thanks I've just noticed that the isbndb server has authors in `Fred Bloggs' format. Yet the PRS505 _requires_ `Bloggs, Fred' to sort properly. And many of the LRF books posted on this forum have `Bloggs, Fred'. I presume that's because they've been posted by PRS owners who are aware of this restriction. Grrrr! Why is there no consistency in this area? It wouldn't matter, except that I'm in the process of importing about 2000 books, and I want to minimse the amount of manual fiddling I have to do.

02-11-2009, 10:41 AM	#13
Xenophon curmudgeon Posts: 1,481 Karma: 5748190 Join Date: Jun 2006 Location: Redwood City, CA USA Device: Kobo Aura HD, (ex)nook, (ex)PRS-700, (ex)PRS-500	Part of the issue is that the LRF format has a second author field (named "reading as" I believe) that is intended to represent the authors name for sorting purposes. But none of Sony's readers pay any attention to the field!!! So what's happening is that Calibre does its best to use the existing Author data to fill in a reasonable guess at the Author Sort field. But that guess is often wrong because author names are really complicated (check out the ALA's guidelines on this sometime -- it's really scary)! Then go file a BUG with Sony for ignoring part of their own format's meta-data. Xenophon

02-11-2009, 02:20 PM	#15
kovidgoyal creator of calibre Posts: 43,860 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Not unless you're willing to a write python code

Advert

Advert