@chaley – Sony PRS505 vs PRS650 - Book sorting & Metadata Plugboard

jackie_w · 12-17-2010, 04:30 PM

Before I start, apologies that this is such a long post. If you don't have a Sony device it probably won't interest you at all.

There have been a couple of recent posts from PRS-650 owners regarding difficulty viewing their complete on-device book list in the sequence – Author, Series, Series_index, Title

As I have just received a PRS-650

as a birthday present, I thought I would document my experience of the differences between the PRS-505 and PRS-650 as regards book sorting and the Metadata Plugboard.

I have the Plugboard set as follows for use with both the 505 and the 650:

Code:

device_db:PRS505 = ([{author_sort}] -> authors), ([{series:lookup(\s,#serinit,series)}{series_index:>2s|| - }{title}] -> title), ([{series}{series_index:0>2s| | - }{title}] -> title_sort)

where #serinit is a custom composite column which abbreviates Series_name to initials, template is:

Code:

{series:re(([^\s])[^\s]*(\s|$),\1)}

Sort all books by Author (see attached image AllbyAuth_505vs650.jpg)

PRS-505: seems to sort by authors, title_sort.
Given the Plugboard setting this means author_sort, series, series_index, title – exactly what I want.
PRS-650: seems to sort by authors only
i.e. titles for any given author are in no particular order that I can figure and is therefore no use to me.

Sort all books by Title (see attached image AllbyTitle_505vs650.jpg)

Relevant info is that I have the following calibre tweaks set:

Code:

title_series_sorting = 'strictly_alphabetic'
save_template_title_series_sorting = 'strictly_alphabetic'
title_sort_articles=r'^$'

i.e. 'The Client' should sort under 'T' not 'C' everywhere.

PRS-505: seems to sort by title_sort.
Given the Plugboard setting this means series, series_index, title – exactly what I would expect.
PRS-650: seems to sort by title not title_sort. And also, despite my calibre tweaks, 'The Hobbit' sorts under 'H' and 'A Time to Kill' sorts under 'T'.

Sort sequence within Sony Collections (see attached image Coll_505vs650.jpg)

Relevant info is that I have a custom composite column named #coll505sort with a template

Code:

{author_sort:titlecase()} - {series}{series_index:0>2s| | - }{title}

and the calibre tweak:

Code:

sony_collection_sorting_rules = [(['*'], '#coll505sort')]

i.e. books in Collections should sort by author_sort, series, series_index, title

PRS-505: Collections sort as expected
PRS-650: Collections sorted 'By Added Order' sort exactly the same as the 505.
I did notice on both readers, that my sample calibre Catalog ({authors} = calibre), sorted after 'Tolkien' and not between 'Baxter' and 'Grisham'. I guess this means the sort is case-sensitive. This is not a problem for me but I did wonder whether it was intentional.
[Edit: Fixed this by amending the coll505sort template above in blue.]

I was still left wondering how I was going to get my PRS-650 to sort the full book list by author, series, series_index, title. I finally achieved this by using
'Sort all books by File Name' (see attached image AllbyFileName_650.jpg).

In addition I also had to change my Preferences – Plugins – Device interface – Sony – Save template to

Code:

{author_sort} - {series}{series_index:0>2s| | - }{title}

to add the author name to the beginning of the FileName. It also meant deleting all the books on the 650 and re-Sending to Device.

The layout is not as neat as the PRS-505's 'Books by Author' because the whole filename is shown under the book title rather than just the author. It looks a bit "busy", but at least the sequence is correct, and if you use Option – View:Thumbnail rather than View:List most of the Filename is truncated leaving not much more than the Author visible.

I am merely passing on information here, not requesting any bug-fixes. I'm not even sure there are any bugs, just differences in how the 2 Sony models do things.

chaley · 12-17-2010, 05:14 PM

It would be interesting to look at the sony DB to see what is in the fields. Does title_sort for, for example, 'The Hobbit' contain 'Hobbit, The'? The bug I am imagining is that the plugboard is for some reason not affecting what is written in the DB. Why this would be happening is a mystery, but that doesn't mean it isn't happening.

The other possibility is that the 650 is always rereading information from the book, tossing all the plugboard database information. This can happen if the 650 uses a different algorithm to detect new books than the other sonys have used up to now. If you look at the database and don't see the information from the plugboard, then something like this might be happening. The definitive test, assuming metadata management is automatic, would be to start calibre, plug in the device, wait for the connection to stabilize, then capture a copy of the DB. Next eject the device, stop calibre, let the device finish its business, then plug the device in again without starting calibre. Capture the DB at this point and compare them. This would tell us what the reader changed after disconnecting from calibre.

Do you have metadata management set to automatic? If so, then if you run calibre under debug mode and plug in the device, do you see the plugboard lines in the debug log?

jackie_w · 12-17-2010, 06:08 PM

Quote:

Originally Posted by chaley

Do you have metadata management set to automatic?

Yes I do have metadata management set to automatic. I will do any tests you think may be useful.

Quote:

Originally Posted by chaley

If so, then if you run calibre under debug mode and plug in the device, do you see the plugboard lines in the debug log?

Sorry... please remind me what to type on the commandline to do this.

To be going on with...

Quote:

Originally Posted by chaley

The definitive test, assuming metadata management is automatic, would be to start calibre, plug in the device, wait for the connection to stabilize, then capture a copy of the DB.

Contents of database/cache in attached file Before.zip

Quote:

Originally Posted by chaley

Next eject the device, stop calibre, let the device finish its business, then plug the device in again without starting calibre. Capture the DB at this point and compare them. This would tell us what the reader changed after disconnecting from calibre.

Contents of database/cache in attached file After.zip

I see no sign of "Hobbit, The" anywhere in media.xml, unless I'm looking in the wrong place.

The plugboard must be working - at least to some extent - because as you can see in the images I attached earlier - authors has been correctly replaced by author_sort and Title has been correctly replaced with the composite of (abbrev series_name series_index - title)

jackie_w · 12-17-2010, 06:48 PM

Quote:

Originally Posted by chaley

... if you run calibre under debug mode and plug in the device, do you see the plugboard lines in the debug log?

OK, I took a guess and typed this on the commandline

Code:

calibre-debug -g > c:\debug.log

If I guessed wrong please let me know and I'll re-do it.

This is the contents of debug.log

Spoiler:

Code:

Starting up...
Started up in 10.6710000038
Job: 1 Get device information finished
No details available.
DEBUG:    0.0 PRS505: starting fetching books for card None
DEBUG:    0.0 USBMS: Fetching list of books from device. oncard= None
DEBUG:    0.0 USBMS: dirs are: H:\ database/media/books
DEBUG:    0.2 USBMS: count found in cache: 20, count of files in metadata: 20, need_sync: False
DEBUG:    0.2 USBMS: Finished fetching list of books from device. oncard= None
DEBUG:    0.2 Building XMLCache... {0: 'H:\\database\\cache\\media.xml'}
DEBUG:    0.2 Done building XMLCache...
DEBUG:    0.2 Updating JSON cache: 0
DEBUG:    0.2 Start build_id_playlist_map
DEBUG:    0.2 Removing playlist id: 71 Purchased
DEBUG:    0.2 after cleaning playlists
DEBUG:    0.2 Finish build_id_playlist_map. Found 20
DEBUG:    0.2 Finished updating JSON cache: 0
DEBUG:    0.2 PRS505: finished fetching books for card None
DEBUG:    0.2 PRS505: starting fetching books for card carda
DEBUG:    0.2 USBMS: Fetching list of books from device. oncard= carda
DEBUG:    0.2 Building XMLCache... {0: 'H:\\database\\cache\\media.xml'}
DEBUG:    0.2 Done building XMLCache...
DEBUG:    0.2 PRS505: finished fetching books for card carda
DEBUG:    0.2 PRS505: starting fetching books for card cardb
DEBUG:    0.2 USBMS: Fetching list of books from device. oncard= cardb
DEBUG:    0.2 Building XMLCache... {0: 'H:\\database\\cache\\media.xml'}
DEBUG:    0.2 Done building XMLCache...
DEBUG:    0.2 PRS505: finished fetching books for card cardb
Job: 2 Get list of books on device finished
No details available.
DEBUG:    2.2 PRS505: started sync_booklists
DEBUG:    2.2 Building XMLCache... {0: 'H:\\database\\cache\\media.xml'}
DEBUG:    2.3 Done building XMLCache...
DEBUG:    2.3 PRS505: collection fields: [u'tags']
Device using plugboard device_db PRS505 [[u'{author_sort}', u'authors'], [u'{series:lookup(\\s,#serinit,series)}{series_index:>2s|| - }{title}', u'title'], [u'{series}{series_index:0>2| | - }{title}', u'title_sort']]
DEBUG:    2.3 PRS505: use plugboards [[u'{author_sort}', u'authors'], [u'{series:lookup(\\s,#serinit,series)}{series_index:>2s|| - }{title}', u'title'], [u'{series}{series_index:0>2| | - }{title}', u'title_sort']]
DEBUG:    2.3 Starting update [u'tags']
DEBUG:    2.3 Start build_id_playlist_map
DEBUG:    2.3 Removing playlist id: 71 Purchased
DEBUG:    2.3 after cleaning playlists
DEBUG:    2.3 Finish build_id_playlist_map. Found 20
DEBUG:    2.3 Updating XML Cache: 0
DEBUG:    2.4 Timezone votes: 20 GMT, 0 LTZ, use_tz_var=True
DEBUG:    2.4 Starting update_playlists [u'tags'] 0
DEBUG:    2.4 Starting get_collections: on_connect
DEBUG:    2.4 Renaming rules: {}
DEBUG:    2.4 Sorting rules: [(['*'], '#coll505sort')]
DEBUG:    2.4 update_playlists: finished building maps
DEBUG:    2.4 Finishing update_playlists
DEBUG:    2.4 In update/ Starting refresh of device_collections
DEBUG:    2.4 Start build_id_playlist_map
DEBUG:    2.4 after cleaning playlists
DEBUG:    2.4 Finish build_id_playlist_map. Found 20
DEBUG:    2.4 Running fix_ids()
DEBUG:    2.4 Finished running fix_ids()
DEBUG:    2.4 Finished update
DEBUG:    2.5 USBMS: starting sync_booklists
DEBUG:    2.7 USBMS: finished sync_booklists
DEBUG:    2.7 PRS505: finished sync_booklists
Job: 3 Send metadata to device finished
No details available.
get_metadata cache: 26511 calls, 83.32% misses

chaley · 12-18-2010, 03:46 AM

Thank you for both things. You did guess correctly on the debug log. On that topic, 0.7.34 has an easier way to do it, directly from the gui. Start calibre, then go to the preferences pulldown and select 'restart calibre in debug mode'. Calibre will restart. When next you quit, you will be presented with a window showing the log.

The debug log shows that everything I expected to happen, did.

As for the databases, I am mystified. There is no explanation there for how the 650 could decide to sort "The Hobbit" by 'H'. I am beginning to wonder if it is ignoring the database and using the metadata from the books themselves. Doing so would be slow. It would, however, explain what is happening. If you wish to play, you can test this. Duplicate your device_db plugboard, using format 'epub' and destination 'PRS505'. Next, resend The Hobbit. If it is sorted correctly, then we have strong evidence that the 650 is getting its info from the OPF in the epub instead of from its DB. Note that the title in the epub's OPF is really changed, so the file can't serve as a backup.

Another explanation, perhaps more probable, is that the 650 is doing article stripping, removing 'A' and 'The' before sorting. You can test this by using the plugboard to replace the blanks with the '_' character, which should defeat the matching algorithm. Start with the title_sort field. If changing that has no effect, then do both title_sort and title. The plugboard would look like:

Code:

device_db:PRS505 = (
[{author_sort}] -> authors), ([{series:lookup(\s,#serinit,series)}{series_index:>2s|| - }{title}] -> title), 
([{series}{series_index:0>2| | - }{title:re( ,_)}] -> title_sort)

Note that the re call is re(SPACE,UNDERSCORE).

Thanks again for your help!

jackie_w · 12-18-2010, 09:36 AM

Quote:

Originally Posted by chaley

Thank you for both things. You did guess correctly on the debug log. On that topic, 0.7.34 has an easier way to do it, directly from the gui. Start calibre, then go to the preferences pulldown and select 'restart calibre in debug mode'. Calibre will restart. When next you quit, you will be presented with a window showing the log.

Thanks, I've updated to v7.34 now

Quote:

Originally Posted by chaley

If you wish to play, you can test this. Duplicate your device_db plugboard, using format 'epub' and destination 'PRS505'. Next, resend The Hobbit. If it is sorted correctly, then we have strong evidence that the 650 is getting its info from the OPF in the epub instead of from its DB. Note that the title in the epub's OPF is really changed, so the file can't serve as a backup.

I added this to the plugboard (I didn't delete the device_db:PRS505 one)

Code:

epub:PRS505 = ([{author_sort}] -> authors), ([{series:lookup(\s,#serinit,series)}{series_index:>2s|| - }{title}] -> title), ([{series}{series_index:0>2s|| - }{title}] -> title_sort)

When sorting by Title on the PRS-650 'The Hobbit' still sorts under 'H' not 'T'.

In case it helps, here is the <metadata> section from the epub in my calibre library.

Spoiler:

Now I'll get on with the next test.

jackie_w · 12-18-2010, 10:02 AM

Quote:

Originally Posted by chaley

Another explanation, perhaps more probable, is that the 650 is doing article stripping, removing 'A' and 'The' before sorting. You can test this by using the plugboard to replace the blanks with the '_' character, which should defeat the matching algorithm. Start with the title_sort field. If changing that has no effect, then do both title_sort and title. The plugboard would look like:

Code:

device_db:PRS505 = (
[{author_sort}] -> authors), ([{series:lookup(\s,#serinit,series)}{series_index:>2s|| - }{title}] -> title), 
([{series}{series_index:0>2| | - }{title:re( ,_)}] -> title_sort)

Note that the re call is re(SPACE,UNDERSCORE).

After removing the epub:PRS505 plugboard I changed the title_sort designation in device_db:PRS505 as you suggested. This had no effect.

I then also changed the title designation. As predicted 'The Hobbit' now sorts under 'T'. I've attached screencaps of the PRS650 sorting all books by Title.

chaley · 12-18-2010, 11:20 AM

Well, I think I now understand what it is doing.

1) Title sorts use the 'Title' attribute, not the title_sort.
2) The sony is doing article stripping.
3) The sony does not do subsorting (regards display by author)

Regarding display by author, I matched the screen shot of the by-author list with the db. The books are shown in the order they appear in the database, which is the order that they were originally sent. I might be able to do something about this, for example sorting the db itself by title, which should fix the 'by author' display. I will look at it.

Regarding the by title display: we now have a workaround, although not the prettiest one. Are you OK with using the underscores? Note that you will need to do the same thing if you ever have a series that begins with an article.

We do have another option. Calibre could create collections 'By Author' and 'By Title', using the added order stuff that seems to work. Would this be useful?

Time for some reflection, while watching the snow fall...

jackie_w · 12-18-2010, 01:06 PM

The following responses are purely personal preferences and may not reflect the majority of PRS-650 owners.

Quote:

Originally Posted by chaley

Regarding display by author, I matched the screen shot of the by-author list with the db. The books are shown in the order they appear in the database, which is the order that they were originally sent. I might be able to do something about this, for example sorting the db itself by title, which should fix the 'by author' display. I will look at it.

If this option was available it would be my No.1 choice. I don't see how the 650's current default for Sort by Author would be useful to anyone. It must be a bug in the 650 (and 350???) firmware as they handled it sensibly on the PRS505. I never had a PRS600 so I don't know about the intermediate model's firmware.

I can live with my 'Sort by Filename' workaround for as long as it takes.

Quote:

Originally Posted by chaley

Regarding the by title display: we now have a workaround, although not the prettiest one. Are you OK with using the underscores? Note that you will need to do the same thing if you ever have a series that begins with an article.

Because I have passed the PRS505 to my husband and the same Plugboard is used for both Sonys I, personally, don't want to replace Title spaces with underscores.

Personally, I don't keep hundreds of books on my reader so I can easily live with the 650's current default. The main thing is being aware what to expect.

Quote:

Originally Posted by chaley

We do have another option. Calibre could create collections 'By Author' and 'By Title', using the added order stuff that seems to work. Would this be useful?

Yes, as long as it was All Books.

I'd already considered adding a special tag to all books in the calibre Library which were currently on the 650. In fact I already did it for this testing and used it to do the screencaps for Collections in my initial post. It works very well, but I know I'd never remember to keep adding/removing the tag.

If you can create the 'By Author' and 'By Title' Collections automatically then they'd be my No.2 choice. No.2 rather than No.1 on the flimsy reason that 'Sort all by Author' is one button-press from the Home screen and Collections, select new 'By Author' is a minimum of two depending on which page the "special" Collections appear. I'm attaching another screencap of the Collections page(s) so you can see what it looks like when you select 'Collections' from the Home screen. The first 4 are Sony defaults and nothing to do with anything I've done.

Thank you, so much for taking the time to look at this in such depth. I will be happy to do any testing you want, so don't hesitate to ask. If the weather continues we'll all be confined to barracks anyway and needing something to keep busy. Any improvement you can come up with will be very much appreciated.

chaley · 12-21-2010, 08:44 AM

OK, time for another experiment, if you are up to it.

I want you to install a test Sony driver that sorts the database records in title order.

What you do is:
1) find your calibre installation directory. It should be c:\program files\calibre2.
2) In that directory, you should be able to navigate to Lib\site-packages\calibre\devices\prs505
3) In the prs505 directory, you should see the file sony_cache.pyo.

Continue only if you have found the file sony_cache.pyo

4) Ensure calibre is not running.
5) rename sony_cache.pyo to sony_cache.pyo.save.
6) Download the attached file sony_cache.zip. Extract the file sony_cache.py from the zip file, and copy the .py file to the folder now containing sony_cache.pyo.save.
7) Start calibre, connect your device, let it update metadata, then check if the order of books in the author list has changed. I hope so.
8) for completeness, post the cache.xml file so I can verify that the sort-by-title worked.
9) now, or certainly before upgrading, delete the file sony_cache.py and rename the file sony_cache.pyo.save back to sony_cache.pyo.

Thanks again!

jackie_w · 12-21-2010, 09:53 AM

Some improvement in the Baxters, but still something amiss in the Grishams and Tolkiens.

Have attached screencap of new Sort by Author and all 3 files from database/cache

chaley · 12-21-2010, 10:13 AM

The sorted database and the screen shot are more out of sync than the last one, which helps understand (perhaps). The 650 seems to be displaying the books in reverse-time order. The latest book added is the one that is first in the list. This correctly predicts the order for all of the Grishams, Rowlings, and Tolkiens.

Unfortunately, there isn't much I can do about the Sony's idea of a date. If I change it, the sony will change it back.

In any event, sorting isn't useful. Delete the file I gave you and restore calibre back to normal. I will now look at automatic creation of collections by_author (actually, by title within author) and by_title

jackie_w · 12-21-2010, 10:35 AM

Gosh, isn't this fun?! You have to wonder whether Sony test their new models with real users.

I'll be more than happy to test the new Collections if/when you need it.

chaley · 12-21-2010, 11:28 AM

OK, you volunteered.

This time what you do is:
1) find your calibre installation directory. It should be c:\program files\calibre2.
2) In that directory, you should be able to navigate to Lib\site-packages\calibre\devices\usbms
3) In the usbms directory, you should see the file books.pyo.

Continue only if you have found the file books.pyo

4) Ensure calibre is not running.
5) rename books.pyo to books.pyo.save.
6) Download the attached file books.zip. Extract the file books.py from the zip file, and copy the .py file to the folder now containing books.pyo.save.
7) Start calibre, connect your device, let it update metadata. when it finishes, you should have two new collections, '%Books by author' and '%Books by title'. Let me know if they exist and if their content is in the right order.
8) now, or certainly before upgrading, delete the file books.py and rename the file books.pyo.save back to books.pyo.

Note that if this works, in the production release these collections will be controlled by tweaks. You supply a collection name. If the name is '', the collections won't be built.

jackie_w · 12-21-2010, 12:16 PM

The good news is that the 2 new Collections appear correctly in the Collection list (1st attached image).

The bad news is that the sort order in each is "unusual".

%All by author (see 2nd attached image)

Authors out of sequence.
Within each Author, books are in "simple" Title sequence, i.e Title as if not modified by Plugboard to have abbrev. series/index added to the front.

%All by title (see 3rd attached image)

Similar to above, Titles are in "simple" Title sequence ... ahh... but is this what you intended for All by Title?

I've also attached the database/cache files.

12-17-2010, 04:30 PM	#1
jackie_w Grand Sorcerer Posts: 6,247 Karma: 16539642 Join Date: Sep 2009 Location: UK Device: ClaraHD, Forma, Libra2, Clara2E, LibraCol, PBTouchHD3	@chaley – Sony PRS505 vs PRS650 - Book sorting & Metadata Plugboard Before I start, apologies that this is such a long post. If you don't have a Sony device it probably won't interest you at all. There have been a couple of recent posts from PRS-650 owners regarding difficulty viewing their complete on-device book list in the sequence – Author, Series, Series_index, Title As I have just received a PRS-650 as a birthday present, I thought I would document my experience of the differences between the PRS-505 and PRS-650 as regards book sorting and the Metadata Plugboard. I have the Plugboard set as follows for use with both the 505 and the 650: Code: device_db:PRS505 = ([{author_sort}] -> authors), ([{series:lookup(\s,#serinit,series)}{series_index:>2s\|\| - }{title}] -> title), ([{series}{series_index:0>2s\| \| - }{title}] -> title_sort) where #serinit is a custom composite column which abbreviates Series_name to initials, template is: Code: {series:re(([^\s])[^\s](\s\|$),\1)} Sort all books by Author* (see attached image AllbyAuth_505vs650.jpg) PRS-505: seems to sort by authors, title_sort. Given the Plugboard setting this means author_sort, series, series_index, title – exactly what I want. PRS-650: seems to sort by authors only i.e. titles for any given author are in no particular order that I can figure and is therefore no use to me. Sort all books by Title (see attached image AllbyTitle_505vs650.jpg) Relevant info is that I have the following calibre tweaks set: Code: title_series_sorting = 'strictly_alphabetic' save_template_title_series_sorting = 'strictly_alphabetic' title_sort_articles=r'^$' i.e. 'The Client' should sort under 'T' not 'C' everywhere. PRS-505: seems to sort by title_sort. Given the Plugboard setting this means series, series_index, title – exactly what I would expect. PRS-650: seems to sort by title not title_sort. And also, despite my calibre tweaks, 'The Hobbit' sorts under 'H' and 'A Time to Kill' sorts under 'T'. Sort sequence within Sony Collections (see attached image Coll_505vs650.jpg) Relevant info is that I have a custom composite column named #coll505sort with a template Code: {author_sort:titlecase()} - {series}{series_index:0>2s\| \| - }{title} and the calibre tweak: Code: sony_collection_sorting_rules = [([''], '#coll505sort')] i.e. books in Collections should sort by author_sort, series, series_index, title PRS-505: Collections sort as expected PRS-650: Collections sorted 'By Added Order' sort exactly the same as the 505. I did notice on both readers, that my sample calibre Catalog ({authors} = calibre), sorted after 'Tolkien' and not between 'Baxter' and 'Grisham'. I guess this means the sort is case-sensitive. This is not a problem for me but I did wonder whether it was intentional. [Edit: Fixed this by amending the coll505sort template above in blue.] I was still left wondering how I was going to get my PRS-650 to sort the full book list by author, series, series_index, title. I finally achieved this by using 'Sort all books by File Name'* (see attached image AllbyFileName_650.jpg). In addition I also had to change my Preferences – Plugins – Device interface – Sony – Save template to Code: {author_sort} - {series}{series_index:0>2s\| \| - }{title} to add the author name to the beginning of the FileName. It also meant deleting all the books on the 650 and re-Sending to Device. The layout is not as neat as the PRS-505's 'Books by Author' because the whole filename is shown under the book title rather than just the author. It looks a bit "busy", but at least the sequence is correct, and if you use Option – View:Thumbnail rather than View:List most of the Filename is truncated leaving not much more than the Author visible. I am merely passing on information here, not requesting any bug-fixes. I'm not even sure there are any bugs, just differences in how the 2 Sony models do things. Attached Thumbnails Last edited by jackie_w; 12-18-2010 at 01:46 PM. Reason: Correction to Collection sort template

12-18-2010, 03:46 AM	#5
chaley Grand Sorcerer Posts: 12,325 Karma: 8012652 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Thank you for both things. You did guess correctly on the debug log. On that topic, 0.7.34 has an easier way to do it, directly from the gui. Start calibre, then go to the preferences pulldown and select 'restart calibre in debug mode'. Calibre will restart. When next you quit, you will be presented with a window showing the log. The debug log shows that everything I expected to happen, did. As for the databases, I am mystified. There is no explanation there for how the 650 could decide to sort "The Hobbit" by 'H'. I am beginning to wonder if it is ignoring the database and using the metadata from the books themselves. Doing so would be slow. It would, however, explain what is happening. If you wish to play, you can test this. Duplicate your device_db plugboard, using format 'epub' and destination 'PRS505'. Next, resend The Hobbit. If it is sorted correctly, then we have strong evidence that the 650 is getting its info from the OPF in the epub instead of from its DB. Note that the title in the epub's OPF is really changed, so the file can't serve as a backup. Another explanation, perhaps more probable, is that the 650 is doing article stripping, removing 'A' and 'The' before sorting. You can test this by using the plugboard to replace the blanks with the '_' character, which should defeat the matching algorithm. Start with the title_sort field. If changing that has no effect, then do both title_sort and title. The plugboard would look like: Code: device_db:PRS505 = ( [{author_sort}] -> authors), ([{series:lookup(\s,#serinit,series)}{series_index:>2s\|\| - }{title}] -> title), ([{series}{series_index:0>2\| \| - }{title:re( ,_)}] -> title_sort) Note that the re call is re(SPACE,UNDERSCORE). Thanks again for your help!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Help!, PRS650, Plugboard, book sort order rearranged into a new title	Glubbish	Calibre	11	11-30-2010 03:38 PM
Problem with Metadata Plugboard	cheveguerra	Calibre	3	10-05-2010 12:00 PM
For Charles/Chaley Merging user defined metadata	Starson17	Calibre	20	07-22-2010 10:27 AM
Sony PRS505 loosing metadata	idt1971	Calibre	2	11-22-2008 06:52 PM
Sony PRS505 loosing metadata	idt1971	Sony Reader	3	11-22-2008 07:35 AM

12-17-2010, 05:14 PM	#2
chaley Grand Sorcerer Posts: 12,325 Karma: 8012652 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	It would be interesting to look at the sony DB to see what is in the fields. Does title_sort for, for example, 'The Hobbit' contain 'Hobbit, The'? The bug I am imagining is that the plugboard is for some reason not affecting what is written in the DB. Why this would be happening is a mystery, but that doesn't mean it isn't happening. The other possibility is that the 650 is always rereading information from the book, tossing all the plugboard database information. This can happen if the 650 uses a different algorithm to detect new books than the other sonys have used up to now. If you look at the database and don't see the information from the plugboard, then something like this might be happening. The definitive test, assuming metadata management is automatic, would be to start calibre, plug in the device, wait for the connection to stabilize, then capture a copy of the DB. Next eject the device, stop calibre, let the device finish its business, then plug the device in again without starting calibre. Capture the DB at this point and compare them. This would tell us what the reader changed after disconnecting from calibre. Do you have metadata management set to automatic? If so, then if you run calibre under debug mode and plug in the device, do you see the plugboard lines in the debug log?

12-18-2010, 11:20 AM	#8
chaley Grand Sorcerer Posts: 12,325 Karma: 8012652 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Well, I think I now understand what it is doing. 1) Title sorts use the 'Title' attribute, not the title_sort. 2) The sony is doing article stripping. 3) The sony does not do subsorting (regards display by author) Regarding display by author, I matched the screen shot of the by-author list with the db. The books are shown in the order they appear in the database, which is the order that they were originally sent. I might be able to do something about this, for example sorting the db itself by title, which should fix the 'by author' display. I will look at it. Regarding the by title display: we now have a workaround, although not the prettiest one. Are you OK with using the underscores? Note that you will need to do the same thing if you ever have a series that begins with an article. We do have another option. Calibre could create collections 'By Author' and 'By Title', using the added order stuff that seems to work. Would this be useful? Time for some reflection, while watching the snow fall...

12-21-2010, 10:13 AM	#12
chaley Grand Sorcerer Posts: 12,325 Karma: 8012652 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	The sorted database and the screen shot are more out of sync than the last one, which helps understand (perhaps). The 650 seems to be displaying the books in reverse-time order. The latest book added is the one that is first in the list. This correctly predicts the order for all of the Grishams, Rowlings, and Tolkiens. Unfortunately, there isn't much I can do about the Sony's idea of a date. If I change it, the sony will change it back. In any event, sorting isn't useful. Delete the file I gave you and restore calibre back to normal. I will now look at automatic creation of collections by_author (actually, by title within author) and by_title

12-21-2010, 10:35 AM	#13
jackie_w Grand Sorcerer Posts: 6,247 Karma: 16539642 Join Date: Sep 2009 Location: UK Device: ClaraHD, Forma, Libra2, Clara2E, LibraCol, PBTouchHD3	Gosh, isn't this fun?! You have to wonder whether Sony test their new models with real users. I'll be more than happy to test the new Collections if/when you need it.