Sudden Problem with "The" and "A" Titles

[Otter]

I've been using Calibre with my 505 for awhile now and have never had this problem before. I recently just upgraded to the latest version of Calibre (I think I was using one of the 0.6.x version before this) and transfered two books to my 505 and now when I list books by title on my reader a book like "The Baker's Boy" is listed under T instead of B. All titles with "A blah blah" and under "A" and all titles with "The blah blah" are showing under T for The. Has anyone else encountered this or know of a way to easily fix it? There are about 1500 books on my 505 and I'd hate to have to re-edit each one.

Many thanks in advance.

[Adoby]

Look under Preferences | Advanced | Tweaks

# Control title and series sorting in the library view.
# If set to 'library_order', Leading articles such as The and A will be ignored.
# If set to 'strictly_alphabetic', the titles will be sorted without processing
# For example, with library_order, The Client will sort under 'C'. With
# strictly_alphabetic, the book will sort under 'T'.
# This flag affects Calibre's library display. It has no effect on devices. In
# addition, titles for books added before changing the flag will retain their
# order until the title is edited. Double-clicking on a title and hitting return
# without changing anything is sufficient to change the sort.
title_series_sorting = 'library_order'

[Otter]

In Calibre's library view, the titles list correctly, The Baker's Boy appears under B but on my 505 for some reason it is now showing up under T.

[jackie_w]

There is a second tweak dealing with titles on the reader.

# Control how title and series names are formatted when saving to disk/sending
# to device. If set to library_order, leading articles such as The and A will
# be put at the end
# If set to 'strictly_alphabetic', the titles will be sorted without processing
# For example, with library_order, "The Client" will become "Client, The". With
# strictly_alphabetic, it would remain "The Client".
save_template_title_series_sorting = 'library_order'

[Otter]

Hrm, there is a mystery here and I think I found another clue...just no idea how to fix it. Earlier in the evening I clicked on an .lrf file to view it but the reader software (sony ugh) never opened so I didn't think much of it. After transfering some books with Calibre the alphabetical problem occurred. When I went to eject the reader within Calibre the icons disappeared but my reader still had the do not disconnect message. When I tried to "safely remove" from within windows it said the drive was still in use. Task manager just showed that the sony reader software was still running in the background even though it had never opened its own window. So it *appears* that both calibre and sony's reader software were running when I transferred the books and encountered the subsequent problem with the alphabetical title listing. I wonder if that caused the problem. And I wonder how to fix it, lol. Thanks again.

[chaley]

Originally Posted by Otter

And I wonder how to fix it, lol. Thanks again.

Some Sony devices have a database value that controls the sort for (guess what!) titles, called title_sorter. Calibre puts this value in the Sony database when it sends books. It sounds like a) your 505 can use title_sorter, and b) the title_sorter values have been removed or reset to the title.

If the database is corrupted, it will be reset and rebuilt by the reader. My guess is that your simultaneous connection resulted in a bogus database.

You can check the values by looking at the sony database, found in database/cache/media.xml. You must use an editor that understands linux line endings. On windows, wordpad does this. Look at a few books (the lines should be clear enough) to see if the title_sorter value is incorrect or missing.

One way to get calibre to reset all the values is to change the Metadata Management option (preferences -> add/save -> send to device) to Automatic. The next time you connect, calibre will update the database for all the books it recognizes. However, and this is a big caveat, it will also reset all collections to correspond to current metadata, *deleting* any collections manually added. Do not change the option if this behavior is not what you want.

[Otter]

Thanks for the info, I will try that. But what do you mean by "current metadata"? The collections currently beside each book in calibre's main library view? That would be ok, but I did "manually add" most of those in Calibre so I guess I am confused by what you mean by manually added too. Thanks much for the help!

[chaley]

Originally Posted by Otter

But what do you mean by "current metadata"? The collections currently beside each book in calibre's main library view?

Yes. "Metadata" means all the metadata stored by calibre with a book. "Current metadata" means the metadata in the library (not on the reader) at the moment the device is connected.

Manually-added collections are names that do not correspond to a value in one of the metadata fields selected in 'device customization' for collection generation. The default is series & tags. Assume you used the Sony SW or calibre's device view to add 'Foo' to the list of collections for a book. If metadata management is not 'Manual', and if 'Foo' is neither a tag on the book nor the series the book is in, then the book will be removed from the 'Foo' collection. If at the end no books are in the Foo collection, the collection itself will be removed.

You could 'try before you buy' by making a backup of the Sony memory, running calibre, changing the option, then plugging in the Sony. If you don't like the results, restore the Sony memory. If you have an SD card, you must back that up as well. If you decide to restore the Sony's memory, be sure to reset the option *before* you plug the device in again.

[Otter]

Hi again-

Thanks for the info Chaley. I ended up deleting the media.xml file and letting the 505 rebuild it upon boot up, but the problem still persists. I also tried setting the metadata mgmt to automatic and calibre sent the metadata when I connected, but I still have the same problem.

I did notice something while browsing through the titles on the 505 itself though, and am now wondering if this problem has existed for awhile and might not have anything to do with the new version of calibre and/or sony's software possibly running in the background. The problem doesn't seem to be global but rather individual files....The Curse of Chalion is listed under C as it should be but The Baker's Boy is under T...The Darkest Fire is under D, but The Caryatids is under T (on the 505, not in Calibre...I am pretty sure calibre lists them all correctly but will have to double check when I get home). Same thing happens with some titles that begin with A, but not all of them.

I'm sure there's a clue in there somewhere, but I am not sure who to proceed with finding/fixing it.

Thanks again for your help!

[theducks]

Originally Posted by Otter

Hi again-
<snip>
The problem doesn't seem to be global but rather individual files....The Curse of Chalion is listed under C as it should be but The Baker's Boy is under T...The Darkest Fire is under D, but The Caryatids is under T (on the 505, not in Calibre...I am pretty sure calibre lists them all correctly but will have to double check when I get home). Same thing happens with some titles that begin with A, but not all of them.

I'm sure there's a clue in there somewhere, but I am not sure who to proceed with finding/fixing it.

Thanks again for your help!

Were those title sent to the Sony under an older version of Calibre?
(did yo fail to remove the old (pre Library order) version from the device (I had to clean up my PEz after this update appeared)

[chaley]

Sounds like a) you should check the tweak that Adoby mentions in post 2 above, b) should run calibre in debug mode when you connect the device, and c) look at media.xml to see if the titleSorter value corresponds to the sort order you are seeing.

If c) is true, then the question becomes 'why isn't calibre resetting it?' We can look at that problem once we know we have it.

If you are on windows, you run calibre in debug mode by starting a cmd box and typing
calibre-debug -g
Look for any exceptions or messages that say something is wrong. Don't worry about 'remapping' messages. Please look for the message about what timezone calibre is using to write dates into Sony's database.

Edit: also note that calibre will update the database only for books it knows about -- that have ondevice = Yes. Other books are on their own...

[Otter]

Hrm, I have so many books on the device that I am sure batches of them have been sent using different versions of calibre. I did wipe the device a few months ago when I had a terrible corrupt database issue come up, but there have been a few updates of calibre since then. I am not sure what you mean though when you asked "did you fail to remove the old (pre Library order?) version of from the device?

[theducks]

Originally Posted by Otter

Hrm, I have so many books on the device that I am sure batches of them have been sent using different versions of calibre. I did wipe the device a few months ago when I had a terrible corrupt database issue come up, but there have been a few updates of calibre since then. I am not sure what you mean though when you asked "did you fail to remove the old (pre Library order?) version of from the device?

Check the Calibre change logs for when (recent) the Library order tweak went into effect.
Not being a Sony user. My reader uses folders:

Old: The_Saga_of_Mess

New:Saga_of_Mess,_The

If I don't remove Old, I now have both

Sony's mileage may vary

[DoctorOhh]

Originally Posted by Otter

In Calibre's library view, the titles list correctly, The Baker's Boy appears under B but on my 505 for some reason it is now showing up under T.

Do you have the tweak in post 2 or 4 set to 'library_order'? If it is not set to 'library_order' then this may be your problem, set it to 'library_order' now and see what happens.

If it is 'library_order' then your problem is elsewhere. I would remove all of my books from the reader, disconnect my reader, reconnect my reader and resend the books back to the reader.