Sony collections and release 0.7.5

[chaley]

Calibre 0.7.5 (to be released soon) fixes a bug that affects people who manage Sony collections themselves. The bug manifested itself if books on the device changed without calibre knowing about it, either through using Sony software or manually adding/changing books using a file manager.

In addition, and the real subject of this post, 0.7.5 introduces a new preference that helps calibre support two kinds of users of collections.

The first kind are those who wish to manage collections themselves. This kind of user will manually add and remove books from collections. Under this scenario, calibre will never remove a book from a collection, but may add books to collections if the user asks for it by putting/leaving values in the collections box in device customization.

The second kind are those who wish to have calibre manage collections using metadata. In this case calibre will add and remove books from collections automatically when books are sent, ensuring that the collections correspond to the book's metadata at the point of sending. Manual editing of collections is not permitted, because the changes will be removed automatically.

A new FAQ section describes the option and its effect:
------------------------------
How does Calibre manage collections on my SONY reader?

When Calibre connects with the device, it retrieves all collections for the books on the device. The collections of which books are members are shown on the device view.

When you send a book to the device, Calibre will add the book to collections based on the metadata for that book. By default, collections are created from tags and series. You can control what metadata is used by going to Preferences->Plugins->Device Interface plugins and customizing the SONY device interface plugin. If you remove all values, Calibre will not add the book to any collection.

Collection management is largely controlled by 'Preserve device collections' found at Preferences->Add/Save->Sending to device. If checked (the default), managing collections is left to the user; Calibre will not delete already existing collections for a book on your device when you resend the book to the device, but Calibre will add the book to collections if necessary. To ensure that the collections for a book are based only on current Calibre metadata, first delete the books from the device, then resend the books. You can edit collections directly on the device view by double-clicking or right-clicking in the collections column.

If 'Preserve device collections' is not checked, then Calibre will manage collections. Collections will be built using Calibre metadata exclusively. Sending a book to the device will correct the collections for that book so its collections exactly match the book's metadata. Collections are added and deleted as necessary. Editing collections on the device pane is not permitted, because collections not in the metadata will be removed automatically.

In summary, check 'Preserve device collections' if you want to manage collections yourself. Collections for a book will never be removed by Calibre, but can be removed by you by editing on the device view. Uncheck 'Preserve device collections' if you want Calibre to manage the collections, adding books to and removing books from collections as needed.

[chaley]

0.7.5 is released. Please use this thread to for comments on calibre's handling of Sony collections.

I will pose one question to start things rolling. The collections editor operates on all books on the part of the device you are looking at (memory, card). Should it instead operate on a selection?

[June]

I'll install 0.7.5 and test it later today (after F1 qualifying)

[nicholbb]

How will this cope with two copies of Calibre? I use one at work and one at home. Also would wonder about multiple devices (not lucky enough to have two, or unlucky enough to have to buy one for GF).

I also think I'm missing the advantage of this change, the tags on the Sony are good enough to find type of book I want to read next (only slightly annoying putting book number in title).

[chaley]

Originally Posted by nicholbb

How will this cope with two copies of Calibre? I use one at work and one at home. Also would wonder about multiple devices (not lucky enough to have two, or unlucky enough to have to buy one for GF).

The answer depends on how the option is set.

If set (the default), calibre behaves in the fashion it always has (as I understand it). Assuming that you permit calibre to build collections from metadata, collections will be added for the book's metadata at the time the book is sent. Calibre also notes the book's unique ID and stores this in the cache file on the device.

If the book is subsequently changed on the device (the book, not the collections), calibre will add collections for the metadata stored in the book, if any (and if permitted). If the other installation of calibre has the same book but with a different unique ID, calibre will add collections for the metadata stored for that book. As such, if you have two calibre installations containing the same book but with different metadata, you will find that the collections become the union of the metadata on the two installations.

If the option is not set, then the tags will reflect those of the installation of calibre that sent the book. If something other than calibre changes the book, then the collections will be built from metadata stored in the book. If the other calibre resends the book or has a different unique ID, then the collections will be rebuilt to match the metadata on that calibre installation.

You need to decide which of these behaviors you want. If could easily be that you want neither, but instead want total manual management: to turn off all calibre's collection building and to manage collections yourself. To achieve this, leave the option checked and empty the collections fields box in device customization.

Multiple devices should not be an issue. Calibre works with one at a time, and does not store any information in the library about what is on or not on a device. When you plug in a device, calibre will read the cache to determine what is on the device. At that point, collections will behave as described above.

I also think I'm missing the advantage of this change, the tags on the Sony are good enough to find type of book I want to read next (only slightly annoying putting book number in title).

I don't understand the point.

First, which change? The behavior of calibre with the option checked is (intended to be) the same as it behaved in 0.6, with the exception of unique ID matching triggering adding collections from current permitted metadata. You get the new behavior if you uncheck the option.

Are "tags" the same as collections? Doesn't seem so, as you mention titles. But if yes, then the collections must get to the Sony somehow. If 'tags' are instead the sony's display by author and display by title functions, and if these are sufficient for you, then don't use collections. Leave the option checked and remove all items from the collections box in device customization, then calibre won't build collections at all.

I confess that I added the option because I don't like calibre's original (0.6 for me) behavior. I want calibre to "make my collections right". I never installed the Sony SW and never use anything but calibre to put books on the device, and I don't want to deal with cleaning away old collections for books that I resend after changing the metadata. If you don't want this behavior, don't uncheck the option.

[Sorraia3]

Originally Posted by chaley

0.7.5 is released. Please use this thread to for comments on calibre's handling of Sony collections.

I will pose one question to start things rolling. The collections editor operates on all books on the part of the device you are looking at (memory, card). Should it instead operate on a selection?

Great idea! That would be wonderful! Most convenient would be a check box, available when entering into the collections manager window that asks to apply change to all books(checked) or only selected books(unchecked).

That would make a very easy way to change items that are tagged in more than one category but don't need to be in more than one collection while others maybe do.

[JLYates]

Nevermind. My son just got up and said "Mom" shook his head and walked away.


Janette

[June]

Success! I installed 0.7.5, checked that I have teh new box ticked and 'tags' set in sony device plugin -imported converted and tagged a pdf book. Hooked up my 505, sent the book over - and my book was sent to the 505 *and* all the 258 books on there (which lost their collections earlier) are now back in their collections

I *know* the recommendation is to delete and resend teh books, but since I'm going to keep running Calibre in debug mode, I decided not to do that, and instead see what - if anything - turns up later

Thank you for the quick fix to the collectons&tags issue

[chaley]

I am glad that it worked for you. And thanks to you for your help.

[chaley]

I have a question for those of you who edit collections (the 'Preserve device collections' option is checked). As described in previous posts, if there is something in the 'build collections from metadata' box in device customization, calibre will add the necessary collections. If calibre notices that these collections are gone, it will re-add them, thinking that is what it is supposed to do.

One consequence of this implementation (#1) is that you will not have joy if you manually delete a collection that came from metadata. Calibre will put it back at some point in the future. Said another way, you can successfully add and delete collections that are not derived from metadata, because calibre takes no ownership of these. You won't be able to make deletion of collections stick if they are derived from metadata.

An alternate implementation (#2), possible when the preserve device collections option is sent, would be to build collections from metadata *once*, when the book is sent. From that point on, calibre would disclaim all responsibility for them. If the collections are somehow changed, the only way to correct them would be to resend the book. [edit] However, collections would only be added. To delete collections, you would need to first delete the book off the device, then resend it. Or of course, delete them manually. [end edit]

This question arises because of one user's frustration, eloquently described in a ticket. The user very much wants behavior #2. However, I hesitate to change calibre in this way without posing the question to the community.

Which of behavior #1 (the current) or #2 (the alternate) would you prefer?

[banjomike]

I don't see how you can have an option which allows users to maintain their own collections and still have Calibre put collections back when it notices thay have gone. #2 seems to be the closest to what you described in post 1.

Having said that, I will probably stick to letting Calibre maintain my collections.

[DoctorOhh]

Originally Posted by chaley

An alternate implementation (#2), possible when the preserve device collections option is sent, would be to build collections from metadata *once*, when the book is sent. From that point on, calibre would disclaim all responsibility for them. If the collections are somehow changed, the only way to correct them would be to resend the book.

I let calibre handle my collections, but until 0.7.x number 2 was the way I always dealt with my books, which is why the user is probably requesting this.

Seems reasonable to me, but I've unchecked Preserve device collections.

[chaley]

Thank you both for your comments.

Originally Posted by banjomike

I don't see how you can have an option which allows users to maintain their own collections and still have Calibre put collections back when it notices thay have gone. #2 seems to be the closest to what you described in post 1.

The thought was that by putting something in the collections box, the user was asking calibre to ensure that the collections honored those metadata attributes. Seemed right at the time.

Having said that, I will probably stick to letting Calibre maintain my collections.

and

Originally Posted by dwanthny

Seems reasonable to me, but I've unchecked Preserve device collections.

It is probably worth clarifying the effect of the proposed change if the option is cleared. The safest implementation (and I want to be safe at this point) will be for calibre to process collections at the same point (sending a book) regardless of how the option is set, but with different effect. If the option is cleared, when a book is sent, calibre will add any collections necessary, *and* (this is the effect of clearing the option) delete any collections no longer indicated by the metadata for that book. Collections will not be rebuilt at any other time.

[nicholbb]

@chaley
Thanks for your detailed reply.

Your explanation of the two differences finally made it past my thick skull. I'll have my (main) home Calibre maintaining and the work one (with just odd bits and some news) just viewing.

With my second point, no not clear was it. I didn't fully understand about the auto updating of the collection information (despite your use of plain English). My point about book name/series is completely out of context and I'm not sure why I wrote it.

Thanks again.

[June]

Personally I'd prefer alternaive #2 - I want the ease of creating collections from tags, but would also like to be able to manually tweak my collections on the device. That's how I've been using Calibre 0.6.x. And I don't like the program "thinking on my behalf".

But I understand that #1 is 'the thing of today' what with iTunes and all, people are getting used to programs sorting stuff for them