Sony collection management changes in calibre 0.7.9

[chaley]

The extra time is between 0.5 and 1.5 seconds (job manager overhead), plus a variable amount based on the number of books on the reader. On my 300 with 180 books that variable amount is 0.4 seconds, so the total varies between 1 and 2 seconds.

[dmapr]

Chaley,

sounds like the option #3 is something I'd like to use. I too have a question about a couple of scenarios and what effect they'll have on the ordering of the books in the affected collection on the device (Sony reader).

1. I have books #1-3 & 5 of some collection in the library and the device; I purchase book #4 and insert it into collection in the library, then send that book alone to the device. Will the order be preserved or will I have to re-send the whole collection?
2. I have books #1-3 of some collection in the library and on the device. While reading book #2 I noticed some typos, fixed them and sent book #2 to the device. Same question about the order of the collection on the device — will it be preserved?

Thanks much!

[chaley]

Originally Posted by dmapr

I have books #1-3 & 5 of some collection in the library and the device; I purchase book #4 and insert it into collection in the library, then send that book alone to the device. Will the order be preserved or will I have to re-send the whole collection?

Order will be preserved in the sense that 4 will be between 3 and 5, regardless of which management option is chosen.

I have books #1-3 of some collection in the library and on the device. While reading book #2 I noticed some typos, fixed them and sent book #2 to the device. Same question about the order of the collection on the device — will it be preserved?

Yes, again regardless of which management option is chosen.

Answering a question you didn't ask: assume that the typos are in the collection name. If manual management is chosen, when you send the book, the book will be added to the now-correct collection and will also remain in the former (incorrect) collection, because collections are not deleted. If manage on send is chosen, then the book will be added to the now-correct collection and deleted from the former collection. If automatic management is chosen, the collection changes will be sent to the device at next connection with no requirement that you send the book. In all cases, collection order will be preserved.

[dmapr]

Thanks chaley!

[slantybard]

Thanks for answering my questions. Can't wait for the new edition. Have been holding out until some of the sony issues were worked out, but the automatic management is going to be awesome as I have been using tags of read/unread for managing my tbr pile on the sony. One question (that may already be answered elsewhere but I haven't searched for it yet) is whether you can use the calibre custom y/n read column to have a corresponding collection on the sony that would easily update automatically with the new management system.

[chaley]

Originally Posted by slantybard

One question (that may already be answered elsewhere but I haven't searched for it yet) is whether you can use the calibre custom y/n read column to have a corresponding collection on the sony that would easily update automatically with the new management system.

Can't yet use custom fields for collections or in save templates. Kovid has been waiting for 0.7 to stabilize before embarking on this journey. We seem to be arriving at that point, so I expect that custom fields + devices (and in OPF and in other places) will arrive sometime in August.

For future planning, what would you want to see on the device when using a yes/no field as a collection? One choice is to name the collection with the field name and populate it with books marked with 'yes', Another is to name the collection something like 'Read:yes' and 'Read:no', with each containing the appropriate list of books. My expectation is that there would not be a collection 'Read:undefined' made.

A similar question arises for text custom fields. Regarding multiple (tag-like) fields: should each 'tag' be a collection, or should the collection be named 'columnName:tag'? The same question applies to single value text fields. I can see arguments either way.

For example, if I have a field named Genre containing entries of Mystery, Thriller, and Science Fiction, should the collections be named Genre:Mystery, Genre:Thriller etc, or simply named Mystery, Thriller, etc. Using the first way permits the same word to appear in different field contexts and makes it clear which context is which, but the name is longer and sorts by the column name. The second will give shorter names and different sort, but opens the possibility that the same field value will be found in more than one context (such as in the custom field Genre and the standard field Tags), resulting in the two being combined into one collection. The second also opens the door to confusion because of lack of context. For example, if I have a 'Rights' field with values Free, PD, and PBook, I would need to remember my metadata structure to know what 'Free' meant. Seeing 'Rights:Free' would remind me.

[slantybard]

Agreed, there are almost too many ways of doing it. I am not sure what the best way of doing it would be. I am quite happy that collections form simply from my tags but have to use a read/unread tag. Not too difficult and manageable, but since columns are becoming another useful tool, I just thought that trading the read/unread tag to a column might be nice.

[timlegge]

Originally Posted by chaley

At the moment, this post applies only to Sony devices, because they have collections and the possibility of setting metadata outside of books. If you do not have a Sony, you do not need to read further. Note: other smart devices, such as iTunes, the Kobo, or readers on Android, may eventually use this option (or not).

HI

I am working on the Kobo support for collections now. I only really care about the devices I'm Reading list since that is pretty much the extent of its collections.

I have no problem getting that out of the database and setting displaying it. However I am trying to figure out how the collection might be managed. Is it done via tags? The current in demand feature for the Kobo is basically providing a method to remove the books from the I.m reading list without actually reading or page turning to the end.

I am assuming the automatic management collections (option 3 I believe) would allow users to manage the I'm Reading list from Calibre once I figure our what I need to add to the driver to process the change.

Tim

[chaley]

Originally Posted by timlegge

I have no problem getting that out of the database and setting displaying it. However I am trying to figure out how the collection might be managed. Is it done via tags? The current in demand feature for the Kobo is basically providing a method to remove the books from the I.m reading list without actually reading or page turning to the end.

This would best be done with a custom field, but you don't have that option yet.

Yes, I would use a tag. Furthermore, I would use the extended customization option to ask the user which tag to use. You would then set/reset the kobo's im_reading flag (a boolean in the database?) depending on the presence/absence of the flag. You do, of course, need to decide on its polarity: tag present -> im_reading == true, or im_reading == false.

There is some help for this in the standard book and booklist classes. In particular, the standard booklist class has a method get_collections that will check tags, etc, and return a dict {collection:[books]}.

I am assuming the automatic management collections (option 3 I believe) would allow users to manage the I'm Reading list from Calibre once I figure our what I need to add to the driver to process the change.

Any of the options would. If you use the 'standard' collections mechanism, then the user could modify it directly on the device view. The other two settings would modify it as described in previous posts. It is perhaps worth pointing out that if a kobo user sets the automatic option, calibre will call sync_booklists in your driver. You need to be prepared for that to happen and do the right things.

Regarding collections: one way to get more collections would be to use the author column in the DB. The idea is to make several rows per book. One would have the correct author. The others would put the collection into the author, perhaps prefixed with a special character so they would sort all together. All the rows would point at the same book file.

This is clearly a hack, but it might be a useful one. Problems: deleting one of the books from a collection would delete the only book file. Where the reader is in a book would be associated with the row used to open the book, as would any times/dates. Your driver would need to know to modify the collection entries if a book was resent and changed file paths. There are undoubtedly other problems. Notwithstanding, if I had a Kobo, I would want collections, even if I had to put up with this hack.

[Christina]

Hi, I have a question. I recently updated my version to 7.8, and then 7.9, from version 6.4-something. I had my books organised in collections on my eReader (Sony PRS-600) and PC. I haven't changed those tags or series data on either device.

But now I find that after plugging my Reader in, it has deleted all my collection data. I added a few new books, unplugged it and the menu read 'no collections.' I'm not sure why, how to prevent it happening again or how to fix it.

My preferences in 'add/save' were set to 'Manual Management' which is why I do not understand how this happened.

Thanks for your help.

[chaley]

Originally Posted by Christina

Hi, I have a question. I recently updated my version to 7.8, and then 7.9, from version 6.4-something. I had my books organised in collections on my eReader (Sony PRS-600) and PC. I haven't changed those tags or series data on either device.

But now I find that after plugging my Reader in, it has deleted all my collection data. I added a few new books, unplugged it and the menu read 'no collections.' I'm not sure why, how to prevent it happening again or how to fix it.

My preferences in 'add/save' were set to 'Manual Management' which is why I do not understand how this happened.

Here we go again. Sigh...

I have no idea why all your collections were deleted. Even more, I have no idea why none were created when you sent the new books. I am not doubting that it happened.

[Edit: the above sounds like I am annoyed with Christina. I am not. However, I am annoyed that she had the problem.]

First, lets determine where the problem was introduced. Did you plug in your reader while you using 0.7.8? If so, were the collections preserved? If so, that means that the problem is in 0.7.9, which I don't understand at all. If you hadn't plugged in your reader in 0.7.8, then the problem is in the upgrade from 0.6, and that I can understand.

Second, what are the settings in your device customization for collection building?

Third, what do you see in the device view? Are all the books marked as 'in library' that should be?

The best thing to do is to run calibre in debug mode and post the results. What OS are you using?

[Christina]

Originally Posted by chaley

Here we go again. Sigh...

I have no idea why all your collections were deleted. Even more, I have no idea why none were created when you sent the new books. I am not doubting that it happened.

First, lets determine where the problem was introduced. Dad you plug in your reader while you using 0.7.8? If so, were the collections preserved? If so, that means that the problem is in 0.7.9, which I don't understand at all. If you hadn't plugged in your reader in 0.7.8, then the problem is in the upgrade from 0.6, and that I can believe.

Second, what are the settings in your device customization for collection building?

Third, what do you see in the device view? Are all the books marked as 'in library' that should be?

The best thing to do is to run calibre in debug mode and post the results. What OS are you using?

I apologise for putting you out.

The problem occurred when I plugged my reader in using version 7.8, last night, which was when all the collections vanished on my reader (but not on my computer). I am using windows vista (for my sins) and most of the rest of my settings are default. I haven't played with any of the new features of version 7. I haven't had the chance, so I have changed nothing since upgrading from version 6.

In the device view, all the books are marked as 'in library' that should be. But the collection column is completely empty.

[chaley]

Originally Posted by Christina

I apologise for putting you out.

You didn't put me out. I am in fact happy that you reported the problem. I am unhappy that you *had* the problem.

The problem occurred when I plugged my reader in using version 7.8, last night, which was when all the collections vanished on my reader (but not on my computer). I am using windows vista (for my sins) and most of the rest of my settings are default. I haven't played with any of the new features of version 7. I haven't had the chance, so I have changed nothing since upgrading from version 6.

In the device view, all the books are marked as 'in library' that should be. But the collection column is completely empty.

Ahhhh, so the problem is in the upgrade from 0.6. I will look at that.

If your collections are built from calibre metadata (series and tags), then you should be able to get them all back quite easily. Start calibre, change the management option to Automatic management, then plug in your 600. Calibre should rebuild the collections from the metadata. Once all the jobs have finished, then eject your 600 and change the management option back (if you want to).

[Christina]

I've done that. Thank you. My collections are now back. I've reverted to Manual Management for now but I may use the automatic one. It seems to work very well and would save me time in the long run. I have thousands of books to add to my library yet. It's a slow process, because I'm picky about how they are organised (I used to be an Archivist ), but version 7 has made that process much more efficient. This was my only issue so far (fingers crossed).

I have no idea why it happened, but I'm sure you'll figure it out

Thanks again

[slantybard]

chaley, I love the new automatic management method. Upgraded from 6.54 without any problems and the collections are reflecting the tags perfectly. The only wish I have would be to have a toolbar button to activate the metadata syncing from the library so that you could update changes made in calibre after the reader is already connected to save out remounting the reader. Overall, not a big issue since I have waited a long time to get the metadata auto sync and it is awesome.