[chaley]

Originally Posted by davidfor

Firstly, looking at the code for the collections, I remember how much I hated it. And, honestly, I didn't understand enough about calibre at the time to do it properly. Plus, it was pretty much my first time coding Python. I think that's why I dropped the SONY options as it was doing my head in at the time.

By way of some explanation for how the collection code is as it is:

I started using calibre when I bought my first Sony PRS 350 in 2010 (IIRC). I wanted better collections support, specifically collections from calibre fields. I opened the idea with Kovid and he said "Go ahead. No promise I will take it though." Given that it was my first foray into python and calibre code, his position was wise. It is also worth noting that this effort precedes my custom column work by a couple of years.

So, as is usual in these cases I built what I wanted: automatic collections management. By this time Kovid and I decided we worked well enough together and he gave me (more-or-less) free rein. I released it, and oh my did the excrement hit the rotating blades.

There were four general responses:

1. Take it out! Don't you dare touch my collections. I know where you live!
2. Add collections when I send a book but otherwise never change or delete them. I also know where you live!
3. I can see the use. Update (add & remove) the collections for a book when I send it to the device, but otherwise don't touch them.
4. I like it. Do everything manually so I never need to touch the collections.

The above explains where I ended up, adding "Metadata management". You can see the correspondence between the management choices and the requests. Adding the management choices made the code much more complex, and my experience with python didn't help. I had to differentiate between collections the user made and collections the driver made and I had to implement the semantics of the management choice. This shows up in the collections code as the '_new_book' and 'device_collections' stuff.

The Sony shows collection entries in the order they were added, which by the way I really wish I could do on a Kobo. This permitted me to control the display order of collections, in particular the series collection. It also explains why the collection code passes around sort values.

Around this time (2012?) we started working on custom columns, which expanded the universe of collections possible to create. Some issues:

• The same value could appear in different columns, with different meaning. This is why the prefix/suffix stuff tweak was added.
• The sorting "problem" grew bigger. Some data was best presented using a different sort. For example, The user could sort a collection by date or some other value. Finally, some columns were best combined into a single collection, for example multiple series columns. I added a tweak to control how a collection was sorted. By default series-style collections were sorted by the associated series index, and other collections were sorted by title_sort.

All this explains some of the complexity in the collections code.

But, looking at it, and the version of "get_collections" in CollectionsBookList, I do not think that it would be that hard to put it back in.

But, I think I'd prefer to add template support.

I agree. There is no reason to re-add the sony tweaks. There are better ways to do it.

Can you post the template you are using? Your code isn't what I was thinking of. I think your intention is to use both the collection column list and a template. And the template is applied to the values from the collection columns. When I thought of this, my intention was use the template to return a list of collection names. And they would be used without any further changes.

The difficulty here is metadata management. If you support all three choices, and it appears that you do, then you must still distinguish between 'device_collections' and collections generated from metadata.

The technique I used avoids this problem by leaving all the metadata management and device collections stuff as it is. Instead, when collection names are being generated from columns I can remove or rename them.

The template below shows what I mean. I have (actually, will have) two libra 2 readers, mine and my wife's. For mine I want the following collections:

• Authors by first letter of last name. My brain doesn't deal well with lists in FN LN order, sorted by LN. It also doesn't deal well with lists sorted by FN LN. This collection solves the problem by giving me an alternate way to get closer to an author using the first letter of the last name.
• My curated tags.

My wife just wants her curated tags.

The "Collection columns" box contains all the columns we both want to use: "#dthtags, authors, #cbhtags". The template cleans things up using the device serial number to select what to do. For columns not wanted on the device it returns '', removing that collection value. It does this by checking the lookup_name and the serial number.

• For my wife's collections it passes through only values for the #dthtags columns, without a prefix.
• For my collections it:
  • adds a prefix to the collection name. For authors I use the heading defined in calibre. For my tags I use "Tags"
  • for authors it renames the entry to be the first letter of the last name.
  • It removes values for the column #dthtags.

NB: I haven't yet received our second Libra 2 so the existing serial number is used twice. I test it by changing the value in the first 'if'.

program:
	globals(val='Last, First', lookup_name='authors', serial_number='N4181B5019336', 
		    heading='Authors');
	if serial_number == 'N4181B5019336' then
		if lookup_name == 'authors' then return strcat(heading, ': ', substr(val, 0, 1)) fi;
		if lookup_name == '#cbhtags' then return strcat('Tags: ', val) fi;
		return ''
	fi;
	if serial_number == 'N4181B5019336' then
		if lookup_name == '#dthtags' then return val fi
	fi;
	return ''

Which explains why you were adding a field in the config. After yesterday's posts, I was thinking of adding a "Use template" checkbox. When selected the current collection columns field would be replaces by the TemplateConfig widget that is in the configuration code. That is the field plus template editor button and validation. That would be safer than parsing for whether it was a template or not.

I certainly have no objection to having a template that produces the list of collection names. I suspect this won't be easy if you continue to respect metadata management.

EDIT: assuming the template returns a list you will probably need to use a funky list separator like ':::'. There is no reason to believe that values don't contain commas or other simple separators.

I'll add the serial number. The driver is already reading the version file, so it should be getting the serial number.

Getting the serial number in the driver was easy to do. Currently I get the value to put in the template by looking at the version file. It would be nice to have a way to get it from the utilities plugin.

I didn't mention it yesterday, but, someone asked something similar to this recently. For that, I suggested using the template function "connected_device_uuid". That can be gotten from the driveinfo.calibre file and is how both the Kobo Utilities and Reading List plugins identify a device. Passing that and possibly the firmware version in might be useful.

Yes, it could be. That said, I have been mildly unhappy with the UUID system for a long time because the UUID changes if driveinfo.calibre is deleted. For the Kobo the serial number will never change.