[davidfor]

Originally Posted by DictatorAlly

I was thinking it could speed up requests. It doesn't sound fit for purpose when it doesn't return all the books though.

The search API doesn't return Amazon exclusive books. And there could be an issue if everyone didn't use their own key.

I would like to suggest some different feature additions to this GR plugin.

I am currently working on them, but any help would be appreciated.

I would like to add 4 options to to the parse_tags function.

The first option would be to add all found GR genre's to the filter list, this would be a user option: ON/OFF. They would be added with a key and value equal to the GR genre.

To the best of my knowledge, there isn't a GR genre list. My understanding is the the genres are actually just the shelves people have created. If you look at a book, it shows the number of people who have used the shelf. And that seems to be the same for the Genre page.

Then I would like the user to choose between 3 methods when adding tags. The current and default method: Exclusive, plus two new methods: No Filter and Inclusive.

The current method will only add tag's if they are found in the user filter list. We keep this method the same and call this option: Method to add found Tags: Exclusive.

The next option No Filter would add all GR genre's found as tag's without passing them through the user filter. This option is the easiest to code, and currently work's in the code below.

I am stuck on the coding for this option. The third option, Inclusive, would work the same as Exclusive (passing all GR genre's through the filter) at first but then it will also include all GR genre's found that are not on the user filter list.

Sorry, I had to read that a few times to understand what you meant. Which is lucky as I completely disagreed the first time through.

You are overcomplicating it. I would add two options:

- Above the list I would add "Map tags". Selecting this would enable the list and the second option. This would default to on for backwards compatibility.
- Under the list, "Add unmapped tags". If selected (and "Map tags" is selected), anything that wasn't mapped in the list would be added as well.

One problem not yet solved with these new feature's, how do we ignore a GR genre? Perhap's with some logic where we put the genre in the user filter with the value "IGNORED". Though this doesn't sound great, as it would break the Exclusive function, I think there would be a better way. Perhaps a new list for only ignored term's .

This is already handled. If the mapping has no tags, then the genre is dropped. But, I'll admit, I hadn't noticed this before. It was only looking at the code that I realised this.

This is what I have so far, I'm having trouble iterating over a list I am removing item's from:

Spoiler:

    def parse_tags(self, root):
        # Goodreads does not have "tags", but it does have Genres (wrapper around popular shelves)
        # We will use those as tags (with a bit of massaging)
        genres_node = root.xpath('//div[@class="stacked"]/div/div/div[contains(@class, "bigBoxContent")]/div/div[@class="left"]')
        #self.log.info("Parsing tags")
        if genres_node:
            #self.log.info("Found genres_node")
            genre_tags = list()
            for genre_node in genres_node:
                sub_genre_nodes = genre_node.xpath('a')
                genre_tags_list = [sgn.text_content().strip() for sgn in sub_genre_nodes]
                #self.log.info("Found genres_tags list:", genre_tags_list)
                if genre_tags_list:
                    genre_tags.append(' > '.join(genre_tags_list))
            no_filter_tags = genre_tags
            exclusive_tags = self._convert_genres_to_calibre_tags(genre_tags)
            inclusive_tags = exclusive_tags + self._remove_filtered_genre_tags(genre_tags)
            calibre_tags = inclusive_tags
            if len(calibre_tags) > 0:
                return calibre_tags

    def _remove_filtered_genre_tags(self, genre_tags):
        # for each tag, remove if we have a dictionary lookup
        calibre_tag_lookup = cfg.plugin_prefs[cfg.STORE_NAME][cfg.KEY_GENRE_MAPPINGS]
        calibre_tag_map = dict((k.lower(), k) for k in calibre_tag_lookup.keys())
        self.log.info("Keys - Found calibre_tag_map keys result:", calibre_tag_map)
        for genre_tag in reversed(genre_tags):
            tags = calibre_tag_map.get(genre_tag.lower(), None)
            self.log.info("Keys - tags result::", tags)
            if tags:
                for tag in tags:
                    if tag in tags_to_remove:
                        genre_tags.remove(tag)
                        self.log.info("Keys - Removing from genre_tags keys result:", tag)
        return list(tags_to_remove)

The following should work with my suggestions:

    def parse_tags(self, root):
        # Goodreads does not have "tags", but it does have Genres (wrapper around popular shelves)
        # We will use those as tags (with a bit of massaging)
        genres_node = root.xpath('//div[@class="stacked"]/div/div/div[contains(@class, "bigBoxContent")]/div/div[@class="left"]')
        #self.log.info("Parsing tags")
        if genres_node:
            #self.log.info("Found genres_node")
            genre_tags = list()
            if cfg.plugin_prefs[cfg.STORE_NAME][cfg.KEY_MAP_GENRES]:
            for genre_node in genres_node:
                sub_genre_nodes = genre_node.xpath('a')
                genre_tags_list = [sgn.text_content().strip() for sgn in sub_genre_nodes]
                #self.log.info("Found genres_tags list:", genre_tags_list)
                if genre_tags_list:
                    genre_tags.append(' > '.join(genre_tags_list))
            if cfg.plugin_prefs[cfg.STORE_NAME][cfg.KEY_MAP_GENRES]:
                calibre_tags = self._convert_genres_to_calibre_tags(genre_tags)
            else:
                calibre_tags = genre_tags
            if len(calibre_tags) > 0:
                return calibre_tags

    def _convert_genres_to_calibre_tags(self, genre_tags):
        # for each tag, add if we have a dictionary lookup
        calibre_tag_lookup = cfg.plugin_prefs[cfg.STORE_NAME][cfg.KEY_GENRE_MAPPINGS]
        add_unmapped_tags = cfg.plugin_prefs[cfg.STORE_NAME][cfg.KEY_ADD_UNMAPPED_GENRES]
        calibre_tag_map = dict((k.lower(),v) for (k,v) in calibre_tag_lookup.items())
        tags_to_add = set()
        for genre_tag in genre_tags:
            if genre_tag.lower() in calibre_tag_map:
                tags = calibre_tag_map.get(genre_tag.lower(), None)
            elif add_unmapped_tags:
                tags = genre_tag # Need to handle tags with > in them.
            if tags:
                tags_to_add.union(tags)
        return list(tags_to_add)

That should work, but. I haven't tested that. There is an issue with genre's that have multiple levels. They could be split, converted to dots for heiarchical tags, or just left like that. I have options for this in the Kobo metadata source plugin, but, the configuration is starting to get complicated if these are added. I'd probably just let them through as they are.

Honestly, I'm not sure if all this is needed. And if I as doing it now, I might just let them all through and rely on the Tag Mapper. This didn't exist at the time the plugin was written. It has the advantage of working all tags from all sources, not just one. The disadvantage of this is that it has to be run separately.