[dailyalice]

Originally Posted by Amalthia

I'm now curious to know how other people use calibre to normalize their metadata, only I'm not sure this thread is the place to find out?

Giving up entirely on AO3's freeform tags is a reasonable thing to do (you don't even have to download them).

I stopped working on my guide to FanFicFare and calibre for a few months and I suspect some of that was because I started a section on normalizing metadata and got a bit overwhelmed. There's a ton you can do! But is it worth it? Key is figuring out which data you find most useful and would find useful in your library.

As far as AO3's tags go, I have two columns in my fic library that are filled using freeformtags.

The first column includes all of the messy AO3 tags: I put them in a separate column and don't touch them. So they're there if I want to search them or use them, but I can also easily ignore them.

I have this under add_to_custom_columns_settings in the AO3 section of my personal.ini:

 freeformtags=>#freeformtags

And I have a custom column in calibre of the tag type with a lookup name of "freeformtags" and column heading of "Tags (AO3)"

But I also use the freeformtags somewhere else. In the [defaults] section of my personal.ini, I have:

custom_columns_settings:
 tags=>#tags

include_in_tags:freeformtags

tags_label:Tags (Edited)

And I've created yet another custom tags column in calibre for that data (tags type, lookup name of tags, column heading of "Tags (Edited)" though I'm not wild about that name and may change it.

And then I do some tag normalization under replace_metadata using the metakey of "tags" to clean up some of those freeformtags.

Thrilling stuff like:

 tags=>^[Ff]luff [Aa]nd [Aa]ngst$=>Fluff\,Angst
 tags=>^[Aa]ngst [Aa]nd [Hh]umo(u)?r$=>Humor\,Angst
 tags=>^[Aa]ngst [Aa]nd [Ff]luff$=>Fluff\,Angst
 tags=>^[Ff]langst$=>Fluff\,Angst

I also do some stuff with tag hierarchies so tags that are specific to certain fandoms wind up being in tag hierarchies. MCU.Post-Endgame, MCU.Shrunkyclunks, and stuff like that. A person can easily go overboard.

But the real key to the whole enterprise is that then I use include_metadata_post to indicate the only tags I want to see in that Tags (Edited) column. They can be some that I normalized a bit using replace_metadata and some of the more useful ones from AO3 that I keep as is.

include_metadata_post:
 tags==Action/Adventure
 tags==Angst
 tags==Crack
 tags==Drama
 tags==Established Relationship
 tags==Fluff
 tags==Humor

You can choose to keep whichever tags from the AO3 freeformtags that you find most useful. If you already have a library, you can use the tag browser to see which tags are used most often; there are also lists online of the most popular tags on AO3. They can be a good starting point, though most folks know which tags they find most helpful.

In one of my bigger calibre fic libraries, I have over 30,000 tags in my Tags (AO3) column (GOOD GRIEF) and about 400 in my Tags (Edited) column.

I've kept these all separate from calibre's main tags column/field which I use for other things.

(Step by step instructions, with screenshots & code you can copy, will be in my guide if I ever get it done and figure out how/where to post it.) (The Google doc was well over 500 pages when last I checked and then there are separate docs on other topics like normalizing metadata.)