It is just an awful lot of questions. Seriously, you could write an entire manual on all the topics you attempt to ask. It was a very polite ask, but it would take hours of posts to cover them all.
I think you just need to get a couple of basic concepts right in your mind:
Custom columns are a way of allowing you to store or display additional information about a particular book. Such as a genre, whether you have read it etc. Whatever you can think of. There are lots of ways of skinning a cat, some people use tags to do the same thing in some cases. In other cases a custom column is the only way to go, such as if you wanted a column recording the date you last read the book.
I think your confusion with user categories comes because you see custom columns appearing in the tag browser on the left. Think of the tag browser as a giant point and click way of accessing your database. It lets you point and click to filter your database by an attribute of a book, such as its author or series. For convenience it also lets you point and click filter for certain custom column types too, that is why if you add a "Genre" custom column it appears in the tag viewer. So a custom column is *not* a user category, it just gets displayed in the tag browser next to genuine user categories.
Custom columns are most frequently used just to allow you to view/store additional information about a book. There are scenarios where people use them for other purposes to do with plugboards and sending books to their devices but ignore that for now, it is a niche usage.
For a genuine user category, you must use the Manage user categories button to create them. There is a
thread here discussing ideas for how people find it useful. Fundamentally a user category is just a way of building a group of books together as a clickable filter by some criteria be it authors, tags, series or publishers. In some ways it is very similar to a saved search, but just a lot more user friendly for some purposes. For instance a "Favourite Authors" user category containing 20 specific author names is easier to manage than a giant "or" based search.
However Calibre has such flexibility it can be difficult to decide which approach to take at times. For instance to manage favourite authors you could instead put a Favourite tag on each book, or a custom yes/no column called Favourite. However if you are thinking about that categorisation at a level other than book (this is a favourite *author*) then to avoid the ongoing maintenance every time you get a new book by that author you would be better off having a user category instead.
As for {author} vs {author_sort}. Say you store your authors displayed on screen as FN LN, like Tom Clancy. The author sort should have been set to "Clancy, Tom" so that when you sort on screen the books appear under C rather than under T. When you save books to disk or to a device, if you were to use the {author} field then "Tom Clancy" will get used, if you use {author_sort} then "Clancy, Tom" will get used. Similar with {title} and {title_sort} - Calibre has the ability to strip the "The" and "A" etc and stick them at the end for display or sorting purposes so depending on your settings you may want something different.
In my library I set the author to be LN, FN the same as the author sort, so it does not matter whether I use {author} or {author_sort}. Likewise I don't like the default title sorting/display of the trailing ", A" and ", The" etc, so after applying tweaks again it makes no difference whether I use {title} or {title_sort}. There are "tweaks" you have to set and hoops to jump through unfortunately, it is an area of Calibre that needs cleaning up with a wizard screen on startup imho.
So - whether to use a tag, a custom column, or a user category depends on whether you want to store book specific data or view across a higher level of abstraction.