No way it can KNOW what tags should or are not needed.
Tags are Per BOOK, not always the same across the series.
Book4 may have different Key players, location or a whole raft of things folk Tag

Select the book you think is correct, copy the Tags field, then close that. Then select all books in the series: Open the MDE (bulk mode) and paste into
Add Tags (calibre will prevent duplicates)