If you switch to a view that only sorts by one column (series column), what happens is that it pushes that column to the top of the sort order, while the first few columns of the previous view (depending on the value of maximum_resort_levels tweak) are still active when the books have the same value for the first column (series column).
Without seeing the data it is harder to know what is happening. But you can see for your self the current sort order, and examine whether the books follow that same order by using the following action in Action Chains (start in debug mode and it will print sort order to the console):
Code:
from calibre.utils.config import tweaks
from calibre_plugins.action_chains.actions.base import ChainAction
class SortingInfo(ChainAction):
name = 'Sorting Info'
def run(self, gui, settings, chain):
db = gui.current_db
sort_history = gui.library_view.get_state().get('sort_history', [])
max_sort_levels = tweaks['maximum_resort_levels']
print('DEBUG: sort_order: {}'.format(sort_history[:max_sort_levels]))