Labels? Do you mean Column titles?
These are per Physical Library (db file) and not per Virtual Library.
Each Library stores the column order, formatting and visibility.
By any chance are you using a special plugin like 'View Manager"? Ones like that takes control