You were all right, the 17 rule custom column was the biggest culprit. Without that, Calibre runs pretty smoothly (gotta test it more, though, with more categories shown in the Tag Browser etc.).
@chaley, thanks for your tips and explanations.
Unfortunately, my programming knowledge is pretty much non-existing. I do like challenges, though

.
I've tried to write a custom template function and it seems that it does work!

(And even scrolling is smooth.) Would you check if it's correct?
program:
a =field ('#target');
b = field ('languages');
test (a,
switch (a, 'English', 'united_kingdom_flag.png', 'Polish', 'poland.png', 'German', 'germany_flag.png', 'French', 'france_flag.png', 'Swedish', 'sweden.png', 'Dutch', 'netherlands_flag.png', 'Spanish', 'spain.png', 'generic_flag.png'),
switch (b, 'eng', 'united_kingdom_flag.png', 'pol', 'poland.png', 'deu', 'germany_flag.png', 'fra', 'france_flag.png', 'swe', 'sweden.png', 'nld', 'netherlands_flag.png', 'spa', 'spain.png', 'generic_flag.png'))
One thing I haven't figured out: there are (rare) cases where one book has more than one language (in 'languages' or in '#target'). What functions/structures should I use to display more than one flag then?