Here, in handy visual comic-strip format, is the very best explanation of the distinction between the officially Nordic and Scandinavian countries and their associated territories I have ever seen:
How the North Works
Everything else can be assumed to be non-Nordic/Scandinavian.
ETA: It does omit Svalbard and the Jan Mayen Islands, but those are unlikely to come up in everyday conversation in most places.