Thoughts towards a solution, but I get lost quickly:
Spoiler:
When there's only two prisoners, and they know BOTH colours are present,
it is obvious to both of them that they have the colour they can't see, and both go.
When there's three prisoners, and they know both colours are present,
one of them sees only one colour, so he knows his is the other one, and goes. When he goes, the next day the other two go.
When there's four prisoners, and they know both colours are present, there are two possible cases. If they are divided in 1-3, again the one who sees only one colour leaves. If they are 2-2... Say you're blue, and you see one blue and two greens. You don't know your colour. You wait one day. If the blue one doesn't go away, you are not green. So tomorrow you can go away. Ok so far.
If there are five, 2 have one colour, 3 have the other. Say you are blue, you see one blue and 3 greens. You wait one day, no one leaves, which means you must be blue. Tomorrow you leave too.
If there are six, and say they are divided equally, because with groups of two it's simple. So you are green, and see two greens and three blues. If you are blue, you see three greens, 2 blues. Now what?
If I get stuck at six, how can I feel confident in a 100?