I am not possessive but I get jealous quite easily.
I definitely won't allow it.
I will feel betrayed and taken for granted if I found out later. I believe in "treat people the way you want to be treated" and I dont't think my husband will like it if I go swimming with another male colleague.
It's understandable if that guy goes with another guy friend but why would he rather go with a female colleague instead of asking the wife? -_-