hi, please calm down, maybe he dont really mean to cheat on that time out as some people just like to flirt around, you dont have to tolerate it but also dont have to over react.
i think it is normal to have dinner/lunch with opposite sex since we are already at 21st century, you love him till very deep right? if he cheated on you once will give you nightmare, you better divorce him because you will never be able to get over it.
he let you check him phone or facebook means he got nothing to hide from you, at least for now, if you die die want to find some fault ( msg at two years ago ) i think you will only make your life miserable.
you dont have to eat with you ex male colleague just for revenge, to be honest, he will not care because it is so normal to have a meal with opposite sex, unless you are going to do something else like hanky banky, but then he will have excuse because you are now unfaithful.
if he is yours, he will be yours, since you decided to live on with him, then you better not to look too detail on his personal life, it will only makes you feel more upset, take it or leave it, that's life, and life's never easy.