I feel for your situation. From a third party point of view, I dont think your hubby is honest with you. He is probably biding time. It seems that he is the unfaithful kind. Even if he breaks up with the current married woman, he seems to be the kind that fools around behind his wife back (judging from what you wrote).
Personally I think its better for you to be on your own raising you kid. I think its a decision you would need to make, whether its now or later.
Personally I wont cherish relationship with such a man.
If you wish to give him a second (or third fourth chance) I wish you well. But do be brave to face reality.