He will be painful for sure no matter how long of his affair

Beacause he will worry about you may choose to leave and the kid custody

If this is your story, why don't you try to improve the intimacy and communication? If it doesn't work, I would suggest you go for a healthy relationship. It is no good for your lovely kid...

I do understand your pain and take good care !