It is good for every one that the divorced people should have no communication and no connection if the divorced couple doesn’t have any kids or very special issues to deal with. It is so simple and so true, especially for your case. To me, your hu*****and is native, foolish or may be stupid unless he has different ideas to his previous marriage and his marriage with you, which he has not shared with you. Simply go ahead and tell him “ stop it”. His behavior is no excuse and not acceptable. It has nothing to do with his former wife.