depends.

For people who separate sex and love or/and who like open marriage, even having sex doesn't mean unfaithful. You need to have a deep and open discussion/ soul searching with your H about love, sex, and marriage.

Have you watched "Californication"? it's a bit too slutty, but raised interesting questions about the nature of human relationship in these aspects, as well as addiction, free will, etc....