It is dangerous if you continue this feeling.

I hope you won’t go to talk to him to ask him what he really thinks. You should be aware your company’s policy about harassment, which including not just physical touching, also may including language, and other behaviors.