My boss is Indian but we got along quite well. Yeah, sometimes they are a little bit too relax and slow. He probably doesn't mean to offend you but just the way he gets used to it.
If you don't feel comfortable with what he did, just tell him upfront like what you did.
Sometimes, I am afraid I misunderstand people, so I would tell them what I feel and give them and myself a second chance being friends.
Just my two cents.