Long time no see, welcome back!
Thanks for sharing the experience.
If I were you, I would insist scanning the VIP card. If he wasn't willing to do that, I would call for manager filing complaint. People may think you are too laud over the trivials. But as long as you don't feel comfortable, it is perfectly fine just being assertive. (easier to say than do ":))
There are always some jerks outside there, just forget about them.