这里,说话者有性别歧视,属于Harassment。Harassment is unwelcome conduct that is based on race, color, religion, sex (including pregnancy), national origin, age (40 or older), disability or genetic information. http://www.eeoc.gov/laws/types/harassment.cfm
However, "Petty slights, annoyances, and isolated incidents (unless extremely serious) will not rise to the level of illegality. To be unlawful, the conduct must create a work environment that would be intimidating, hostile, or offensive to reasonable people."
具体什么是reasonable,大概就是楼主的问题。但是这没有一致的答案,一般来说,老板的看法最重要。