Don't pay too much attention to it

Don't know your situation, so cannot give you any more suggestions. Just one thing, don't be afraid of others' reaction. Say what you think is right. After the meeting, you can still say hi to him, pretending nothing has happened... It's better to make him unhappy then being made unhappy by him...