Suing somebody is not in your best interest now. Keep in mind you goal is to sell the house with highest price (if you are the seller).
I think there is mis-understanding between you and your agent. If you tell him it is your bottomline, it is common he will intepret it as it is your counter offer.
Remember, YOU are the BOSS and YOU tell the agent what to do, not the other way around.
