I do not think that dirty carpet is an acceptable reason for you to cancel the deal and get the deposit back because it constitute no structural damage to the house. Many people paying 510K to buy a house would buy new carpets they like. That's why the seller feel that you're wasting his time.
However, you may have a very small chance to win the case if you can proof that the carpet was made dirty after you looked at the house. It was clean when you looked at the house and made the offer.