回复：我该赔 这笔钱 吗?

He might be just threating you to get the money.
Hiring a lawyer or even go to the small claim court would cost him more than $400. I personally do not think he will really sue you.
But what you did was wrong and you, not the friend, should be resposible for the damage.

If you leave the country and the guy sues, you will not be able to respond and that leads to outstanding arrest warrant from the court. As long as it is valid (would be for many many years), you may never be able to enter US again.
Good Luck, dude.