Here are what you may consider doing

Told him verbaly and text him the same thing. And warn him that if he continue making advance, you will report this to HR. 

You do not have to compromise your life routine for his action. 

Do not do anything silly or stupid like mailing a bullet or things like that nature. 

And rest assured that he can not and will not get away with this if he continues. 

Good luck and remain calm!