But it does not hurt to reschedule and meet him again.
If you are always so stressed out with him, it may not be a very good choice. You should be comfortable with the Mr. Right.