The Only thing wrong here was that he selected a wrong referenc

His boss is definitely not a good person in my opinion.Words from this kind of persons are just useless for me.If you really care about his work ethics,you should ask more references from him and make your own comments.Then I think you treat him fairly.

If you decided not to bring him in,you should give him a hint to select another reference.It will be very helpful for his next interview.