Whoever say good things about U, could be used as a reference.
Specially for those who have left the current company, are very good candidate to use as your reference.
If the guy do not say good things about us, then don't use that guy as reference.
Thanks!