His family didn't have enough money to pay for the trip to H and for the balance of school cost so he stayed to attend a small local college (thus he could also help the family chores while attending a local school).
Guess a great man will be a great man no matter what school he attends (but then, Duke Law is a top school anyways) 