Evaluation won't hurt

From what you described (can't concentrate during story time, can't follow directions, etc), i think you might want to get your son evaluated and see if he needs speech therapy. Usually you can do the evaluation at local school district (even you don't go to public school).