How to deal with this neighbor

I have 2 boys: 5 and 2 years old. They sometimes played at my backyard. My neighbor who says he himself is a writer, keeps complainning that he can not tolerate their voice, he think they are noisy. And he kept asking me to bring my boys back to house instead of playing at my backyard. In fact, my boys' voice is just reasonable. I am so frustrated that I can not let my sons play at my own backyard. Is there any laws to protect my right? Or, what is the best way to deal with him? We tried to come up a solution for him, such as buying a headset for him, since my sons only plays at some evening time or weekend, all the other times they are in preschool. But no solution with this neighbor, he insists that my boys should not play outside. I appreciate your advice and help!