Do you use bluetooth headset?
I now kinda suspect it's related to bluetooth headset. I tried 3 different bluetooth headsets, all have random freeze issue with Nexus One (when answering a call). I never tried without bluetooth though, maybe it's OK without it?