No big deal to be denied claim

Your insurance company will deny the other party's claim based on your story.
If the stories are different and the two insurance companies cannot agree, it is up to you and the other driver to settle this or to go to court. The key is witness. Passengers on your car can be witness if they are not your relatives. If large amount of money is involved, hire a lawyer and maybe put up a newspaper ad and look for witness.