If you cannot get the loan and you have a loan contigency, then you can withdraw without penalty.
But please be careful to follow the contract. Usually you have 10 days or so to clear the loan contigency. If you do not do anything within this time period, then you may lose your EMD.
It is common for buyers to withdraw offers. If it is not by the last minute, people usually understand. Not much negative to you and your agent.
But if you withdraw by the last minute, both agents and the seller would be very mad at you.