Even if you don't remove contingency, since seller don't want the perception of "something wrong with the house so buyer backed off", seller will work will buyer and even give small concessions. Eg, if you find something in inspection, seller may be willing to lower the price a bit or do the repair at their cost.