It emphasizes that you cannot expect to accomplish great things in a short period of time.
More per dear Wikipedia:
"Rome wasn't built in a day" is an adage attesting to the need for time to create great things. It is the usual English translation of a medieval French phrase, Rome ne fu[t] pas faite toute en un jour, from the collection Li Proverbe au Vilain, published around 1190. The modern French form is «Rome ne s'est pas faite en un jour».
The expression, (as "Rome was not built in one day") is given in English in John Heywood's A Dialogue Conteinyng the Nomber in Effect of all the Prouerbes in the Englishe Tongue (c. 1538), while Queen Elizabeth I referred to the idea in Latin in an address at Cambridge in 1563. The present perfect and oratio recta version of the Latin saying—the version one would use for a stand-alone quotation—would be Roma uno die non est condita.
Here comes the important part: next time when others rush you, you know what to say. 
