I also supplement with formula. When I went to my OB for the six "WEEK" check up, I mentioned that I was going back to work soon. She immediately said, without my prompting, that if I want to wean the baby, that is OK. There is diminishing return on breastfeeding, and most benefits have already been provided by week number 6. The marginal benefit of breast milk at that stage may not be worth the stress of pumping at work and associated negative impact on the body. I was shocked to hear that. I did go on pumping until 6 months.
I also have a friend who is an OB. She weaned her kids at four months, and gave me the same speech about "diminishing return on investment".
I think it is up to you. If it does not give you too much incovenience and stress, then breast milk is always better than formula (I think, it just feels that way). But if it is too much of burden or pain for you, then, according to the two OBs I have talked to, may not be worth the stress.