I do recommend driving the sea-to-sky highway (the road goes to Whistler) to at least the town of Squamish and if you have time to Whistler (even if you are not a skiier). You can pull out along the way and stop at Horse Shoe Bay for example. It is a beautiful drive(and difficult for some people).
I think you can spend almost a whole day in Stanley Park/Englis Bay. English Bay, close to Stanley Park, would be a good place to watch sunset. If I were you, I'd pack a to-go box and take walks and picnic in Stanley Park. I also like to watch the night view of Vancouver downtown from Stanley Park. I also spend some time around Canada Place (the landmark building with big sails).
You could spend a couple of hours in the Granville Island (if you like boutique shops, markets, etc. this place is neat)and it's not out of the way. Or Gastown is you like history.
Keep in mind in the summer months, sun doesn't go down till almost 9:30. So you have very long days.
Time permitting, you could also venture outside of Vancouver to Fraser Valley area going east or Sunshine Coast. I have always wanted to go island hopping in Gulf Islands. Think it would be fun.
Be sure to make a rental car reservation. If you don't, you'll be paying much higher rate at the counter.
For lodging, B&B is highly recommended (unless you have a big family). There are so many B&Bs very reasonably priced, beautifully decorated, and scenic enough to be a tourist attraction by itself. Nice full breakfasts are often included and some (if they haven't had enough business year to date) do not charge sales tax (over 10% saved right there). I have stayed at B&Bs in British Columbia, where the owner would let us use his canoe for free. We can sit in the living room, watching sunset on bay and snow-capped mountains or even sit in a jaccuzzi with heavenly views.