Flying into Vancouver Airport and then renting a car is one way of getting here, but you will save big bucks if you take the Greyline bus shuttle from the airport to the TsawwassenFerry Terminal and then travel on the ferry as a foot passenger. The foot passenger fare is less than $20 per person . Even with paying the bus fare, you will still save as you won't need to rent a car until later in your trip, and you won't be paying for your rental car on the ferry either!
The bus line will check your baggage and have it waiting for you at the Swartz Bay Ferry Terminal if you state this as your preference. From there we are are safe, 10 minute downhill walk or a one minute cab ride.
From our B&B there are a couple of really nice eating places within walking distance. You can get to Butchart Gardens on the bus, and save your car rental days for the time when you really want to explore. There is also an express bus to downtown Victoria, where parking is difficult to find and expensive.
If you do rent a car during your stay, you can return it and travel back to Vancouver Airport using the Greyline service on the ferry once again - they generally have buses on every odd hour crossing ie: 7am, 9am 11am etc. See their site for full schedules.