We actually don't go on that many true vacations, though last year we did manage to see New York and Boston, and we do go to Chicago with some frequency. For me, however, family vacations are really about packing up the car and driving. We don't do that for a number of reasons, including we have to rent a car, since we don't own one.
The trip to Hamilton yesterday went fairly well, and I think in mid-October we'll try to drive out to the McMichael Collection, which we did two years ago. I'd still like to get to Windsor and Detroit, though I haven't decided if it is worth trying to do that by train. In 2017, we might look into Ottawa or Montreal by train, though I want to make sure to combine it with something worth seeing, such as the upcoming Chagall exhibit in Montreal. There is an exhibit on Sudjek and his special brand of photography at the National Gallery, though I am not entirely sure I would venture up to Ottawa just for that.
While this is all extremely tentative, I am thinking that over spring break in 2017, we may go to D.C. and North Carolina (or D.C. and Philadelphia, which I would prefer, mostly because the travel there and back would be considerably easier). Maybe next summer it is worth investigating renting a cabin for a weekend or even a cottage for several days (a week might be too much...).
I am thinking perhaps the following summer it might be worth trying to get to Calgary and going to Drumheller (while the kids are still somewhat interested in dinosaurs), seeing a bit of Calgary and then driving into the Rockies to see Banff and ideally go up to the glacier park north of Banff.
Beyond that is very unclear. My daughter really, really wants to see England and there are certainly a number of places in Europe I would love to visit again, but there is no point in trying to map out something quite that far in advance.