Bust out the chalk and mark where the tires could rub the fenders, Get some friends in the car and see if it rubs. If there's no rubbing then start checking the bushings.

If it's more noticable with more weight I'm inclined to think that it's fender rub. I had the same issue with another car I had.