How old are your tyres and how much tread left?
When tyres get older and warn, they tend to square off and can make the car tram-line quite badly and feel unstable.
May be worth swapping front for back to see whether it makes a difference to test this theory?
I had a geo on my 911 with Centre Gravity, one of the best in the business, but still had really bad tram-lining/unstability on poor roads.
I changed all 4 tyres, and hey presto, it handles like it is on rails.
Also had a problem with a Mini Cooper S that I owned, changed the front tyres and it again transformed the car.
Tyres can make a massive difference, even when you think that they are fine because there is plenty of tread left.