No matter how perfect apt is, XFree is a black art.
Try running the script "xf86config" - it's text oriented - step by step - and never fails to make a workable X configuration.
If you've entered the world of X binary incompatibilies - my prayers are with you :) Currently I'm trying to modify the Cirrus server code - X is a huge, ugly mess - amazing it works so well most of the time.