I'm not recommending you read it for you; I'm recommending you read it for your son.
[link|http://robertmoore-phd.com/0062506064.cfm|King, Warrior, Magician, Lover]
Your son is going to have serious abandonment issues, and without a strong male archetype around for himself to model after, he is going to need some serious help. I went through a lot of the same growing up, albeit for somewhat different reasons, and it took me a while to really recover from it. In some ways, I'm still recovering from it, and I'm in my early thirties. The above book was key in helping me to understand the process of maturation and passing beyond being a boy into manhood.
(Something I think your ex doesn't seem to have done yet, IMO.)