The problem is that on a lot of bases they disposed of stuff by putting it in barrels, and burying the barrels. The problem is that the barrels leak, and after leaking the contents got into the ground. From there it leaches into the water table.
Nobody has yet figured out an affordable way to clean up that kind of mess. (The most effective would be to remove the contaminated soil. Unfortunately per contaminated base you'd have to remove a few acres of soil to a depth of 15 feet or so - and that is a LOT of toxic waste to dispose of!)
Cheers,
Ben