Research suggests that handfree headsets are as bad as traditional cell phones. The distraction is the issue.
In particular distraction that pays no attention to what is happening in the car. Humans in the car naturally modulate their conversation in deference to driving conditions. Humans on the phone, don't.
Cheers,
Ben