Have you verified what you're references SHOULD be saying about you? Did you prep them to handle the types of trick questions some hiring managers throw out? Double check with these people to be sure whatever they say can only be complimentary about you and your work ethic.
After all that, then someone at your ex-employer MAY be saying things that you hope they wouldn't. Best way to find out is to hire a reference checking service, have them contact your ex-employer, saying that they're looking at you as a potential employee and want to verify background, etc. Let them try to pull answers to questions that your former employer should be smart enough to avoid. Make sure they get a transcript or a recording of the call for confirmation, and then contact the legal profession to get someone on your side. There's gotta be someone hungry enough in town to go to bat for you.