I've been polishing my own custom procmail spam filtering rules for the past couple of years. I've tossed them after less than a month using [link|http://spamassassin.taint.org/|Spamassassin].

My consistent results with ~6000 messages is ~99% true positive, 99.2%+ true negative, ~1.5% false negative, and ~1% false positive identification of spam. This is with a load heavily biased toward administrative mail (eg, systems admin), and with a few whitelist rules added to help push down the false positive rate. This is better in any dimension than my own rules.

As of two nights ago, I've added Ricochet back into the mix. Spamassassin flags spam scoring more than five points. I've set an autorespond threshold of ten points, missing a few items, but dropping the false positive score. IF YOU'RE GOING TO DO THIS, ADD *ALL* DOMAINS AND ADMIN CONTACTS ON YOUR MAILING LISTS TO THE .ricochet/skip-list FILE. This means getting the primary domains and adding their hosting providers, etc., to the skip lists. Note that I've still got procmail filters, but the main focus is on filtering lists and various other categories of mail to where they should go.

The result is that most spam I get automatically generates a report to the Razor distributed database, is reported to the spammer's ISPs, and is forwarded to uce@ftc.gov for possible fraud investigation (I hope their own filtering methods are substantial).

I know at least one web hosting provider that's looking at providing this as a service to its customers, as soon as the harried admin can tie spamassassin in to qmail ;-)

I could wing it, but [link|http://mathworld.wolfram.com/StatisticalTest.html|Wolfram's already wung it good].

Measurement of type I and type II error are independent. You're measuring the percentages within two independent populations: spam and non-spam. The overall error would be the sum of the type I and type II errors, over the sum of all mail received.

Any way you add it up, SA rocks hard.