Barry,
How many system do you have on it?
What operating systems?
To answer your question, our current environment has 4 RH Enterprise Server 3? (what used to be RH 9.1) VM's, 27 W2k/W2k3 servers. The ESX "farm" is comprised of 4 servers (IBM Model 440's with 4 processors).
What happens when you have an W2K burb, ie: does MS laugh at you or provide support?
Not so much MS as the third party vendors that have apps that run on IIS/SQL. When we call for support they tend to say "oh, really!" a lot. If MS doesn't get behind us, IBM bends over backwards to push ESX. The biggest challenge we've faced putting many of these vboxen in place is the need for failover/clustering and MS is completely unreachable in these situations (but, to be fair, Windows clustering is an absolute joke on physical boxen as well...)
How many CPUs do you use?
SMP Option?
Disk IO performance, ie: can I run Oracle "fast enough" (as fast as a Sun 450?)?
Sort of touched on this early. We run 4 way IBM 440's so SMP is yes. Disk IO is somewhat subjective, in that it depends on what other client OS's and what they are running on them. I don't believe that it is fair to mention Sun equip because VMWare does not support Solaris. My gut reaction is that there is some overhead associated with ESX that would cause the standalone Sun to solidly trounce a VMWare implementation. As you know, all client OS's are vying for the same virtual arrays (over simplification, I know). That said, with most applications, the performance has been good enough. Hell, excellent for many applications. We currently have backend 9i and SQL servers and the apps on VMWare are front ends that point to dedicated db's. I hope I've answered your question. The problem is, there are so many variables in our architecture that its hard to generalize.
What I strongly suggest is that you bring VMWare in to replace standalone W2k/W2k3 boxen. It gives the added flexibility of .dsk (iso's) files with or without VMotion. This gives you many more options for viable disaster recovery plans (such as DVDs stored off-site, etc.) I'm not yet warm and fuzzy about using VMWare for "strong" production boxes. We are mostly using it for our pre-prod and development boxes and then moving the production services onto blades or other physical standalone machines. I have some serious Prod environments running in VM and have had good luck so far. YMMV.
Does Vmove REALLY work?
How much overhead does it take?
I assume you are referring VMotion? We don't currently have that installed. We have evalled parts of it... Looks good from that vantage point but I can't comment on overhead. We HAD to use the P to V tool (physical to virtual) to grab a web site (IIs/SQL) as any attempts to move it by normal means failed. The HAL on VM warrants a $30,000 tool to accomplish this. Again, YMMV.
Gotta run now.