Especially since 9/11, most companies are very rapidly becoming very closed about their transactions, what's in them, and what they do.

Though 9/11 has had an effect, the main impetus for companies guarding their data and processes more closely is just simply due to the business cycle. When times are lean, companies tend to become much more defensive, trying to hold onto whatever advantage they can - real or perceived.

In my industry (pharmacy claims), we're about to implement another proprietary protocol, completely ignoring XML. Medicare/Medicaid are about to standardize on some ANSI protocols for HIPAA that have nothing to do with XML.

I work on the front and back end sides of medical benefits (setting up eligibility and doing the expensing on our self-insured plans), and I just don't see that XML and web services is going to revolutionize the field. For our medicare people, most of the providers require paper for each and every little thing - having a disdain for anything electronic when it comes to dealing with government agencies. They want that signed piece of paper in their files for their own protection.

The HIPAA requirements aren't so much a standard on how to share information, as they are policies for protecting data from prying eyes - privacy . As such, HIPAA regulations are not intended to improve efficiency of information exchange so much as they are to provide a means where the necessary stuff can be exchanged without divulging too much. That's not to say that it isn't a positive move. In the past, many of our vendors received their data via email or http uploads in raw text files. Nowadays, more and more of them are setting up secure ftp and requiring pgp encryption - a good thing.

As for the file formats themselves, there is currently a certain amount of inefficiency due to each vendor having a different file layout. But even with XML, you have to cater the data to each vendor because their plans are structured quite differently. You also get into the trap of universality with the XML formats. For example, BCBS used their own proprietary format until very recently which was described in a 50 page document. Starting next year, they are shifting to an XML format for their eligibility files that is much more generic. Unfortunately, the document for creating these XML files is well over 250 pages long and leaves many unanswered questions about what is (a) required; (b) optional; (c) deprecated or irrelevant.

The beauty of flat files is that they have been trimmed to the bare necessities over the years. With XML, it seems that the vendors are using the kitchen sink approach - because it doesn't cost anything to make them optional in the DTD, they see no need to filter out the inane. Problem is that some of the optional stuff is necessary for our business - so we have to spend an inordinate amount of time dissecting the file layout to determine what is safe to ignore. In the end, you still have to make educated guesses about how the information is used by the vendor in their claims processing software and how it impacts the data that will be returned to you at the other end.