It sounds like his initial application wrote this in the stupid naive way that C programmers love - he wrote a double loop.

With 600,000 files that means that it wants to do about 180,000,000,000 comparisons of one file against another. Sure, C is fast, but that will take a while.

Cheers,
Ben