When iTunes starts up, it will load the entire iTunes Library.itl into memory. As this file grows in size, iTunes will become sluggish and it will finally become so slow that RAM memory upgrades and high core counts of processor speeds will not help. Thus it is a moving target the hardware's ability to keep up. Of course one will always be limited by the size of a single hard drive.
In the record industry I have seen clients with 500 GB to 4 TB worth of songs (based on encoding this could be about 30,000 to 100,000 song files) in iTunes running the fastest possible systems with the maximum RAM and there is a very pronounced delay that starts at about 500 GB of songs and perhaps about 50,000 files (assume 256 kps encoding with avg. 4 min song, ~50,000 songs).
There are things that one could do to optimize the situation but it will have little impact if you start growing larger than what your hardware will allow. I would limit playlists to the minimum and keep the library as simple as possible with naming conventions kept as traditional as possiable. As an aside, the iOS devices currently have a 1,300 playlist limit. One could also opt not to have album art downloaded to save the processor clicks these loads will take away, everything will count with gargantuan libraries.
For most people today (9/27/2011) running a typical system, try not to go over 30,000 songs if you value the user experience. There is no doubt this will not be an issue in the future.
I've seen other posts that talk about a 25,000 song limit in iTunes Match.
They should have a solution for people who have vast collections. Even if they are corner cases now, they won't be for long.