The competent ones have dictionaries of a few thousand likely passwords that they try on every login first. These will crack a significant fraction of accounts if there is no strong password policy, and running through them can be done reasonably fast.
Now suppose that there are 10 characters in the password, which could be upper case, lower case, or numbers. 62 possibilities. That is 62**10 possible combinations or 8.39299e17. Suppose that you are trying combinations at the rate of 1 billion per second. (You aren't, your CPU doesn't go nearly that fast.) Then it will only take you 8.39299e8 seconds to run through the possibilities. So after 13 years of hard work, the odds are still against you succeeding. Too bad the password was changed on you 12 years and 11 months ago!
Only incompetent bad guys use brute force on this problem unless the set of possible passwords is very limited.
Cheers,
Ben