...to fuck off.
I don't care if your legs light up the sky light a great searchlight. During the daytime. If you're in your own yard doing work, and you're not breaking any statutes, then, well, see above.
I've a neighbor who mows his lawn wearing dark-blue relatively nice slack-style shorts, boat shoes, and a polo shirt. I think he's pretty damn ridiculous. I put on just about my rattiest clothes available to do yardwork like that.