Well, firstly Ogg Vorbis support is Still Coming. Experience on the [link|http://www.xiph.org/archives/vorbis/masterindex.html|mailing list] suggests most manufacturers aren't rushing to Vorbis. The key problem is that many of them have been made with MPEG decoding chips which means the CPU can be much smaller and cheaper. Vorbis therefore currently requires a hefty CPU by comparison. That said, the prospect is [link|http://www.xiph.org/ogg/vorbis/hardware.html|looking brighter all the time].
Secondly, if you want the one real format designed for things like jogging, you want MiniDisc. It does require an investment in the format, however, as it is a consumer audio format, not a computer peripheral. This means you can't copy your MP3s to a MD; it doesn't work that way. That said, most MiniDisc recorders will record digital audio from a CD player and AFAIK the "MD-LP" mode will get you about 140 minutes on a MiniDisc (that's twice normal) or double again if you don't mind some audible degradation. The best way to think of MiniDisc, BTW, is all the advantages of a CD (i.e. digital audio, random access, robust media) with all the advantages of a cassette (i.e. recordable, robust media) with some neat tricks of it's own, i.e. track labelling.
Wade.