Ok, I've reinstalled from scratch, and clicked the X11 install as well (the only extra option, figures it would not be on by default).
When "executing" X11 it spawns a bash xterminal. I can then use this to ssh -X to another box, which in turn correctly displays the X sessions back to my laptop.
Thanks.
Note to Peter: nyah nyah.