I suppose a big part of what makes it look so scary is that shit -- drop-offs, trees, curves -- comes at you suddenly and unexpectedly, "out of nowhere" in the video. But I suspect that is (at least in large part) an artifact of the camera angle: He has his GoPro angled to show us the front end of his bike at all times. I suppose that's because without those visual cues to "anchor" the viewer to the bike, the video would feel weirdly "disembodied", like you're just a viewpoint zooming through space; might make viewers seasick.

But the rider himself, I think, doesn't actually look at his hands, handlebar, and front wheel all the time: His gaze is directed higher, farther out; he sees farther ahead. The faster you go, the farther ahead you focus. Where you're aiming at shifts more gradually, as you're already focusing on the next hundred yeards out, or the next two hundred after that. Driving a car at sixty miles an hour, this mad rush of landscape that you're zooming across at tens of yards every second would be a much more nausea-inducing blur than this is, if that were what you were looking at. But it isn't; that swishes by more or less unnoticed in your peripheral vision, while you're actually looking further ahead. Same here for the rider, but not for us viewers "riding" in the GoPro that looks more downwards and less forwards than he does.

::TL;DR: This probably looks much scarier on the video than to the rider, because much of what we get to see is in his peripheral vision, while he focusses farther out than we can see.

I think.