Sometimes there is a reaction including fever, and sometimes that has neurological consequences. No sane person on either side denies that.
The questionable idea is that the large increase in the number of autism cases is explained by something unique to vaccinations. That would have to be a more common reaction than the high fever one, a subtle reaction that is not usually noticeable soon enough for people to make the connection.
The big philosophy question is whether you should take a personal risk in order to help protect the population. In real life (assuming there isn't some big hidden danger like the bogus autism study said) the personal risk calculation for those without some special contraindication and the population risk calculation point the same way: you are safer, and your population is safer, if you take the vaccine. But then you don't have anything to talk about in ethics class.