Toxins and irritants from cigarette smoke settle in hair and clothing and can cause severe reactions in asthmatics and people with sensitivities when exposed to this. You dont need to be blowing smoke in someone's face to trigger a reaction. At work, if I have a child admitted for asthma, the first thing I ask is if there are smokers in the home. Doesnt matter if they smoke outside. If they want their kid to get better, they have to quit smoking or have no physical contact with their child. Pick your poison, but dont poison your child.
Perfume and cleaning products are an irritant from many people. Whether people are actually allergic to fragrances is up for debate, but the symptoms are real. Wheezing, itching, burning eyes, burning mucus membranes, headache, nausea, vomiting. Sometimes fragrance can trigger migraines, or even an anaphylaxis reaction.
So yeah, people smell. But it's not our natural organic odors that make people sick. (usually).