Does that fall under "your do no harm?"
What about a pharmacy selling medication?
What about a food store?
If the white only food store has prices 20% less than the black serving one next door, it that harmful?
What about if the nearest black serving food store is 20 miles away?
What if the area that that has the most jobs only has food stores that serve whites only? Blacks couln't work there because they couldn't eat.
I say the "cause no harm" could be twisted any way you want.