This would certainly explain why we bring dogs and cats into the house.
They look so much like us.
What you describe is not "defense of the gene pool", but the apology for racism. Humans have accepted those "not like us" for millennia, including interbreeding with humans who have different physical characteristics. Some societies go out of their way to train people not to do this. That is not a biological imperative. If it were, it would be universal, and it isn't.