In Spanish colonies where African slaves were not imported in large numbers, very often the indigenous peoples were the lowest caste. In some places in Latin America, The more "European" you looked, the higher your social status. The mixed bloods, or mestizos, were in the middle, the Indian peoples were at the bottom. Cuba's indigenous peoples were exterminated by the sword and smallpox. But at one time African slaves outnumbered the white population. The Spanish crown had to encourage immigration from the Peninsula and the Canary Islands to keep the population white.

Watch a Mexican telenovela sometime. The actors are always playing upper-class twits, and they often look N European: blonde, fair skinned, blue-eyed. They do not reflect the racial mix of the average Mexican population. Spaniards, particularly from the N part of the country, are fair, often of Celtic stock. The Cubans even have a saying "He was as red-haired as a Spaniard". Wherever they took over, their features were a marker of the ruling class. The rich could even afford to import white girls, or bring their families over with them. The lower-class adventurers and freebooters, the conquistadudes, had to consort with the natives. After a while, skin color became associated with your colonial status.