It's just our DNA talking. The basic programming:
Women find attractive a man who commands respect and has influence in the community because he provides a secure environment to protect her DNA in her children (Note: he need not be the father).
Men find young, healthy women attractive because they maximize the probability that their DNA will survive until birth. After that, it doesn't matter who takes care of the offspring.
Of course, we overlay on top of this a lot of cultural and emotional/intellectual motivation and modification, but this is the basic programming we come with from the factory. We are not slaves to our biological imperatives, unless, of course, we refuse to admit to ourselves we have any.