Big Brains Are High Energy
A low calorie environment could select against large brains, but I think once you reach our level of intelligence you basically have to get to starvation level before you could kill us off. We've come to dominate this planet because brains have proved more useful than big teeth, high speed, big claws, big horns, and even sheer size. We are the deadliest hunter and the most dangerous plant eater to attack (at the group level at least).
Regards,
Mark