If we are going to have a spacefaring civilization either

1) we get extremely cheap access to space through some mechanism like space elevator, or some unknown propulsion system
or
2) We access resources in cheap gravity wells like asteroids and moons and build in situ or in orbit.

Why would you want to spend energy to bring back heavy stuff to earth. The only things you want to bring back to earth would be extremely unique (extrateresterial samples) or some yet to be discovered substance that can only be manufactured in micro gravity and is worth the cost of bringing back. In general, right now the cost to bring anything back exceeds its worth (exceptions are very small samples of things and experiments and of course human beings). Of course right now, it is not really worth sending human beings for what they've been able to accomplish versus teleoperation or robotics.

As nanotechnology progresses and miniturization continues, it gets more and more obvious that we should be shrinking mass requirements (as we would if we didn't have to worry about supporting human beings). Human beings should be late arrivers after bases have been built robotically. When we learn how to bootstrap up these bases with minimum mass by accessing off world resources, we will be much more prepared to sustain humans off world.

Overall, we need to be approaching everything with a very cost conscience attitude. Exponential growth into space will only occur when it is economically feasible (it can pay for itself). If the private sector can bootstrap some sort of space tourism industry, that will be a start. NASA doesn't help things by constantly creating absurd and unsustainable designs that don't deliver on performance and overrun costs by orders of magnitude.

The mars program acknowledges the need for robotic probes to perform recon as a prerequisite to human visits. What we haven't really recognized is that we should be currently concentrating almost solely on information gathering by maximizing robotic probes. We would be closer to the possibility of sustainable visits to the moon and mars had we gathered information on water and other resources instead of expending the billions we have wasted on ISS and shuttle operations the last decade.

Why does space exploration have to be so political so full of pork barrel wellfare to a few huge aerospace companies? Hmmm, i guess i answered my own question. We should be ruthless in trying to squeeze the most out of every dollar we spend on space. That is the quickest way off world.