The reasons for this are clear--in hindsight.

Nothing ever gets done unless someone sees a profit in it. Exploration has always been about profit, particularly, short term profit. It's what has always driven exploration: gold, furs, spices, tobacco, slaves. Sure, a few other social or religious motives emerged; but the major attractor for investment in exploration has always been easy money, riches: something for nothing.

In space, this has meant mostly communications satellites, and to a lesser extent, earth resources satellites. The Cold War provided another motive, but it essentially was an investment in the competition between Capitalism vs Communism. We (and they) did it for propaganda, for marketing reasons. There were military advantages which also encouraged space technology, ICBMs and espionage satellites, but neither required much more than low earth orbit to realize. We didn't have to put missile silos on the moon.

The scientists around the edges of the great marketing process have succeeded brilliantly in extracting a few crumbs for their missions, and they have been phenomenally successful in scientific terms, but if they have actually produced any direct financial benefit, either through spinoff or basic research, it has been totally unanticipated and those who profited were not those who invested. That isn't exactly the sort of payoff that brings in investment capital, is it? The science community was essentially parasitic on the military, from the early V-2 to the Great Observatories. How long do you think that's going to last in the current economic environment?

The free enterprise effort in space is a lame joke. Space tourism? Mining the asteroid belt? Ice on the moon? Space infrastructure to service a non-existent space industry? Give me a break. There is no forseeable future for capitalism in space, the typical investment house is not going to be pumping dollars in space, although that doesn't stop some individual entrepreneurs from being space groupies. Bless their hearts, I wish them the best, and I hope they find a way to make money there and get filthy rich, but I'm not holding my breath. Let's face it. As far as Wall Street is concerned, there is nothing up there but some blasted lumps of slag and some drifiting icebergs. And if there were milled plutonium slugs lying in neatly stacked heaps on the lunar surface, it would still not pay in dollars or Joules to go up there and harvest them.

There are almost certainly opportunities in space worth our while; energy, knowledge, perhaps even entire biochemistries with unforseen industrial applications. And planetary defense against impactors may be the wisest investment of all. The science which can be done in space will almost certainly have a vast payoff--eventually. But modern capitalism doesn't give a fig about the future. It's interested in next quarter's balance sheet.

At the end of the Middle Ages, the Low Countries saw the agricultural potential of the shallow seas around them. They built dikes, canals, an entire infrastructure of water control projects, even new technologies (windmills), designed to reclaim land from the sea and bring it to productive use. They thought centuries ahead, planetary engineering where only their grandchildren would prosper, but they invested anyway. History is full of other long-term engineering projects which were only expected to turn a profit in the far future. But I don't see much of that kind of thinking in today's captains of industry.

We're much more likely to find something of immediate financial value in space while we're in space. So for the time being, we will have to rely on what little we can skim off national budgets, and direct as much of it as we can into robotic scientific surveys. That's our best bet at finding the next Big Thing that might justify space exploration and even colonization. Until we finally find something up there that will get the attention of the money people, we should be thankful we have any presence in space at all.