"Why the h*** is it so hard to get back to the moon, or to Mars, over 30 years later!"

Well, there has been robotic landers to Mars, recently. And we had orbitors to both Moon & Mars recently.

The most interesting place to visit on the Moon are the polar regions, doing a Ranger type (or impactor probe) would be something that could be done in polar region fairly inexpensively- I'm using "ranger type" rather loosely, an impactor probe actually is intended to survive impact [though no impactor probe has yet to be used successfully].

But to answer your question more generally, getting anything off Earth is not cheap. Communication satellites are currently the only significant market in the space environment, without markets you are left to the efforts of govt. And these govt efforts are mostly controlled by science interest and selected by peer review type process. So you have govt and committee type selection- is there any further need to explain?