Maybe we're asking the wrong questions.
You seem to be seeking a spiritual understanding. Perhaps you should be trying to determine what kind of spiritual life you want.
In other words, what kind of a world do you want to live in? How would you like god to be? What is your vision of the soul? What kind of relationship with the infinite would you like to have?
I know that seems kind of presumptious, but it is pretty clear that we don't seem to have much to go on in our search; no chart, no compass, no star to guide us. Maybe we have every right to demand one.
Personally, I refuse to get pushed around by the cosmos. Or at least, I refuse to acquiesce to it meekly. I may be powerless in this mad universe, but I don't have to like it. You don't get to heaven by kissing god's ass.