Even if "on average," life gets wiped out prior to developing advanced civilization, Fermi's Paradox would still operate for the X percent of the time where civilization does develop. Any civ that's just a few thousand years more advanced than ours could anticipate and survive all those problems you described - they could either prevent them (asteroid collisions), move away, or use their technology to withstand them. So the question remains - where are they?

Nick's idea is interesting - advanced technology is better at destroying than protecting (the sword improves faster than the shield), and eventually all civs destroy themselves before doing much interstellar exploration. I'm probably overstating what Nick said though.