Real safety...
...is in getting all our eggs out of the one basket. Thus a Mars Colony supported by rapid transit links to Earth i.e. Mag-Beam or similar. Can't properly colonise with near Hohmann trajectories and Mars Semi-Direct... though they're a good start, but ultimately unsustainable.
And industrialising the NEOs and the "stealth" comets etc etc. would permanently remove that threat. But that will take time. We need to learn how to live in space and give the asteroids and comets a real value - make them too valuable to let crash into planets randomly.