The Moon is a handy near-Earth object that records the impacts without atmospheric effects or erosion. This paper details that 222 young rayed craters of 5km+ diameter have been catalogued over a surface area of 1.4 e+07 sq km, and they are assessed to have an age of up to 800 Ma. (the scaling law on Earth is that a crater will be 20 times the size of the impacting object and this will be very similar for the Moon, because most of the velocity component of the impact is due to its different orbital velocity, rather than passive accelleration into Earth's gravity well. these craters therefore represent a population of impactors of 250m+ in diameter)

If we apply these statistics to the Earth, with a surface area of 5.1 e+08 sq km, we have an expectation of 1.01 e-05 per year that such an impact will occur on Earth. In other words, there is a recurrence time of around 100,000 years.

This is not particularly long, and is a figure that we should be aware of in our long-term planning, but it doesn't presage the immediate demise of the human race or the end of civilisation.

Equivalent statistics for other sized objects/craters are easily available. For instance, Wikipedia on NEA (not a great scioentific source, but usually a good compilation)

Object size____crater size____Frequency
5-10m_______(atmospheric)___Yearly
50m____________1km________1000 years
250m___________5km________100Ka
1km____________20km_______500Ka
5km___________100km_______10Ma

You can interpolate quite well between these data points and calculate the expectation of any scale of event.

One derivative of the data is the expectation of the largest impact that will occur/have occurred over the 10,000 years of human civilisation. The answer is an object of about 100m diameter, with an associated 2 km crater (and a rather wider devastation zone). The odds are that such an impactor will fall in the oceans, and on land the largest extant recent crater should be around 1.3 km diameter.

If you can hold yourself to a rational argument, then you might like to discuss these statistics...