"If we examine what happened, I'd say an earthquake causing a tsunami has more chances to happen than a meteorite causing a tsunami."

That seems right, it might be about 10 to 1 or higher chance of the cause being an earthquake. Though if you count tsunami with waves only higher than 30' the odds could closer to even and wave 50' or higher probably favor impactors.

"Also, the East-West rivalry led to the conquest of space which led to a manned space program which led to building rockets with a high payload capacity, which might be useful against asteroids.

From our knowledge of rocks circulating in the Earth's vicinity, shouldn't we assume the chances of being hit by a small rock are higher than by a big one?"

We have detected and plotted most of these rocks 1 km in diameter or larger, so in the next century or so there is a near zero chance of being hit by these larger rocks.

"There is a rock size/mass threshold above which it becomes very dangerous. Are there plenty of them?"

Any rock bigger than 100 meters in diameter is equal to power of many nuclear weapons. These impact the earth in time periods of greater than once every 100 years- I would say the odds favor that 100 meter or larger rock will not hit Earth in the 21 Century.
If lower the diameter to say 50 meters or larger in diameter, it seems to me that it's somewhere close to 50% chance such a rock will hit earth in the 21 century.
Rocks the size of school buses or houses are and might have already hit Earth in the 21 century.
Most rocks the size of a house or smaller are going to explode harmlessly in the atmosphere.

There are different kinds of threats from potential impactors. One threat is a nation mistakenly assuming a impactor is evidence of a nuclear attack. This issue has been and is being addressed as far as I know. But there are no shortage of fools on this planet and circumstances could be such that errors are always possible in this regard.

Since 70% of the planet area is ocean, an impactor large enough size to actually hit the surface will probably hit the ocean. If a space rock impacts the ocean it could cause a tsunami. With the exception that some space rock [about 5%] are mostly iron and a iron space rock the size of a house could impact the earth surface- a house size impactor hitting the ocean would not cause a tsunami- but such a rock could destroy a city if it hit it- though obviously if such a rock hit land it would probably hit in areas where few people were living. And an air burst such as Tunguska event that occur over ocean would also probably not cause cause a tsunami or any noticeable damage.

I would say an impactor in the range of 100 meter or larger in diameter impacting the ocean is biggest threat to human life and property in the next hundred years, but say compared to the weather [hurricanes, tornadoes, rain, lightning, ect- weather will easily cause more deaths and property damage as compared to a possible impactor]. In terms of threat I think impactors in the near term are more comparable to threat level from earthquakes or volcanoes.

Another risk is a Tunguska type event over a populated area.
Tunguska event:
"Although the meteor or comet burst in the air rather than directly hitting the surface, this event is still referred to as an impact. Estimates of the energy of the blast range from 5 megatons to as high as 30 megatons of TNT, with 10–15 megatons the most likely—roughly equal to the United States' Castle Bravo thermonuclear explosion set off in late February 1954, about 1,000 times as powerful as the bomb dropped on Hiroshima, Japan and about one third the power of the Tsar Bomba..."

There is a very low chance of this happening over a city, even low chance it happening in a populated rural area. Tornadoes are obviously a larger threat- as they occur often and over larger areas.

The next threat regarding impactor is what we don't know.
For example are asteroids mostly solid bodies or are they mostly rubble piles.

And finally the threat of impactors in long term [thousands of years] and broadly in regards to human and animal species. I think eventually impactors would be one of many factors which could cause extinction. If you include broadly all elements of the space environment- such as supernova, and things we don't know about yet, we will have deal with all these if we want life to survive.

"It seems the chances of being hit by a meteorite capable of destroying a city are far higher than an asteroid impact causing major cataclysmic changes on Earth. And yet, even the former hasn't happened yet, on recorded history."

I think it's fair to say we almost didn't know know about Tunguska- and that is one example of an impactor that could destroy a city. Suppose instead of Tunguska happening in June 30, 1908, it had occurred June 30 1608, would anyone have investigated it? And if someone had, would anyone be interested?
What words would you use 300 years ago to describe Tunguska? Suppose someone was willing to pay to publish a book you wrote, what would you write?
The simple fact is no one could even understand it 300 years ago- just as they couldn't understand what nuclear blast was or if it was even remotely possible.
Fifty years ago, most people didn't think a space rock 1 mile in diameter could possibly hit earth and that the craters seen on the Moon were probably volcanic in origin.

The towns and cities of the modern world occupy a very small area of the land of the land area of this planet- obviously, 1000 years ago they occupied less land area.

So, what you need is evidence of a crater.

"Most of the terrestrial impact craters that ever formed, however, have been obliterated by other terrestrial geological processes. Some examples however remain. To-date, over 160 impact craters have been identified on Earth. Almost all known craters have been recognized since 1950 and several new structures are found each year."
http://www.unb.ca/passc/ImpactDatabase/essay.html

Example in database:
Morasko Poland N 52° 29' E 16° 54'
0.1 km
< 0.01 million years

Another:
Boxhole Northern Territory, Australia S 22° 37' E 135° 12'
0.17 km
.0540 ± 0.0015 million years ago
[so older than "recorded" history]

Another:
Henbury Northern Territory, Australia S 24° 34' E 133° 8'
0.157 km
.0042 ± 0.0019 million year ago
[well within "recorded" history]

Another in Austrailia:
Wolfe Creek:
0.875 km
< 0.3 Ma
Again older then recorded history, but pretty picture:

"When it does and causes huge damage, then priorities will change. "

No doubt what you say is true, but what you predicting is not particularly intelligent behavior.