Greece's lack of planning and preparedness has alarmed many nations
In terms of security Greece has lots of experience with domestic terrorists but little with international ones like Al Queda. The worry is that Al Queda sleeper cells could already be in Greece and just waiting for the Games to come.
With a population of barely 10 million Greece is one of the smallest countries to ever host the Olympic games, and many including even the IOC have questioned whether they really possess the resources and the wherewithall to pull it off. I know that a lot of projects, and one major stadium, are way behind schedule.