I never considered this a primary threat, whether real or not, since there are many easier ways to get a few pounds of anthrax into the United States. But that being said:

A model airplane built by an amateur flew across the Atlantic Ocean. It was limited to 11 pounds by contest rules, but obviously a plane built without such a limitation could easily do the job and carry several pounds of something nasty. And with anthrax or other biologicals, all it would have to do is blow up somewhere in the sky over the U.S.

Do I think Iraq was doing this? Not really. They were building missiles that they weren't supposed to, not airplanes they weren't supposed to. But it's far from a fantasy.