Find some people who don't want it to work, and ask them, is my advice.
If you want to count those who are po'd about trashing the ABM treaty as not wanting it to work, then I think you are misguided.
If you want to count those who are looking for things to campaign on in '08 and think that Bush failures to advertise make good candidates, then I think you are misguided.
If you want to count those who know what a boondoggle "star wars" was, and tend to blame Republicans/Reagan for it and think that the similarities to this project are too obvious to ignore, then I think you are misguided.
I would have added the technological doubters, but you dismissed them already in your post.