Your "other hands" make some good points.
Obviously, the optimal outcome is to have as few of these weapons around as possible, but there are limits to what we can do unilaterally, and there are very few nations where I would see their access to nuclear weapons as being dangerous enough to warrant serious action, including military.
Iran and North Korea come immediately to mind, and they are still a major threat. Libya and Iraq would have been two others. I'm not deliriously happy about Pakistan having nukes, but that horse is long out of the barn.