Of course its a privilege, and not a right.
Even a soldier, when assigned his weapon, must log its number and he becomes completely accountable for its proper use. And the same goes for automobiles and airplanes, any machine which has a potential to kill and which the user must be responsible for.
Think of any other device with the capability to be used as a weapon: explosives, toxic substances, virulent microbes, radio-isotopes, even dangerous beasts. Ownership of any thing which can be used as a weapon is not a right, it is privilege, and it demands public review and transparency. And that is certainly the case for secret ownership.