"This crucial difference seems to elude a lot of people."
Your post is a good illustration.
Let me try and make the example a little more blatant:
In a free country you might have the right to march through a black neighborhood in Klan robes, carrying effigies of black people hanging from nooses.
This does not make it the right thing to do. The ability to make moral judgments is the underlying basis of any freedom.
Opinions like yours from a political philosophy where almost anything in your daily life that is not compulsory is forbidden, is kind of a hoot. Do we need to move into economics and other fields here? From where I stand, the liberal definition of "freedom" is kind of a narrow one, too.