Were you a Boy Scout?
I was. Never made Eagle, but my brother and nephews did. My Dad was a Scout leader for many years.
I still say the hostility toward the Boy Scouts by the ACLU is deeper than religious issues. IMO, it comes from the same contempt for middle America that is expressed in a lot of other areas. A good touchstone: ask what they think of Norman Rockwell.