This is a lot more than just another wick dip.
Not only is a CIA boss vulnerable to blackmail, either by his paramour or by a third party who might find out about it, but he must also be able to discipline subordinates who get caught in similar embarassing positions. How could he fire someone for an indiscretion that he himself was guilty of and was public knowledge? His ability to function was compromised.
I once interviewed for a job at the CIA and I had to fill out a form detailing everything questionable I had ever done in the past. As the interviewer explained it to me, "We don't care if you smoked marijuana in college or had sex with another man, but we need to know it now and we need to know you don't do it any more."