Only when businesses decide the market served by the government agencies isn't worth going after.
We have private schools, private health insurance, private security companies, private retirement brokers, private housing builders, private parks maintained by private companies, private zoos, private museums...
What these institutions have in common is that they serve people with money to pay for their services. The cases of schools are a bit different: often the organization will make some sort of allowance for children from families who can't afford the whole freight. (Please note how well they pay their teachers...)
Government agencies have no competition in areas where private businesses see slim to no margin for profit.