The latest number I remember reading for the US population is about 291 million. Out of that number, how many are children too young to work? Well, from this table (2002 figures), about 60 million Americans are aged 14 or younger. Another 20 million are aged 15-19 and not all of them work. If you're talking full time employment, I doubt if 25% of them work. Let's pick a value of about 50% to count part time employees, so that gives a rough estimate of about 70 million Americans too young to work.

Now, how about those too old to work? Scrolling down on that last table indicates that a bit over 35 million Americans are age 65 or more. Some of them still work but the percentage probably isn't all that high. Let's guess about 30 million of them no longer work. Combined with the 70 million too young to work, that comes to roughly 100 million out of 290 million.

That still leaves 190 million potentially able to work. However, at the younger end, there are millions more who don't work while they're in college or who stay at home spunging off their parents. Let's guess about 5 million, leaving 185 million.

From here, it gets harder. There are still millions of people, primarily women, who choose to stay at home with their children. There are millions of others who can't work due to physical or mental disability. I can't find an estimate for that. There are people who are unemployed, or on welfare but I don't know how many.

There are as many as 2 million in prison, and others who are finding other means for supporting themselves (e.g. working under the table or perhaps through crime) that aren't counted as being employed. I couldn't begin to estimate how many people fall into these categories.

It still seems like a lot of people between the 185 million mentioned above and the estimated 140 million in the labor pool, but the differences aren't all that significant. Still, the size of the labor pool compared to the population sounds about right.