Interesting idea
A general who'd commanded in Iraq would certainly bring experience to the job.
A possible objection to any general as SecDef, on general principles, is the separation of civilian and military leadership. Near as I can remember, George C. Marshall was the only Secretary of Defense with military command experience, and he served as SecDef for just a year at the start of the Korean War. The rest have mostly had some military service, but most of their careers were spent outside the service.
According to the DoD bio, Marshall was chosen for reasons similar to your logic:
Given the controversy surrounding Johnson's [predecessor's] performance, and especially the military situation in Korea, President Truman recognized the need to choose a person of great national prestige to head the Department of Defense. His selection of General Marshall eminently met this requirement.
The key question here, in terms of bucking precedent and arguing for somebody with direct military command experience, is whether any of the generals currently serving in Iraq rise to the level of "national prestige". With all due respect, they're not household names.
Hmmm....I know this is an absolutely whacked idea, but what about replacing Rummy with Colin Powell? I know, I know, he and Dubya didn't part on good terms, but in a sort of only-Nixon-can-go-to-China sense, wouldn't you find it reassuring if Bush could turn to Powell at a time like this? And wouldn't Powell love the vindication, after warning Bush of the "Pottery Barn rule"? Nahhh, never happen, too much face for Bush to lose. Still, as long as we're speculating about perfect SecDefs, forgive me my flight of fancy.