Execution of the innocent
As Eri has pointed out, the biggest problem with capital punishment is the execution of the innocent. Sure, it takes a long time and costs a lot of money to go through the appeals process, but the alternative is killing far more innocent people than we already do. I read a number of years ago that the proven executions of innocents in the US in the 20th Century is (approximately) 45 people. That's 45 innocent human beings who were premeditatedly murdered by the government.
I have already stated I support capital punishment in only the most severe cases. I also believe this should only be done when the evidence is beyond any doubt vs beyond a reasonable doubt.