A major part of the SBVT controversy centered on the group's testimony. Among the first to question the first ad was Republican Senator John McCain, a Bush supporter, Vietnam veteran, and former POW. He said, "I condemn the [SBVT] ad. It is dishonest and dishonorable. I think it is very, very wrong".[27] As a naval aviator in Vietnam, McCain had no firsthand knowledge of Kerry's service. The SBVT statements were accompanied by sworn affidavits, although one affiant, Al French, later admitted he had no firsthand knowledge of what he had sworn to.[50]

The first SBVT ad was contradicted by the statements of several other veterans who observed the incidents, by the Navy's official records, and, in some instances, by the contemporaneous statements of SBVT members themselves.

Several major newspapers were also skeptical of the SBVT allegations. For example, a New York Times news article stated, "on close examination, the accounts of Swift Boat Veterans for Truth prove to be riddled with inconsistencies."[51] ABC News's The Note opined, "the Swift Boat ad and their primary charges about Kerry's medals are personal, negative, extremely suspect, or false."[52] Regarding the medal dispute, a Los Angeles Times editorial[53] stated, "Not limited by the conventions of our colleagues in the newsroom, we can say it outright: These charges against John Kerry are false." The editorial argued this position on the basis that "Kerry is backed by almost all those who witnessed the events in question, as well as by documentation." On August 22, 2004 The Washington Post reported: "An investigation by The Washington Post into what happened that day suggests that both sides have withheld information from the public record and provided an incomplete, and sometimes inaccurate, picture of what took place. But although Kerry's accusers have succeeded in raising doubts about his war record, they have failed to come up with sufficient evidence to prove him a liar."[54]

The ABC television show Nightline traveled to Vietnam and interviewed Vietnamese who were involved in the battle for which Kerry was awarded the Silver Star. These witnesses disputed O'Neill's charge that there "was little or no fire" that day; they said that the fighting was fierce.[55] SBVT supporters question whether these witnesses are reliable because they spoke "in the presence of a Communist official",[56] but their account of enemy fire is substantially the same as that previously given by another former VC to an AP reporter[57] and by the American witnesses, including the only SBVT member who was actually present that day, Larry Clayton Lee.[58][59][60][61]

Jerome Corsi has said that a picture of Kerry's 1993 visit to Vietnam hangs in the War Remnants Museum in Ho Chi Minh City as a gesture of "honor" by the Communists "for his contribution to their victory over [the] United States",[62][63] and John O'Neill has stated that Kerry "is in the North Vietnamese war museum as a hero... . one of the heroes who caused them to win the war in Vietnam".[64] The statement is also repeated in "Unfit for Command" (pp 167-174). However, Josh Gerstein of the New York Sun stated in this regard:

“ While the museum clearly honors opponents of the war from America and other countries, it is not clear that the photo of Mr. Kerry is part of that tribute. The picture of the senator hangs among a set of photos devoted to the restoration of diplomatic relations between America and Vietnam in the 1990s.

It was apparently taken as Kerry took part in a delegation President Clinton sent to Hanoi in 1993. Other photos nearby show visits during that period by former American officials who played key roles in the Vietnam War, including a Navy admiral who has since died, Elmo Zumwalt, and a defense secretary, Robert McNamara. A secretary of state during Clinton’s term, Warren Christopher, is also shown meeting Vietnamese officials
”
— Josh Gerstein[65]

In this connection, the webpage Corsi and another anti-Kerry veteran originally published on the Kerry museum photo contained the picture of Robert McNamara's 1995 meeting with General Giap, who was misidentified as "Mao Tse-Tung".[1](Photo #10).

In addition, John O'Neill said that in 1971 John Kerry "wanted to abandon ship and leave the POWs [in Vietnam]" and that "[o]n the Dick Cavett show and elsewhere, John Kerry‘s position was that we should accept the Madame Binh seven-point proposal, which called for unilateral withdrawal, setting a date after which at some future time, we‘d negotiate the return of the POWs. So we would set a date. We would withdraw and then we would begin to discuss how to bring them home".[66] However, in the Cavett debate, Kerry actually said:

“ Now, if we were to set a date for withdrawal from Southeast Asia, we can – the Vietnamese, first of all, have said it will be settled prior to the arrival of that date, but we can set a time limit on that. If the prisoners of war aren't back prior to the arrival of that date, then I think we would have – for the first time in all of our history in Vietnam we would have a legitimate reason for taking some kind of reaction to it. ”
— John Kerry[67]

Early in the advertising campaign, Time magazine surveyed public credence in the SBVT advertisements among those who viewed them. The poll, conducted August 24 through 26, showed that about one-third of viewers believed there was at least "some truth" to the allegations. Among swing voters, about one-fourth felt there was any truth to the ads.[68]

More recently, an early member of the group, Steve Hayes, stated that he came to believe that the group was twisting Kerry's record, and broke with the group and voted for Kerry. Hayes told the New York Times:

The mantra was just 'We want to set the record straight,' Mr. Hayes said this month. It became clear to me that it was morphing from an organization to set the record straight into a highly political vendetta. They knew it was not the truth.

Hayes also told the New York Times that he provided a long interview to Kerry's supporters, backing their version of the incident for which Kerry received the Bronze Star.[69]

The Bush campaign became part of the general SBVT controversy when McCain condemned the first SBVT ad, and said, "I hope that the president will also condemn it." The Bush campaign did not condemn SBVT or the SBVT ads. The campaign did not endorse the group either, stating "We have not and we will not question Senator Kerry's service in Vietnam."[86] Kerry was dismissive of this statement, saying, "Of course, the President keeps telling people he would never question my service to our country. Instead, he watches as a Republican-funded attack group does just that."[87] Kerry also alleged that SBVT was "a front for the Bush campaign. And the fact that the President won’t denounce what they’re up to tells you everything you need to know — he wants them to do his dirty work".[88] When pressed on the issue, President Bush called for an end to all 527 group political advertisements, and challenged Kerry to do the same.[89]

Critics and the Kerry campaign pointed to several specific connections between SBVT and the Bush campaign. The Kerry campaign asserted that Bush campaign headquarters in Florida distributed fliers promoting SBVT events, a charge the Bush campaign denied.[90] Kenneth Cordier, former vice-chair of Veterans for Bush/Cheney (in 2000) and volunteer member of the Bush campaign veterans steering committee, appeared in the second SBVT advertisement. The Bush campaign asked him to resign and stated that it had been unaware of his SBVT involvement.[91]

On August 25, 2004, Benjamin Ginsberg, the top election lawyer to the Bush campaign on campaign finance law, also resigned after it was learned that SBVT was one of his clients. Ginsberg stated that he was withdrawing to avoid being a distraction to the campaign. He declared that he had acted "in a manner that is fully appropriate and legal,"[79] arguing that it was not uncommon or illegal for lawyers to represent campaigns or political parties while also representing 527 groups. He also maintained that he did not disclose to the Bush campaign that he was simultaneously representing the SBVT group. After leaving the Bush campaign, Ginsberg retained his status as counsel to SBVT.

In January 2005, Governor Jeb Bush, the President's brother and Florida chairman for his 2004 campaign,[92] sent a letter to SBVT member and former POW Bud Day, thanking him for his "personal support of my brother in his re-election." In addition, Governor Bush said of the SBVT:

"As someone who truly understands the risk of standing up for something, I simply cannot express in words how much I value their willingness to stand up against John Kerry."[69][93]