Yeah, you really do.
But only if you want to make a credible counterargument.
Saying the guy is a scumbag may be correct, but it does nothing to counter the content of the post, which is where the argument rests. Not on the character of its author.
As the old saying goes, even a blind pig finds an acorn once in a while. Maybe this is Mr. Milloy's acorn, or maybe not. It's up to you whether you want to challenge evidence with evidence, or if you're content with using character attack.
Mr. Milloy's pedigree is enough to make me suspicious of him, but saying he has cooties is not enough to refute the list at JunkScience. I do notice that he uses bits and pieces, but so do we all. Maybe a closer look at the pieces he doesn't use would be a place to start.