Brown says "Dyson is in effect telling us that we should believe our own senses rather than the experts in the field."

No, that is not what Dyson said. You've posited a straw man arguement. Dyson actually says "It is much easier for a scientist to sit in an air-conditioned building and run computer models, than to put on winter clothes and measure what is really happening outside in the swamps and the clouds. That is why the climate model experts end up believing their own models." What he is saying is that instead of writing dubious modeling programs, climate scientists should go out and take real data measurements. Measurement is the essence of real science, not models. For example, remember Sagan's computer models of extra-solar planetary systems in the 80s/90s, the models with circular orbits and NO hot Jupiters, his computer models that said the oil field fires in Gulf War I would give us years of global cooling?

Unfortunately, computer models can and often are 'tweaked' to produce the expected results. I'm in the field and have maintained that here for years. Dyson says "Concerning the climate models, I know enough of the details to be sure that they are unreliable. They are full of fudge factors that are fitted to the existing climate, so the models more or less agree with the observed data. But there is no reason to believe that the same fudge factors would give the right behavior in a world with different chemistry, for example in a world with increased CO2 in the atmosphere." Dyson has apparently looked at the models themselves in some detail. Perhaps he's just incompetent.

The real problem is that Dyson is a heretic in the Church of Anthropic Panic Global Warming and must be discredited, even by mis-stating his words in a disengenuous manner.