Wednesday, January 04, 2017

The Statistical Tie Fallacy

Regarding the question of whether UAH LT's annual average set a new record or not -- this question has come up before, in the context of political polling and who's ahead in the political race.

I remembered reading about it long ago, on Kevin Drum's blog, who was writing about two hypothetical political candidates where a poll showed their different percentages (of voters favoring them) less than the statistical error of the blog.
In fact, what we’re really interested in is the probability that the difference is greater than zero — in other words, that one candidate is genuinely ahead of the other. But this probability isn’t a cutoff, it’s a continuum: the bigger the lead, the more likely that someone is ahead and that the result isn’t just a polling fluke. So instead of lazily reporting any result within the MOE as a “tie,” which is statistically wrong anyway, it would be more informative to just go ahead and tell us how probable it is that a candidate is really ahead.
Drum asked two statistics professors at California State University, Chico, who gave him formulas to calculate this table:

I'm not sure if we can directly use this to calculate the UAH case or not. But if we take the percentage lead (which really should be labeled "percentage point lead") of 2%, and a margin of error of 5% (5 percentage points), we get a probablility of 65%, almost identical to my calculated value of 66%.

Drum followed this up with another post on the same subject a few days later, and over the years others have weighed in on the topic, all agreeing with him.

So now I'm pretty sure that UAH is wrong, 2016 and 1998 aren't in a statistical tie, and they were perhaps looking to spin the numbers toward the non-warming side. And I wonder, if 2016 had been 0.02°C cooler than 1998, if they'd have claimed it a "statistical tie," or just never mentioned it. 


David in Cal said...

As an expert in probability and statistics, I'm befuddled by Spencer's article. I don't think there's any real meaning to a statistical measure of whether 2016 was the warmest year on record.

Anyhow, it's unimportant. Models and conclusions aren't based on one data point; one looks at all the data available. Whether or not 2016 was the warmest year, the UAH satellite-based temperature has clearly been rising since 1979, at a rate of around 1.5 to 2 deg C per century.


mandrewa said...

Actually David in Cal,

UAH shows a rise of 1.2 deg C per century and RSS says the same thing.

David Appell said...

David, what mandrewa said -- UAH's LT trend is 0.12 C/decade.

But extrapolating over a century is meaningless, because there are too many changes going on now and will in this century (more GHGs in atmo, positive feedbacks, etc).

You could only extrapolate 40 years of data over a century if you thought the century have the same governing factors as the 40 years does. That's just not true with modern climate change.