Thursday, June 06, 2013

Speaking of "Epic Fails"

CMIP5-73-models-vs-obs-20N-20S-MTRoy Spencer has a very unprofessional post, gloating over the failure of climate models to accurately project middle troposphere temperatures in the tropics.

I can't imagine any other science where a professional scientist would openly mock the work of his colleagues in this manner, as if it's all a competition between good and evil -- especially by a group that has seen its own share of epic failures over the years.

Surely this result, which I doubt is anything new to modelers, says more work on modeling and understanding climate is needed. (Remember, that is the whole point of all this: understanding climate and its changes.)

But then there is this: the linear trend for the entirety of the two datasets on middle tropospheric temperatures in the tropics is

RSS MT 20S-20N:   0.090 ± 0.028 °C/decade 
UAH MT 20S-20N:   0.030 ± 0.028 °C/decade

That's right -- their trends differ by a factor of three, with UAH by far the lowest -- a fact which is neatly hidden away in Spencer's graph by taking their average. (Added 6/8: How can Spencer claim "...the observations (which coincidentally give virtually identical trends)...."?)

Most people would consider that a failure of its own, and quite a bad one.

Maybe the people doing the measurements should be paying attention to getting their own piece of the science right, and they ought not be giving everyone else cause to wonder if perhaps their own data is extremely inaccurate or biased low.

48 comments:

dana1981 said...

Perhaps worse than the gloating is Spencer's inability to consider the possibility (high probability, in my opinion) that a chunk of that model-data discrepancy is due to a cool bias in the satellite observations.

Wotts Up With That Blog said...

If he is plotting the 5-year running averages of the UAH and NSS datasets, how is he getting such a good match if the trends differ by a factor of 3?

Wotts Up With That Blog said...

Okay, maybe I see. Are his datapoints the averages of the NSS and UAH datasets?

Anonymous said...

NOAA 20N-20S TMT trend nov1978-Apr2013: +0.12°C/decade:
ftp://ftp.star.nesdis.noaa.gov/pub/smcd/emb/mscat/data/MSU_AMSU_v2.0/monthly/Monthly_Update/Merged_Deep-Layer_Temperature/TMT_merged_msu2_amsua5_monthly_1978-April2013_v2.0.anomaly.dat

Anonymous said...

So, let me get this right: two acceptable scientific bodies give predicted results which differ by a factor of three. Someone then combines these, and a host of others (all acceptable), and produces an average of all predicted results; he then compares this average with reality – but is hauled over the coals for averaging the wide range of acceptable projections. Not sure where the argument is…

Radical Rodent

Anonymous said...

Ah, sorry. My error; as a bear of very little brain, these things happen. The average is of two actual readings, though why they should differ by such a large factor remains a mystery to me. However, I do acknowledge that the actual numbers are not very big, the higher still being less than 0.1°C; I have yet to see a thermometer that accurate, but will accept that they might exist. Perhaps the question should be: does the higher of the two actual readings come close to matching the lowest of the predicted figures?

If not, why the gloating?

Radical Rodent

Anonymous said...

I can't imagine any other science where a professional scientist would openly mock the work of his colleagues in this manner, as if it's all a competition between good and evil…

Actually, if you look at history, this is quite common; Newton had his vociferous detractors, with many resorting to non-scientific vitriol, as did Galileo, Darwin, Einstein… The list is quite long. Are you on the side of reasoned argument, or spiteful name-calling?

Radical Rodent

Dodgy Geezer said...

...says more work on modeling and understanding climate is needed..

But I thought that the science was settled?

Remind me again how much the model predictions vary from each other. And from reality, which is rather the point of the exercise...

Note also that, if you are truthful about climate data, the temperature variations are VERY small. under these circumstances, where you are measuring a figure pretty indistinguishable from zero and at the limits of your instrumentation, getting one reading to be twice or three times another is not at all surprising. I have seen alarmists claim that their made-up figures must be correct because they are roughly the same order of magnitude...

igsy said...

"..their trends differ by a factor of three..". How would you have described the situation if RSS-UAH had the same difference, but UAH was zero (and RSS .06)?

Anonymous said...

"Perhaps worse than the gloating is Spencer's inability to consider the possibility (high probability, in my opinion) that a chunk of that model-data discrepancy is due to a cool bias in the satellite observations."

Reality doesn't match with the models so there must be something wrong with reality?

Do you have any evidence for this 'opinion'?

Njall

David Appell said...

Radical Rodent: My post is about

1) Spencer's unprofessionalism.
2) The large discrepency in the measurements.

Regarding 2: given the discrepencies between RSS and UAH, and how their difference has changed over the years, it's not clear either is correctly measuring tropospheric temperatures.

David Appell said...

Anon 2:54 am -- that others did it doesn't make it any less professional....

As William Connolley wrote, "...its just not what you write, if you have any hope of belonging to a scientific community. Its what you write if you know you’ve marginalised yourself and there is no way back."

http://scienceblogs.com/stoat/2013/06/06/dr-roy-spencer-is-sad-and-lonely-and-wrong-part-ii/

David Appell said...

But I thought that the science was settled?

If you thought that, then you were mistaken.

Enough of the science is settled to know that putting more heat trapping gases into the atmosphere traps more heat, and, at the rate we're doing it is and will lead to significant changes in climate. That doesn't mean every last detail is worked out. Calculating climate is extremely difficult, and waiting until it's all predicable to 3 signficant digits will be far too late.

David Appell said...

Reality doesn't match with the models so there must be something wrong with reality?

RSS and UAH aren't measuring "reality" -- their results themselves rely on models, and relatively complex ones at that.

*All* data in science relies on a model.

David Appell said...

Njall: There is precedence for a discrepency between modeled and measured tropospheric temperatures being resolved in favor of the models:

http://www.realclimate.org/index.php/archives/2005/08/et-tu-lt/

David Appell said...

How would you have described the situation if RSS-UAH had the same difference, but UAH was zero (and RSS .06)?

I might have said the trends differ by a statistically signifcant amount. Which they do. So which, if either, is correct?

Anonymous said...

Looking at the data for the trends in temperature. I do not get the significance attached by the author to the factor of 3. These are just results with very significant errors attached to them.

The multiple is irrelevant. In fact given the size of the uncertainties in the two results they are very nearly in agreement with each other. Averaging seems pretty reasonable scientifically. But error bars are always be useful!

But even drawing in some error bars the models still look to be overestimating the rate of temperature change.

Dodgy Geezer said...

...But I thought that the science was settled?

If you thought that, then you were mistaken...


When I say I thought it, what I meant was that I was told it by all the major scientific authorities, and so there should not be any surprises...


...Enough of the science is settled to know that putting more heat trapping gases into the atmosphere traps more heat, and, at the rate we're doing it is and will lead to significant changes in climate. That doesn't mean every last detail is worked out....

Ah. So the fact that CO2 is increasing but the climate is remaining the same is a proof of that, is it? Air temperatures have gone up, but they're now coming back down again. I can't see how much longer you can ignore that when even the senior climate scientists are saying that this wasn't in the predictions, but it will be interesting to find out...

David Appell said...

In fact given the size of the uncertainties in the two results they are very nearly in agreement with each other.

No, they aren't. The error bars are the 95% confidence limits (without autocorrelation). The results do not even overlap at the 2-sigma level.

David Appell said...

When I say I thought it, what I meant was that I was told it by all the major scientific authorities...

Did you read their conclusions in full? It appears not....

David Appell said...

So the fact that CO2 is increasing but the climate is remaining the same is a proof of that, is it?

This is simply an incorrect statement.

The science does not support it, nor do macroscopic indicators: ocean warming, ice melt, sea level rise.

I think you are purposely choosing to keep a myopic view in order to maintain the position that climate is not changing.

Anonymous said...

According to the numbers provided by David Appell the temperature difference is 0.06 and the standard deviation of each estimate is given as 0.028. This means that the standard deviation of the difference (given standard assumptions about distribution etc.)should be 0.028 times the square root of two or approximately 0.04. The t-test for 0.06 being different from zero results in a test-indicator of 0.06/0.04= 1.5 that is too small for the difference to be significant.

It follows that taking the simple average between the two change estimates is the statistically most efficient and best way to proceed.

Since there is no significant difference the accusation of hiding information is off target.

More to the point: for the problem in hand the focusing on relative and not absolute size differences is reckless and at odds with proper statistical procedure.

The attack on Spencer thus seems to serve rhetorics rather than enlightenment.

David Appell said...

The number 0.06 +/- 0.04 is statistically different from zero.

David Appell said...

Note that Spencer writes, "Note that the observations (which coincidentally give virtually identical trends)..."

They do only only if you ignore than UAH and RSS do not agree.

http://www.drroyspencer.com/2013/06/epic-fail-73-climate-models-vs-observations-for-tropical-tropospheric-temperature/

Anonymous said...

Isn't Spencer's result exactly, what you would have expected ?

The best constrained observational based climate sensitivity (Nic Lewis) now stands at 1.6K (and the Otto result would be 1.7K with updated data).

Then, the tropospheric hotspot does not exist, neither in satellite nor baloon data, though models compute tropospheric anamolies at about 1.5 surface temperatures.

Both combined results in a model deviation of almost factor 3 from measured data.

NuclearEnvironmentalist said...

But in looking at Spencer's figure, I also see reference to an average of four radiosonde series, which do appear to agree quite closely with the average of the two satellite series. Hard to imagine that is pure coincidence.

Anonymous said...

"David Appell said...
The number 0.06 +/- 0.04 is statistically different from zero."
Not if you ability to measure zero is
0.0 +/- 0.04.

Anonymous said...

Mr. Appell,

Thank you for the courtesy of replying.

Now we are getting to the point that is my major bone of contention: “…putting more heat[-]trapping gases…”. According to NASA, it is because we have an atmosphere that the surface temperature is what it is, and not 0°F; the implication here is that ALL gasses are heat-trapping. This, I can accept; I can accept that some gasses may be more effective than others at retaining this heat; what I have trouble with is the idea that a very, very small proportion – <0.05% is small, whichever way you look at it; it is less than 5 cents from a hundred dollars – of the atmosphere can have such a massive effect. It is like having a greenhouse with 100m2 of glass, 5 square inches of which is super-greenhouse glass; increase this to 8 square inches, and all your plants are going to burn and wither away. Sorry, I cannot see that.

What I can see is the global temperatures and climates changing for reasons and ways not yet known over the past few billion years, and this slight yet blessed rise since the Little Ice Age a boon for civilization; what I look forward to is longer, balmier summers in England; what I do dread is a return of glaciers to Scotland.

Finally, could you please explain what is wrong with the measurements taken, and how they should have been derived.

Radical Rodent

Anonymous said...

The science is settled” is a trope churned out by the MSM. It may not be scientifically correct, but it has been said by self-declared scientists, and is endlessly repeated by politicians as they try to justify their ever-increasing taxation, and persistence in driving civilization backwards to the Dark Ages.

In desperate attempts to prevent the long-suffering public from seeing through this plethora of lies, there are many, many schemes in effect to distract their attention. However, when the blackouts start, and the public can no longer get their fix of “X Factor” and other such tripe, then will the pianos be broken up. Beware, because it is not only the politicians that will be in their sights, but also those who fed them the (dis)information.

Radical Rodent

Anonymous said...

When you say that 0.06 with a standard deviation of 0.04 is statistically different from zero you are at odds with over 100 years of statistical practice
See https://en.wikipedia.org/wiki/William_Sealy_Gosset

MikeR said...

"Note that the observations (which coincidentally give virtually identical trends" I am not sure what you complaint is against this statement. Here is the full context: "Note that the observations (which coincidentally give virtually identical trends) come from two very different observational systems: 4 radiosonde datasets, and 2 satellite datasets (UAH and RSS)." In other words, the (averaged) trend from the balloons is virtually identical to the (averaged) trend from the satellites. I think that is clearly his meaning.

I was also not too sure what was so very unprofessional about his post. Where exactly was he sneering or mocking? The phrase "Epic Fail" in the title is all I could come up with; the claim that the models are failing is in itself not mocking.
Anyhow, I'm a little surprised that you're criticizing it. I just glanced at Open Mind from Tamino, and every single post I could see was sneering at some stupid+lying+villain. The same with W. Connolley: they are always mocking, and I think much more obviously than Spencer's lone post. Are they not scientists, or not professionals? Have you posted criticizing them? (Maybe you have.)

MikeR said...

By the bye, I didn't see the egregiousness of the "three times the trend" of one satellite dataset over the other. Looking at a time series of each dataset, they look pretty much the same, matching each other well - with hardly any trend at all. If one of them is drifting a little with respect to the other, they should correct it if they can find it. But I don't see how that fixes the issue Spencer is mentioning. Isn't it true that every single trend of those balloons and satellites is much less than every single trend of the models? It's a little hard for me to swallow that you are suggesting that the models might be right instead. I'm afraid that "Epic Fail" isn't such a bad description.

It suggests two issues: 1) The models are off really badly, and can't be relied on for anything important. And 2) They are all off in the same direction. I doubt that indicates corruption or a fraud or any of the stuff fools like to suggest, but it probably does indicate some kind of serious systematic mistake that all the modelers made together. Can you blame the average American who is no longer sure that these very confident-sounding guys have a clue?

David Appell said...

what I have trouble with is the idea that a very, very small proportion – <0.05% is small

The level of stratospheric ozone is less than 10 ppm, yet without it we'd all be dead.

The dose makes the poison.

David Appell said...

Not if you ability to measure zero is
0.0 +/- 0.04


The number 0.06 +/- 0.04 is statistically different from zero. It is a question of statistics, not measurement accuracies. (In fact, you'll note that neither RSS nor UAH give error bars on their monthly numbers.)

The trends from RSS MT Tropics and UAH MT Tropics are not the same, to a confidence level of at least 95%.

David Appell said...

Mike R wrote:
It's a little hard for me to swallow that you are suggesting that the models might be right instead.

The primary point of my post was about Spencer's unprofessionalism.

But there have been times in the past where models and measurements disagreed, and it was the models that were right, not the measurements.

One can't simply assume the measurements are right, especially when they themselves are heavily dependent on their own models. They really aren't measuring tropospheric temperatures so much as they are modelling them.

David Appell said...

When you say that 0.06 with a standard deviation of 0.04

0.04 = 2 standard deviations, not one.

Hence 0.06 is 3 sd's away from zero.

charlesH said...

I'm having a hard time understanding how you got your numbers from the sources linked.

For the UAH source I see at the very bottom a line labeled "Trend" and the value is 0.03 for the "Trcps" column (Trcps defined as 20s-20n). This agrees the 0.030 value stated in your original post.

For the RSS source I do not see at the very bottom a line labeled trend. Instead I see a value of 0.090 for the year 2013 in the -20/20 (tropics)column. This 0.090 value agrees with the one in your original post but I don't think it is a trend value is it?

Where did you get the 0.090 trend value for RSS?

In this graph there doesn't seem to be much difference between UAH and RSS data over the last 30yrs.

http://wattsupwiththat.files.wordpress.com/2013/05/dnes_33_6-8-13.gif

What am I missing?

David Appell said...

I calculated the RSS trend myself.

charlesH said...

"I calculated the RSS trend myself."

Show your work?

If you do a global calc for RSS do you get close agreement UAH as in shown in this graph?

http://wattsupwiththat.files.wordpress.com/2013/05/dnes_33_6-8-13.gif

charlesH said...

Does Spencers point fall apart if one used the same area coverage used in this graph rather than the tropics only?

http://wattsupwiththat.files.wordpress.com/2013/05/dnes_33_6-8-13.gif

David Appell said...

Show your work?

I did -- there is a link to the RSS MT data.

The calculation is too trivial to "show" -- if you need to be shown it, you won't understand it anyway.

David Appell said...

Does Spencers point fall apart if one used the same area coverage used in this graph rather than the tropics only?

Charles: The data is readily available. Learn to do your own calculations.

Anonymous said...

"
David Appell said...
So the fact that CO2 is increasing but the climate is remaining the same is a proof of that, is it?

This is simply an incorrect statement.

The science does not support it, nor do macroscopic indicators: ocean warming, ice melt, sea level rise.

I think you are purposely choosing to keep a myopic view in order to maintain the position that climate is not changing.

"

We were told via various "predictions" (Hansen, IPCC etc ) that the "temp is increasing, dramatically so. This was going to imperril life etc. They presented graphs, with temps. We have the whole climate sensitivity issue, greater than 2.0 but less than 6.0.

My question Mr. Appell is this, at the current trend of actual air surface temp increasing a rate just above 0, compared to the "PREDICTED" increase rate of 0.2C/ Decade (or more - depending on model) - at what point can you no longer defend the models? Where was all the predictions of this heat being lost in the oceans before?

David Appell said...

Do you expect surface temperature to increase monotonically, year after year?

That's absurd.

The 15-year trend of HadCRUT4 is currently 0.05 C/decade. Just 6 years ago it was 0.29 C/decade.

The trend over short intervals fluctuates greatly, due to noise and oceanic weather.

Why is that so difficult to understand??

Most of us are talking about climate. You are talking about noise and weather.

David Appell said...

Where was all the predictions of this heat being lost in the oceans before?

When AGW and (especially) the PDO were going in the same direction, it was easy to see the GW signal on the surface.

Now it is not as easy, since the PDO is going the other way. So you have to look at the entire climate system to see it.

See how understanding advances?

bahamamamma said...

The "Radical Rodent" is quite persuasive. I particularly enjoyed this:

"What I can see is the global temperatures and climates changing for reasons and ways not yet known over the past few billion years, and this slight yet blessed rise since the Little Ice Age a boon for civilization; what I look forward to is longer, balmier summers in England; what I do dread is a return of glaciers to Scotland."

A cooler Earth is much more to be feared than a warmer Earth so why would anyone want a cooler climate?

David Appell said...

Rodent: Has it ever occurred to you that not everyone lives in England?

It's true -- you can look it up.

David Appell said...

A cooler Earth is much more to be feared than a warmer Earth so why would anyone want a cooler climate?

Has anyone, anywhere, said they want a cooler climate?

Please provide a citation -- because I have yet to read that anywhere.

What *I've* read are worries about are ability to adapt to rapid climate change, of order 0.15-0.2 C/decade.

That is enormous, by historical and geological standards. Why should it not be worrisome?