From: Ben Santer <santer1@llnl.gov>
To: "Thorne, Peter" <peter.thorne@metoffice.gov.uk>, Leopold Haimberger <leopold.haimberger@univie.ac.at>, Karl Taylor <taylor13@llnl.gov>, Tom Wigley <wigley@cgd.ucar.edu>, John Lanzante <John.Lanzante@noaa.gov>, "'Susan Solomon'" <ssolomon@al.noaa.gov>, Melissa Free <Melissa.Free@noaa.gov>, peter gleckler <gleckler1@llnl.gov>, "'Philip D. Jones'" <p.jones@uea.ac.uk>, Karl Taylor <taylor13@llnl.gov>, Steve Klein <klein21@mail.llnl.gov>, carl mears <mears@remss.com>, Doug Nychka <nychka@ucar.edu>, Gavin Schmidt <gschmidt@giss.nasa.gov>, Steven Sherwood <Steven.Sherwood@yale.edu>, Frank Wentz <frank.wentz@remss.com>
Subject: [Fwd: Re: JOC-08-0098 - International Journal of Climatology]
Date: Fri, 25 Apr 2008 13:19:18 -0700
Reply-to: santer1@llnl.gov

<x-flowed>
Dear folks,

On April 11th, I received an email from Prof. Glenn McGregor at IJoC. I 
am now forwarding that email, together with my response to Prof. McGregor.

Prof. McGregor's email asks for my opinion of an "Addendum" to the 
original DCPS07 IJoC paper. The addendum is authored by Douglass, 
Christy, Pearson, and Singer. As you can see from my reply to Prof. 
McGregor, I do not think that the Addendum is worthy of publication. 
Since one part of the Addendum deals with issues related to the RAOBCORE 
data used by DCPS07 (and by us), Leo responded to Prof. McGregor on this 
point. I will forward Leo's response in a separate email.

The Addendum does not reference our IJoC paper. As far as I can tell, 
the Addendum represents a response to discussions of the original IJoC 
paper on RealClimate.org. Curiously, Douglass et al. do not give a 
specific source for the criticism of their original paper. This is 
rather bizarre. Crucially, the Addendum does not recognize or admit ANY 
ERRORS in the original DCPS07 paper.

I have not yet heard whether IJoC intends to publish the Addendum. I'll 
update you as soon as I have any further information from Prof. McGregor.

With best regards,

Ben
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel:   (925) 422-2486
FAX:   (925) 422-7675
email: santer1@llnl.gov
---------------------------------------------------------------------------- 


</x-flowed>

Attachment Converted: "c:\eudora\attach\[Fwd Re JOC-08-0098 - Interna.pdf"
X-Account-Key: account1
Return-Path: <santer1@llnl.gov>
Received: from mail-1.llnl.gov ([unix socket])
	 by mail-1.llnl.gov (Cyrus v2.2.12) with LMTPA;
	 Fri, 11 Apr 2008 11:19:24 -0700
Received: from smtp.llnl.gov (nspiron-3.llnl.gov [128.115.41.83])
	by mail-1.llnl.gov (8.13.1/8.12.3/LLNL evision: 1.6 $) with ESMTP id m3BIJN5F012995
	for <santer1@mail.llnl.gov>; Fri, 11 Apr 2008 11:19:24 -0700
X-Attachments: None
X-IronPort-AV: E=McAfee;i="5200,2160,5272"; a="31695223"
X-IronPort-AV: E=Sophos;i="4.25,642,1199692800"; 
   d="scan'208";a="31695223"
Received: from dione.llnl.gov (HELO [128.115.57.29]) ([128.115.57.29])
  by smtp.llnl.gov with ESMTP; 11 Apr 2008 11:14:37 -0700
Message-ID: <47FFAA8D.8040308@llnl.gov>
Date: Fri, 11 Apr 2008 11:14:37 -0700
From: Ben Santer <santer1@llnl.gov>
Reply-To: santer1@llnl.gov
Organization: LLNL
User-Agent: Thunderbird 1.5.0.12 (X11/20070529)
MIME-Version: 1.0
To: g.mcgregor@auckland.ac.nz
CC: Leopold Haimberger <leopold.haimberger@univie.ac.at>,
        "Thorne, Peter" <peter.thorne@metoffice.gov.uk>
Subject: Re: JOC-08-0098 - International Journal of Climatology
References: <363780847.1207875178234.JavaMail.wladmin@tss1be0004>
In-Reply-To: <363780847.1207875178234.JavaMail.wladmin@tss1be0004>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

<x-flowed>
Dear Prof. McGregor,

Thank you for your email, and for your efforts to ensure rapid review of 
our paper.

Leo Haimberger (who has led the development of the RAOBCORE* datasets) 
and Peter Thorne would be best placed to comment on the first issue 
raised by the Douglass et al. "Addendum". As we show in Figure 6 of our 
IJoC paper, recently-developed radiosonde datasets which do not rely on 
reanalysis data for correction of inhomogeneities (such as the Sherwood 
et al. IUK product and the Haimberger et al. "RICH" dataset) yield 
vertical profiles of atmospheric temperature change that are in better 
agreement with model results, and quite different from the profiles 
shown by Douglass et al.

The second issue raised in the Douglass et al. "Addendum" is completely 
spurious. Douglass et al. argue that their "experimental design" 
involves involves "comparing like to like", and satisfying "the critical 
condition that the model surface temperatures match the observations". 
If this was indeed their experimental design, Douglass et al. should 
have have examined "AMIP" (Atmospheric Model Intercomparison Project) 
simulations, in which an atmospheric model is run with prescribed 
changes in observed time-varying sea-surface temperatures (SSTs) and 
sea-ice distributions. Use of AMIP simulations would allow an analyst to 
compare simulated and observed tropospheric temperature changes given 
the same underlying changes in SSTs.

But Douglass et al. did NOT consider results from AMIP simulations, even 
though AMIP data were freely available to them (AMIP data were in the 
same "CMIP-3" archive that Douglass et al. accessed in order to obtain 
  the model results analyzed in their original IJoC paper). Instead, 
Douglass et al. examined results from coupled model simulations. As we 
discuss at length in Section 3 of our paper, coupled model simulations 
are fundamentally different from AMIP runs. A coupled model is NOT 
driven by observed changes in SSTs, and therefore would not have (except 
by chance) the same SST changes as the real world over a specific period 
of time.

Stratifying the coupled model results by the observed surface 
temperature changes is not a meaningful or useful thing to do, 
particularly given the small ensemble sizes available here. Again, if 
Douglass et al. were truly interested in imposing "the critical 
condition that the model surface temperatures match the observations", 
they should have examined AMIP runs, not coupled model results.

I also note that, although Douglass et al. stipulate their "critical 
condition that the model surface temperatures match the observations", 
they do not actually perform any stratification of the model trend 
results! In other words, Douglass et al. do NOT discard simulations with 
surface trends that differ from the observed trend. They simply note 
that the MODEL AVERAGE surface trend is close to the observed surface 
trend, and state that this agreement in surface trends allows them to 
evaluate whether the model average upper air trend is consistent with 
observed upper air trends.

The Douglass et al. "Addendum" does nothing to clarify the serious 
statistical flaws in their paper. Their conclusion - that modelled and 
observed upper air trends are inconsistent - is simply wrong. As we 
point out in our paper, Douglass et al. reach this incorrect conclusion 
by ignoring uncertainties in observed and modelled upper air trends 
arising from interannual variability, and by applying a completely 
inappropriate "consistency test". Our Figure 5 clearly shows that the 
Douglass et al. "consistency test" yields incorrect results. The 
"Addendum" does not suggest that the authors are capable of recognizing 
or understanding the errors inherent in either their "experimental 
method" or their "consistency test".

The Douglass et al. IJoC paper reached a radically different conclusion 
from the conclusions reached by Santer et al. (2005), the 2006 CCSP 
report, the 2007 IPCC report, and Thorne et al. (2007). It did so on the 
basis of essentially the same data used in previous work. Most 
scientists would have asked whether the "consistency test" which yielded 
such startlingly different conclusions was appropriate. They would have 
applied this test to synthetic data, to understand its behaviour in a 
controlled setting. They would have applied alternative tests. They 
would have done everything they possibly could to examine the robustness 
of their findings. Douglass et al. did none of these things.

I will ask Leo Haimberger and Peter Thorne to respond to you regarding 
the first issue raised in the Douglass et al. "Addendum".

Best regards,

Ben Santer

(* In their addendum, Douglass et al. erroneously refer to "ROABCORE" 
datasets. One would hope that they would at least be able to get the 
name of the dataset right.)

g.mcgregor@auckland.ac.nz wrote:
> 10-Apr-2008
> 
> JOC-08-0098 - Consistency of Modelled and Observed Temperature Trends in the Tropical Troposphere
> 
> Dear Dr Santer
> 
> Just to let you know that I am trying to secure reviews of your paper asap.
> 
> I have attached an addendum for the Douglass et al. paper recently sent to me by David Douglass. I would be interested to learn of your views on this
> 
> 
> Best,
> 
> Prof. Glenn McGregor


-- 
----------------------------------------------------------------------------
Benjamin D. Santer
Program for Climate Model Diagnosis and Intercomparison
Lawrence Livermore National Laboratory
P.O. Box 808, Mail Stop L-103
Livermore, CA 94550, U.S.A.
Tel:   (925) 422-2486
FAX:   (925) 422-7675
email: santer1@llnl.gov
---------------------------------------------------------------------------- 



</x-flowed>

