From: "Michael E. Mann" <mann@virginia.edu>
To: Ed Cook <drdendro@ldeo.columbia.edu>
Subject: Re: Comments on "Extending NAO Reconstructions ..."
Date: Thu, 17 May 2001 13:15:02 -0400
Cc: Juerg Luterbacher <juerg@giub.unibe.ch>,  Keith Briffa <k.briffa@uea.ac.uk>, Phil Jones <p.jones@uea.ac.uk>,  "Michael E. Mann" <mann@virginia.edu>,  Scott Rutherford <srutherford@virginia.edu>

<x-flowed>
Hi Ed,

On the road, but just had to chime into this debate briefly.

What you say is of course true, but we have to start somewhere. Step #1 is 
producing a reconstruction. Without some reasonable estimate of 
uncertainty, a reconstruction isn't
very useful in my opinion. Step #2 is producing some reasonable estimate of 
uncertainty. In my mind, this is based on looking at the calibration 
residuals, seeing if they pass some basic tests for whiteness, normality, 
etc., looking at the verification statistics, and seeing if this continues 
to hold up in an independent sample. It is important to use the longest 
instrumental records we have for independent verification where possible. 
Of course, there may be additional biases in the predictors that are 
difficult to identify even in a relatively long verification interval 
(e.g., ultra low-frequency problems w/ fidelity). Step #3 is trying to 
evaluate this as best we can (looking at the frequency domain structure of 
the predictors themselves, seeing if there is loss of variance at very long 
timescales, looking at the robustness of long-term trends to 
standardization issues, etc.), etc...I see this as a successive series of 
diagnostics and self-consistency checks that iterate towards getting a 
reasonable handle on the uncertainties. This is the approach
that we have taken, and I think it is the most appropriate...

I firmly believe that a reconstruction w/ out some reasonable estimate of 
uncertainty is almost useless! If the community wants to use paleodata for 
signal detection, model validation, etc. I believe that this is absolutely 
essential to do, whether or not we can do a perfect job.

I would be very surprised if Hans would disagree w/ my statement above!

anyways, my two cents on the matter...

mike

At 09:50 AM 5/17/01 -0400, Ed Cook wrote:
>Hi Juerg,
>
>I've done an admittedly quick read of your paper "Extending NAO
>Reconstructions Back to AD 1500" and find it to be fine overall. One slight
>correction on pg. 3 concerning the Cook et al. (1998) recon. The tree-ring
>records used also included some from England, as well as the eastern US and
>northern Fennoscandia. On pg. 10, sentence 8-9 in Conclusions, the wording
>is a little confusing. You say "Including station pressure of Gibraltar and
>Reykjavik as predictors in 1821 lead to a decrease of the confidence
>estimates". This almost sounds like you are doing worse when adding in
>Gibraltar and Reykjavik, when I know you mean the opposite. So, a change in
>wording to something like "... lead to increased confidence in the
>estimates of monthly NAO". Also in Table 1, is the Cullen R4 NAO
>reconstruction the one with instrumental data in it? If so, it has used
>some of the same data as yours. I don't recall if R4 is the one with
>instrumental data. But if it is, you ought to mention that.
>
>
>On a thematic note that doesn't have much direct bearing on the paper as it
>stands now (but which may be of interest to Keith, Phil, and Mike as well),
>I have growing doubts about the validity and use of error estimates that
>are being applied to reconstructions, such as those you have applied in
>Fig. 3. First, as you say at the end of the paper, there is a clear
>frequency dependence in the strength of relationship between the actual and
>proxy-estimated data that is not being considered, i.e. "the SE ... become
>smaller when considering low-pass filtered time series" (pg. 10). The
>assumption of the error estimates as now estimated and applied is that the
>error variance is truly white noise, i.e. equally distributed across all
>frequencies. That is surely not the case. This is different from questions
>about autocorrelated residuals, which tell you nothing about the frequency
>dependence of the quality of the estimates. This is where classic
>regression theory falls down. It is based on the notion that each
>observation is a random sample with no time history or frequency domain
>structure. When we use long time series of observations (climate or proxy)
>to reconstruct some climate variable, we are also using predictors that
>have time series structure and history that cannot vary in a completely
>random fashion even if the data could be completely resampled. This is
>because they represent a series of prior "observations" of
>climatic/environmental conditions. This lack of randomness of the
>observations used for reconstructing past climate again causes me to doubt
>the validity of the error estimates being applied. The degree to which the
>reconstruction can actually vary from year to year within the prescribed
>error limits is itself constrained by the time history of the observations
>themselves used for reconstruction. In contrast, the 2SE limits in your
>Fig. 3 prior to 1821 contain almost all of the estimates. This result could
>be used to claim that there is effectively no useful time history of
>variation in the NAO reconstruction prior to 1821 because each estimate may
>fall with equal probability anywhere in the error envelop. I would regard
>this interpretation as completely wrong. Thus, I would say that the decadal
>period of above-average winter NAO in your reconstruction around AD 1700 is
>real, assuming that the predictors used are providing unbiased estimates,
>even though it is fully enclosed by the 2SE limits that intersect zero.
>This is getting towards the debate with Von Storch over "most probable"
>estimates. I am probably not explaining myself well here and undoubtedly
>need to think more about it. But I really think that error bars, as often
>presented, may potentially distort and unfairly degrade the interpreted
>quality of reconstructions. So, are the error bars better than nothing? I'm
>not so sure.
>
>Cheers,
>
>Ed
>
> >Hello Ed
> >
> >thanks very much for your nice mail. I hope these little
> >comments were useful for you and yes of course
> >we hope too that we can merge the data base sometime
> >later on. This would be great.
> >
> >Do you think that you could send me some comments
> >on our paper by tomorrow?
> >Is your paper for the Orense book?
> >
> >Many greetings and till later
> >
> >Juerg
>
>
>==================================
>Dr. Edward R. Cook
>Doherty Senior Scholar
>Tree-Ring Laboratory
>Lamont-Doherty Earth Observatory
>Palisades, New York 10964  USA
>Email:  drdendro@ldeo.columbia.edu
>Phone:  845-365-8618
>Fax:    845-365-8152
>==================================

_______________________________________________________________________
                     Professor Michael E. Mann
            Department of Environmental Sciences, Clark Hall
                       University of Virginia
                      Charlottesville, VA 22903
_______________________________________________________________________
e-mail: mann@virginia.edu   Phone: (804) 924-7770   FAX: (804) 982-2137
          http://www.evsc.virginia.edu/faculty/people/mann.shtml              

</x-flowed>

