HR: 0800h
AN: IN31A-0078    [Abstracts]
TI: Reuse Requirements for Generating Long Term Climate Data Sets
AU: * Fleig, A J
EM: albert.fleig@gsfc.nasa.gov
AF: PITA Analytic Sciences, 8705 Burning Tree Road, Bethesda, MD 20817, United States
AB: Creating long term climate data sets from remotely sensed data requires a specialized form of code reuse. To detect long term trends in a geophysical parameter, such as global ozone amount or mean sea surface temperature, it is essential to be able to differentiate between real changes in the measurement and artifacts related to changes in processing algorithms or instrument characteristics. The ability to rerun the exact algorithm used to produce a given data set many years after the data was originally made is essential to create consistent long term data sets. It is possible to quickly develop a basic algorithm that will convert a perfect instrument measurement into a geophysical parameter value for a well specified set of conditions. However the devil is in the details and it takes a massive effort to develop and verify a processing system to generate high quality global climate data over all necessary conditions. As an example, from 1976 until now, over a hundred man years and eight complete reprocessings have been spent on deriving thirty years of total ozone data from multiple backscattered ultraviolet instruments. To obtain a global data set it is necessary to make numerous assumptions and to handle many special conditions (e.g. "What happens at high solar zenith angles with scattered clouds for snow covered terrain at high altitudes"?) It is easier to determine the precision of a remotely sensed data set than to determine its absolute accuracy. Fortunately if the entire data set is made with a single instrument and a constant algorithm the ability to detect long term trends is primarily determined by the precision of the measurement system rather than its absolute accuracy. However no instrument runs forever and new processing algorithms are developed over time. Introducing the resulting changes can impact the estimate of product precision and reduce the ability to estimate long term trends.Given an extended period of time when both the initial measurement system and the new one provide simultaneous measurements it may be possible to identify differences between the two systems and produce a consistent merged long term data set. Unfortunately this is often not the case. Instead it is necessary to understand the exact details of all the assumptions built into the initial processing system and to evaluate the impact of changes in each of these assumptions and of new features introduced into the next generation processing system. This is not possible without complete understanding of exactly how the original data was produced. While scientific papers and algorithm theoretical basis documents provide substantial details about the concepts they do not provide the necessary detail. Only exact processing codes with all the necessary ancillary data to run them provide the needed information. Since it will be necessary to modify the code for the new instrument it is also necessary to provide all of the tools such as table generation routines and input parameters used to generate the code. This has not been a problem for the people that make the first set of measurements of a given parameter. There was no similar predecessor global data set to match and they know what they assumed in making their measurements. But we are entering an era when it is necessary to consider the next generation. For instance the entire 30 year global ozone data set that started with the Total Ozone Mapping Spectrometer instrument launched in 1978 on the Nimbus 7 spacecraft was produced by a single science team. Similar measurements will be made well into the middle of the coming century with instruments to be flown on the National Polar Orbiting Environmental Satellite System but the original science team (unfortunately) will not be there to explain what they did over that period
DE: 0429 Climate dynamics (1620)
DE: 0520 Data analysis: algorithms and implementation
DE: 1600 GLOBAL CHANGE
DE: 1640 Remote sensing (1855)
SC: Earth and Space Science Informatics [IN]
MN: 2007 Fall Meeting