National Aeronautics and Space Administration

National Climate Assessment

NASA National Climate Assessment (NCA) Activities

NASA Centers Call for Proposals

National Climate Assessment - 2011 NASA Centers Call For Proposals

Lead PI and Center: Duane Waliser, Jet Propulsion Laboratory
Title:  Enabling Regional Climate Model Evaluation: A Critical Use of Observations for Establishing Core NCA Capabilities
Proposal type: Enabling Tools
Project website:
Research article:


Regional Climate Model Evaluation System (RCMES)

The Regional Climate Model Evaluation System (RCMES). The database component (Regional Climate Model Evaluation Database: RCMED) extracts and stores data in a common format for easy processing. The Regional Climate Model Evaluation Toolkit (RCMET)  regrids data from RCMED and/or the model data archive so that the observation and model data are on a spatially and temporally identical grid. It also calculates and visualizes evaluation metrics.

State, national and international assessment reports (ARs) are growing in their importance to provide a scientific resource for the understanding and assessment of climate change and its impacts crucial for economic and political decision-making. A core element of these ARs is the model projections that not only provide a foretelling of physical indicators of future climate but also indirectly provide information on societal impacts and thus provide a key resource for addressing adaptation and mitigation questions. These quantitative projections are based on multi-component, coupled dynamical global and regional climate models (GCMs and RCMs). Because of the critical input such models have to assessments; it is a high priority to bring as much observational scrutiny to them as possible. This requires the systematic application of observations from gridded satellite and reanalysis products as well as in-situ station networks. Enabling such multi-variate evaluations is essential for advancing model development, and performing quantitative model comparison, evaluation and uncertainty analyses. While systematic multi-model experimentation and evaluation have been undertaken for years and in many facets for global-scale assessments (e.g., IPCC), the development and application of infrastructure for a systematic, observation-based evaluation of RCMs is less mature, with a significant portion of (NASA) satellite and other data being highly underutilized for this purpose.

The Regional Climate Model Evaluation System (RCMES) development was initiated via the American Recovery and Reinvestment Act of 2009. The present, initial system is populated with a number of contemporary climate and regionally relevant data sets (e.g., AIRS, TRMM, CRU, ERAInterim), with the ability to ingest additional data sets of wide variety of forms, ingest RCM or regionally subset GCM output, apply a number of useful metrics (e.g., RMSE, bias, correlation, PDF analysis) and visualize the results in terms of time series, maps, Taylor Diagrams, etc.

Evaluation of Annual-mean Precipitation in the NARCCAP Hindcast Study using RCMES

(Left) RCMES allows to calculate model biases (b-h) against selected observation (a). (Right) It also can compare variability of multiple observations and model outputs to visualize model errors and uncertainties due to different observation data. Adapted from Kim et al. 2013, J. Climate.

Proposed Effort:
This proposal plans to implement and exercise a RCMES for the U.S. NCA. The plan will increase the utility of NASA satellite and reanalyses data sets, as well as local/regional in-situ data sets, for the systematic evaluation of regional-scale fidelity of RCMs and GCMs. This activity is being proposed in close partnership with the North American Regional Climate Change Assessment Program (NARCCAP), the leading U.S. multi-model RCM activity that represents the core database of RCM simulation and projection information for the U.S.

The specific objectives are to tailor and apply RCMES to:

  1. identify and incorporate additional key observations for direct comparison to climate model output;
  2. provide additional metrics and visualizations to comprehensively illustrate model fidelity,
  3. apply RCMES to the 3 key types of regional-scale model output relevant to the U.S. NCA.

These include a U.S. regional evaluation of:

  1. CMIP3 and (available) CMIP5 20th century simulations,
  2. NARCCAP RCMs with boundary conditions (BCs) from reanalyses, and
  3. NARCCAP RCMs with BCs from CMIP3 20th century simulations.

This suite of observation-driven comparisons provides a characterization of coarse-grid GCMs, fine-grid RCM-only, and fine-grid RCMs in conjunction with GCMs. The results of the analysis will be presented in a fashion(e.g. “portrait”, Taylor diagrams) and in time to be utilized by the 2013 NCA. Moreover, with RCMES’ utilization of a plethora of NASA satellite and other observations and its state-of-the-art, expandable infrastructure, it will provide a core capability for the ongoing/long-term component of the NCA and allow for the exploration of climate model metrics/scoring/weighting in regards to their use for more formal multi-model climate projections.

This proposal will contribute critically needed deliverables to the NCA, and specifically, facilitate NASA’s role by contributing key technical capabilities in conjunction with unique and valuable observational resources.