Promising Test Results for SPoRT Machine Learning Stream Height Model

Written by Ben Houser
Jul 23, 2020

Currently, forecasters who try to predict the risk of flooding have a limited toolbox to work with. Forecasters at National Weather Service Weather Forecast Offices (WFOs) generally rely on NOAA’s River Forecast Centers (RFCs) for stream height forecasts. However, when RFCs produce stream height forecasts, forecasted precipitation is typically accounted for the first one or two days of the forecast. This means that if a flooding event is more than two days out, the forecast will not pick it up. To solve this problem, SPoRT is using machine learning to develop a powerful new stream height product that will produce five day forecasts based on precipitation, soil moisture, and observed stream height data.

The machine learning project, led by Dr. Andrew White of UAH and SPoRT and Kris White of NOAA and SPoRT, is one of the first of its kind in the stream height forecasting field. The SPoRT product produces similar graphics to RFC forecasts, displaying the stream height prediction as well as precipitation forecasts. The machine learning model’s graphics are easy to understand, and can be used by forecasters in a wide range of applications.

Image 1

Stream height forecast for Flint River in Brownsboro, Alabama from 4 February, 2020. The blue and red bars represent two different forecasts for precipitation, and the blue and red lines represent the machine learning model’s forecast of stream height based on their respective precipitation forecast.

In order to produce accurate predictions five days in advance, SPoRT’s machine learning product combines data and forecasts of rainfall, observed stream height, and soil moisture data from the SPoRT-LIS product. The machine learning is powered by a Long Short-term Memory (LSTM) network, which is fed historical data on rainfall, observed stream height, and soil moisture and then determines how useful each dataset is to predict future stream height. After this training dataset is analyzed, the LSTM network will understand how soil moisture, observed stream height, and precipitation values impact future stream heights. The network can now make accurate predictions of future stream height from current observed soil moisture and stream height values and forecasted precipitation. The model’s accuracy is largely dependent on the accuracy of precipitation forecasts, so poor precipitation forecasts can lead to poor stream height forecasts. In this application, LSTM networks are especially powerful because they are capable of learning both the long-term and short-term importance of data. Since forecasters desire long term flooding predictions, especially when heavy rainfall is forecasted confidently, the LSTM network’s ability to weigh data based on its importance over longer periods of time is especially useful.

In order to test the effectiveness of the new machine learning forecasting method, SPoRT has partnered with a number of National Weather Service offices and the Lower Mississippi River Forecast Center. Testing partners were given information on the project’s background, and, from 1 February to 30 April 2020, were given access to training and forecasts from the LSTM stream height model. At the Flint River in Brownsboro, AL, the LSTM forecast did well. The forecasts’s probability of detection for predicted periods of twelve hours or less was fairly high, and the LSTM forecasts showed improvement over the River Forecast Center’s forecasts in many cases. Despite lower probability of detection towards the end of the testing period, surveys taken by testing partners suggest that situation awareness and flood warning lead time improved. The LSTM’s accuracy value was also high, but this is partly due to the relatively low amount of flooding that occurred during the testing period. The accuracy value represents the total amount of correct predictions divided by the total number of predictions; predictions of no flooding are included, so accurately predicting little flooding can result in a high accuracy value.

Image 2

SPoRT LSTM stream height product test results for the Flint River in Brownsboro, AL. Dotted purple lines represent the River Forecast Center’s predictions, and other colors represent configurations of SPoRT’s LSTM product. Probability of detection is the model’s probability of detecting a flooding event, and accuracy is the number of correct predictions, including predictions of both flooding and normal conditions, divided by the number of total predictions.

Other test sites also exhibited promising results. LSTM forecasts of Big Wills Creek in Fort Payne, AL, exhibited a root-mean-square error of under one foot up to 84 forecast hours. At five days, the root-mean-square error was only about 1.2 feet. A small error value is good, and indicates that the model’s forecasts were close to the observed values.

Image 3

Mean bias and root-mean-square bias statistics for SPoRT LSTM stream height forecasts of Big Wills Creek.

Forecasts of Big Wills Creek stream height are not provided by an RFC, so the LSTM model is especially useful. According to testing partner feedback, feeding historical data through a machine learning model will be much quicker and simpler than configuring and calibrating a hydrological model for a new gauge. The machine learning model can be used on any river or stream with a gauged basin, allowing for the production of stream height forecasts that RFCs may not provide. In fact, the Arkansas-Red Basin RFC has recently requested machine learning forecasts for streams in their region, which will be included in later rounds of testing and evaluation. Along with the improved forecasts over longer periods, simplicity and usability make SPoRT’s LSTM model uniquely powerful.

So far, the model has proved itself effective, and its increased forecast time makes it potentially lifesaving in dangerous flooding situations. Early testing indicates that the LSTM product can provide forecasters and officials a longer period to prepare for dangerous floods, which could help minimize the damage from severe flooding.

Back to Homepage

Technical Contact: Dr. Emily Berndt (emily.b.berndt@nasa.gov)
Responsible Official: Dr. Gary Jedlovec (gary.jedlovec@nasa.gov)
Page Curator: Paul Meyer (paul.meyer@nasa.gov)
Support     Disclaimer