Home
Summaries of Ozone Seasons
Forecasting Procedures
Forecasting Tools
Forecasting Performance
Spare The Air Program
Back to STI Home Page
|
Forecasting Performance
The Spare The Air program required that forecasters produce accurate ozone forecasts to help staff at the Sacramento Metropolitan Air Quality Management District decide whether or not to issue a Spare The Air advisory. Forecasts were issued for the maximum ozone concentration in the four-district (Yolo-Solano, Placer, Sacramento, and El Dorado) region and also for each individual district. Meteorologists issued forecasts for the current- and next-day’s peak ozone concentrations.
Two fundamental questions were asked to help evaluate forecast performance and accuracy:
- How large were the differences between the forecasted values and the observed values?
- Did the forecast accurately predict days above or below the ozone standard?
|
Question 1 answer: The differences between the forecasted and observed maximum ozone values were computed for each forecast period (current day and next day) for all days. The resulting forecast performance charts show the percent of forecasts that were within 0.01 and 0.02 ppm of the observed values. During the 2001 ozone season, 95% of the next-day forecasts were within 0.02 ppm of the observed maximum.
|
Note: Computing forecast accuracy and other statistics can be problematic if the information used to verify the observed values does not accurately capture the phenomena that are predicted. For example, if the existing monitoring network does not measure the "cloud" of highest ozone, the forecast may not verify. Thus, no one statistic can fully reflect the performance of a program so many verification statistics are computed to evaluate completely the quality of the forecast program. This situation must be considered when judging ozone forecast accuracy.
|
Several conclusions can be gleaned from these charts:
 |
Over time, accuracy is generally improving for both the current- and next-day forecasts. This positive trend is very good given that the forecasting criteria changed from the 1-hr ozone standard (in 1996-1998) to the 8-hr standard (in 1999- 2001).
|
|
Accuracy has generally improved by about 1 to 2% per year, which corresponds to improved forecasts for an additional 2 to 3 days each year.
|
|
Question 2 answer: Performance statistics are shown in the table below for the next-day forecast.
|
.
|
1996
|
1997
|
1998
|
1999
|
2000 |
2001 |
|
Ozone standard
|
1-hr
|
1-hr
|
1-hr
|
8-hr
|
8-hr |
8-hr |
|
Accuracy (%)
|
85
|
89
|
94
|
82
|
86 |
82 |
|
False Alarms (%)
|
32
|
58
|
35
|
40
|
39 |
38 |
|
Probability of Detection (%)
|
70
|
67
|
76
|
64
|
81 |
72 |
| Accuracy |
The percent of forecasts that correctly predicted the ozone levels to be below or at/above the ozone standards. |
|
False alarm rate
|
The percent of times a forecast of high ozone did not actually occur.
|
|
Probability of detection
|
Ability to predict high ozone events (i.e., the percentage of forecasted high ozone events that actually occurred).
|
The table shows that
|
In 2001, 82% of the forecasts were correct.
|
|
False alarms were more frequent than we would like, but the percent of false alarms has continually decreased over the past three years. In 2001, 38% of high ozone forecasts did not actually occur.
|
|
The probability of detecting high ozone events has generally improved over the past six years. In 2001, 72% of high ozone forecasts did occur.
|
|
|
