UrbanStat publishes its Wildfire Risk Map before each fire season and compares it with what burned afterwards. The map had rated as High or worse 90.7% of the area burned by the most destructive fires of 2019, over 85% of the area burned in 2020, and over 95% of the area burned by the 2021 Dixie Fire. About 35% of California carries that rating.
Against the US Forestry map
Season by season
| Test | What was measured | Result | Source |
|---|---|---|---|
| Back-test | Decades of historical wildfires | Accuracy over 80% in any given year | Read |
| 2019 season | First blind test. Areas burned by the most destructive wildfires | 90.7% rated High or worse before the season | Read |
| 2020 season | All burned areas, a record year of over 4 million acres | Over 85% rated High or worse | Read |
| 2020 season | Largest 10 wildfires | Average score between Very High and Extreme | Read |
| 2020 season | August Complex Fire, more than a million acres | Over 90% rated High or worse, over a year earlier | Read |
| 2021 season | Dixie Fire, burned areas | Over 95% rated High or worse in January 2021 | Read |
| 2021 season | Dixie Fire, affected structures in Greenville | About 80% rated High or worse | Read |
How the test works
- The map is released before the wildfire season. After the season it is compared with fire perimeter data.
- “High or worse” means the three highest levels: High, Very High and Extreme.
- About 35% of California is rated High or worse. A result of 85% or more therefore means burned areas fell overwhelmingly inside roughly a third of the state.
- The map is updated every year after the season ends, between January and April, so the model learns from the latest data.
- Over the last 20 years the model performed 50% better than the US Forestry model. In Arizona, Nevada and Washington it doubled that model’s performance.
Common questions
How is wildfire model accuracy measured?
By the share of area that actually burned which the map, published beforehand, had rated High or worse. It is a blind test: the season had not happened when the map was released.
Why compare against the US Forestry map?
It is the public wildfire hazard map many insurers rely on, so it is the natural baseline. In Nevada and Arizona it had rated about 42% of the areas that burned as High or worse; UrbanStat’s model had rated 84% and 81%.
Does a high hit rate mean the map rates everything as high risk?
No. About 35% of California is rated High or worse. The burned areas were concentrated in that portion.