OK folks, I’ve written a good bit on weather modeling most of which in one way shape or fashion has its roots in the work of Karen Clark, a woman with a keen financial intellect, that literally pioneered the field. Her story is reminiscent of HAL 9000 and a South Park episode. It is also my considered opinion Ms Clark is one of the good guys in the insurance wars. I mentioned this general topic and its importance with the last State Farm rate hike here in Mississippi using very plain english but no journalist here in Mississippi was up to the challenge of reporting it. I say that because if there is one area in the rate setting process that is completely suspect it would be in the loss assumptions indicated by the models. These concepts are equally applicable in other states besides Mississippi that now face insurance rate hikes such as our neighbor Alabama. Finally in the land where people get it, FLOIR Commish Kevin McCarty busted Allstate using bogus short-term models to calculate a 65% rate hike in Florida in 2008.
Besides Sop, Karen Clark understands the implications. She left the company she founded, AIR Worldwide (most likely forced out IMHO), founded another company and went public with her concerns on how the information produced by those models was being misused. It was no surprise to me then when a reader sent me this link from the National Underwriter:
Catastrophe modeling firms’ hurricane damage predictions overestimated insured losses for a second year, according to a catastrophe prediction consulting firm.
Karen Clark & Company in a report said models designed to project U.S. Atlantic hurricane insured losses for the five-year period ending in 2010 “have significantly overestimated losses for the cumulative 2006 through 2009 seasons.”
“Hurricane activity is very difficult to project because the Earth’s atmosphere is very complex and has many feedback mechanisms,” said the report. “Given all of the uncertainties, near-term projections do not have sufficient credibility to be used for important insurance applications such as product pricing and establishing solvency standards.”
Predictably Karen’s wayward babies were having none of it:
Modeling firms said the reports conclusions were flawed, based on wrong assumptions and misrepresentations.
But clearly Karen knows her stuff and lays out her methodology for all to see:
This year’s Karen Clark report said catastrophe modelers AIR Worldwide, Eqecat and Risk Management Solutions initially projected insured loss levels at least 35 percent above the long-term average for the period 2006 through 2010. AIR lowered its figure to approximately 16 percent in 2007, and Eqecat has made only minor adjustments to its original estimate of loss increases of between 35- and 37 percent. RMS introduced modifications to its model in 2009, but still predicted losses at 25 percent above the long-term average.
Assuming long-term average annual insured hurricane losses of $10 billion per year, these figures translate into cumulative insured losses for 2006 through 2009 of $48.8 billion, $54.5 billion and $54.6 billion, respectively, for the AIR, EQECAT and RMS models, according to the report’s calculations.
Karen Clark said the actual cumulative losses were $13.3 billion, far lower than the model predictions, and only one-third the long-term cumulative average of $40 billion. The 2009 Atlantic hurricane season was below average in the number of named storms, hurricanes and major hurricanes, and was the lowest frequency year since 1997, it was noted.
The report noted that hurricane activity is influenced by many climatological factors, many of which are known, but some unknown, by scientists. It stated that there are complicated feedback mechanisms in the atmosphere that cannot be quantified precisely even by the most sophisticated and powerful climate models.
The report recommended that insurers, reinsurers and regulators evaluate the efficacy of the near-term hurricane models in light of this uncertainty.
Outside of the insurance trade press and writer Michael Lewis Ms Clark never made it into the realm of mass media coverage but it is equally clear she feels the topic of policyholders being raped on their rates due to misuse of loss models is important as her the news page at her website illustrates. Ms Clark also addresses what she calls the “Hurricane Frequency Paradox” and how it relates to modeling:
Some scientists suggest there has been an increase in Atlantic tropical activity, based on growth in the tropical cyclone counts since data was first compiled in the late 19th century. Paradoxically, this apparent increase has not resulted in an increase in hurricane landfalls in the United States. If in fact there are more Atlantic tropical cyclones, then over the past four decades the percentage of storms making landfall has declined to about 60 percent, compared to an average of about 75 percent prior to 1965.
Researchers at the National Oceanic and Atmospheric Administration (NOAA) have now concluded that the increase in annual storm frequency is in large part attributable to improvements in observational technology leading to the increased detection of tropical storms and hurricanes. This is particularly true for short duration storms originating in the Eastern Atlantic, far removed from potential landfall. Prior to the introduction of satellite technology, such storms were dependent upon oceangoing ships for detection.
“According to the most recent NOAA study, the occurrence of short-lived Atlantic tropical storms and hurricanes, those surviving no more than two days, has increased dramatically over time, while the number of longer-duration storms has not,” said Ms. Clark. “If one estimates the number of storms prior to 1970 that were not detected, there appears to actually be a slight decreasing trend in storm frequency. In addition, we have not seen an increasing trend in hurricane losses when historical losses are normalized to current exposure values.”
And this body of work relates to rates in what way? I’ll let Tim Evershed with Global Reinsurance Magazine explain:
The promise of up to 35% reductions in loss estimates that accompanied new earthquake models this year has set alarm bells ringing, with some experts concerned about the ongoing unpredictability facing underwriters.
Both RMS and AIR Worldwide have updated US quake models this year, incorporating the 2008 US Geological Survey (USGS) national seismic hazard maps. According to RMS, the new models are likely to lead to a reduction of 10%-25% in US earthquake insured loss estimates for the average insurer across all lines of business, with more modest changes in loss estimates for commercial business lines and larger reductions for residential lines.
The most significant changes in North America will be in California, where modelled loss estimates will reduce by approximately 5%-15% for most commercial portfolios and 25%-35% for the majority of residential portfolios, RMS says.
Catastrophe models and the reinsurance industry have a chequered history, with heavy criticism of the models following losses such as Hurricane Katrina….
California quake is globally the second-largest pool of natural catastrophe aggregate, following Florida windstorm. Even a small drop in loss estimates could mean significant reductions in rates and the capital necessary to support portfolios.
“You would not expect to see the full impact of the changes immediately, but I think reinsurers will be reducing their prices for California quake, although the quantum is yet to be decided. Primary insurers should expect to free up some capital,”
Guy Carpenter’s managing director, Dickie Whittaker, says.
Of course the fact that weather modeling in general have been used for junk science purposes is a poorly kept secret. Last April, the Wall Street Journal’s Numbers Guy, Carl Bialik ran a column and a blog entry on the subject. I noticed the official party line from Robert Hartwig’s band of shills at the Insurance Information Institute was typical in its intellectual dishonesty. Yes guys, the key is how many will strike the US and Ms Clark says the models are no good for predicting that as the Numbers Guy points out (his related blog entry can be found here):
If analysts did no better than predicting stock prices would equal the average of the last five years, one would hope they’d find a different career — or at least take their work private while they refined their techniques.
That’s the sorry track record of climatologists who each year predict the number of hurricanes that will threaten the Caribbean and Southeastern U.S. before the storm season begins on June 1. Yet their seasonal forecasts continue to garner headlines in the spring as reliably as groundhogs and their shadows.
In early 2005, predictions ranged from 11 to 14 tropical storms — compared with an average of 14 in the prior five years — with seven or eight hurricanes, compared with a five-year average of seven. The storm season instead brought Katrina, Rita and 13 other hurricanes among the 27 named storms.
The forecasts’ flaws were evident before that big miss and have continued since then. The next two years they overshot; last year, at last, they were right in predicting a typical year. This year, most forecasters are calling for below-average activity.
I laughed when I read the article as its publication date was just a month or so after I ripped Rob Young, an academic that is well liked in certain French tree hugging circles and who makes some sense in this article as we continue:
“It’s as if they’re presenting their data in the middle of a study, before they reach their conclusions,” says Robert S. Young, director of the program for the study of developed shorelines at Western Carolina University. “They should keep doing what they’re doing, and they shouldn’t tell anyone about it until they’ve figured it out.”
Yet even as academics, government agencies and private industry crowd into the forecasting arena, they’re bumping up against obstacles that may render accurate forecasting so far ahead of time impossible. Some forecasts are based on past years with similar patterns, but the climatology record doesn’t go back far enough to lend much confidence. And it’s hard to even detect these weather patterns far in advance — even giant patterns that determine the intensity of a season. El Niño, or warming of Pacific Ocean waters, tends to suppress hurricanes; La Niña, unusually cold Pacific waters, tends to increase storm activity. Yet neither of these seasonal effects can be predicted with much reliability before the late spring.
Nassim Taleb covers all these concepts and more in his two books and it is one of Taleb’s favorite statistical tools, the Monte Carlo Simulation, we examine next as I think I detect it in this recent profile of Ms Clark by Zach Phillips of Business Insurance that also gets to the heart of her heartburn with the misuse of Cat modeling:
Karen M. Clark, among Women to Watch that Business Insurance honored in 2006, made a dramatic career shift one year later.
The recognition honored her work with AIR Worldwide Corp., the catastrophe modeling company she founded in 1987. She stepped down in 2007 as chief executive officer and founded Boston-based Karen Clark & Co., a new catastrophe modeling and consulting firm.
Following criticism of catastrophe modeling stemming from Hurricane Katrina in 2005, Ms. Clark met with clients and found two trends she described as “garbage in, gospel out”: Insurers and other companies used inadequate data in the catastrophe models and relied too heavily on the results.
Today, Karen Clark & Co. aims to help companies and insurers improve their exposure data.
She said companies often misunderstand the meaning of modeling results such as 1-in-100-year catastrophe loss estimates. While some may see that figure as the maximum loss for a storm that will not happen for a long time, it really means there is a 1% chance that a storm will occur in a given year that would cause damage equal to or above that figure.
Mr. Clark said her firm is trying to “wean” companies off of using such “point estimates,” which she said are inherently unstable because of a lack of scientific data. Instead, she helps clients develop a range of scenarios to represent their risk.
Next up I’ll finally highlight one of the definitive articles which marries these important concepts not just to rates but the world of high finance and the Cat Bond when we visit with Karen again in Michael Lewis’ excellent piece from August 2007, In Nature’s Casino.