A Bit More About that “F Word” Part Two and a Half: Politics, Prostitution and Statistics

I’ll once again begin by offering advance apologies to the good people at Dimechimes for including them in the same post as FEMA circa Baby Bush.  However, the larger topic of statistics applies here to the sampling methodology used by the Department of Homeland Security Office of the Inspector General in their examination (a term I use loosely in that context) of Katrina Wind-Water claims and to the recent salary survey article contained in Claims Magazine, a claims adjuster trade publication.

As Nowdy pointed out in her post The “F” word hits the road – finds Wall is dead end Street (Part 2 of 3) I have some experience in this area having pulled a statistical sample or two in my career.

First a warning – those without mathematical proclivities may find this post less than stimulating. However I’ll start off with a recent Nassim Taleb interview Russell recently emailed me since he has become a pop culture darling of sorts. While the 10 minute interview is really a post on it’s own Taleb distills the key concept inherent in sampling:

one single observation, OK, can destroy thousands of years of confirmation.”

Taleb, of course has risen to fame using the analogy of the Australian black swan to illustrate that point. Sampling is inherently limited like that, each time we draw a ball from the barrel of balls we learn something about the characteristics of the population of balls inside the barrel such as their relative sizes, color and the like. Projecting the results to the larger, unknown population becomes much trickier. It is there where we also enter the realm of the twisted statistic. More on that later.

So against this backdrop enters the Inspector General for the Department of Homeland Security which office was forced to look at this issue by Congress, specifically then Senator Trent Lott and Representative Gene Taylor:

We are recommending that FEMA (1) require WYOs to document the rationale and methodology for calculating flood and wind damage, and revise the NFIP Adjuster Claims Manual to reflect these requirements; (2) expand the reinspection and operational review process to include a review of and determination of flood and wind damage on the same structure; (3) provide clear and concise guidance for adjusting total loss claims when structures are completely destroyed; (4) begin a dialogue with WYOs, insurance associations, state regulators, and Congress to explore ways to address the perception of conflict of interest when adjusters for flood and homeowner policies represent the same company; (Emphasis Sop)

For those now wondering why I emphasized the “perception of conflict of interest,” it is because the use of the word perception goes to the heart of my heart burn with this Bush Administration sanctioned hatchet job OIG report. The GAO addressed this conflict of interest of course but they didn’t call it “perceived”, rather they labeled it “inherent” which is a huge difference in meaning. The nuance was obviously lost on the inspector General who concluded this on page 12 of the report:

GAO concluded that an inherent conflict of interest exists when a single WYO insurance company is responsible for adjusting both the wind and flood claim on a single property. FEMA must ensure that its internal controls are sufficient to minimize the potential adverse impacts of this conflict on the accuracy of damage determinations and flood claims. GAO recommends that FEMA be able to access wind damage claims information from the WYO insurer in certain circumstances. We agree with GAO’s assessment and our review supports its conclusion on this issue.

So we are left to wonder how this inherent conflict of interest played into the OIG examination. You see, when I audit cash for one of my clients for instance, I understand going in there is a high inherent risk that it will sprout legs and walk off. Cash tends to be like that and that inherent risk associated with it does not differ much between organizations.  Thus when an auditor examines details relative to cash that basic fact must underpin the entire approach. Did the OIG take that approach? Let’s start with what they found:

44 out of 131 cases (34%) included errors that related to cause of damage resulting in some degree of duplication, e.g., flood and homeowners policies paying for the same type of damage (ceiling repair, loss of personal property), only two (1.5%) of these cases clearly identified wind as the preponderant cause of damage, thus resulting in an improper payment by NFIP in the amount of $432,600.

That’s right, an error rate of 34% including 2 that were clear cut example of claims dumping. Now if an experienced auditor instead of a prostitute had conducted the OIG exam we could rightfully expect a conclusion somewhere along the lines of there being systemic problems with internal controls covering claims payments made by the National Flood Insurance Program. Instead Richard Skinner and his employees Gina Smith, Cliff Melby and Raul Adrian take the Roxanne path and associate their hereto for good names with this conclusion:

Based on the review of files in our sample, we did not find material evidence that the NFIP paid for wind damage.

That is an amazing conclusion, especially considering the level of bias in their sample. Insurers by law did not have to share their wind payment data with FEMA. We find the total sample drawn by OIG was 227 NFIP claims files. However they could only test 131 because that was the extent of the response they received to the administrative subpoenas they issued. Given what they found in the 131 files they actually tested it is impossible to conclude anything about the population except that it manifests major problems, especially the two instances where claims dumping was clear cut. Beyond the silly conclusions not supported by their own data, this report was a complete waste of taxpayer dollars spent to prepare it. These problems and their solutions will have to wait a few more months for the next (Obama) administration to tackle.

Next up is the Claims Magazine salary survey that was bought to our attention by the Dimechimes blog in part because of Nowdy’s post Fat Cats and mine Life of a 6 Figure Cat Adjuster. I took the Dimechimes challenge and read the entire article since this had become an area of interest to us after the anti Scruggian related meme about the Rigsby sister’s cashing their “low paying” adjuster jobs in by chipping in with Dickie Scruggs.  We have since found they probably took a pay cut but lest I digress.

I am familiar with salary surveys in the construction industry. I personally find them widely varying in their results which is understandable since these survey’s are response driven. While I did learn several things anecdotally and did get a flavor for compensation issues in the claims adjusting segment of the insurance industry, I found the Claims Magazine survey pretty much useless absent more color from Miss Bramlet @ Claims Magazine.

In the end I think the true statement would be income in Cat Adjusting is a very lumpy affair with huge years like 2005 followed by quiet ones like 2006. I’ve been reliably told that if one is willing to travel and endure tough working conditions the money is still good every year and I believe the salary survey article bears that out.  Numbers I like to see accompanying such surveys would include median salaries, salaries broken down geographically and by size of employer along with an idea of the standard deviation from the average salary. Such “color” adds life to the numbers.

sop