Lies, Damn Lies and Internet Headlines

A couple of weeks ago there was a breathless news cycle on how Tesla did on the most recent Insurance Institute for Highway Safety testing.  Based on the headlines, you might think the cars are rolling coffins, so, curious, I decided to do a bit of digging.  The headlines spawned from a recently completed test of six large cars. Since I encourage folks to do their own research, the original IIHS report can be found here and Model S specific detail can be found here.

While Tesla did not earn a “Top Pick”, based on much of the news coverage, you might think it completely failed (it did not), finished last for its category (it did not) and left a smoking crater at IIHS’ test facilities.  The Model S did not earn a “Top Pick” because it scored an “acceptable” for the small overlap front test (the second highest rating and ahead of “marginal” and “poor”) as did two of the other six cars tested.  It scored “Good” in the remaining four categories.  The results are what they are.  I have full faith that Tesla will continue to make running changes to improve the real-world crash safety of the car.

So, about that “real world” thing. There is a saying that a map is not the terrain and I am going to go out on a limb and extend that and say that a test is not reality.  Even the very best test is going to be an imperfect predictor of real-word results, and some tests are very far from that ideal  (I am looking at you, SAT).  I am not smart enough to test you where the Small Offset Front test falls on that spectrum, but I do believe the value of lab testing has its limits. If you are looking at car safety, or perhaps writing an article on car safety, the very same IIHS website also some some other interesting data to help round out your perspective..

If you go to Topics > Insurance Loss Information > Insurance Losses by Make and Model, you can pull up the actual historical loss data for cars–basically what happened in the real world.  If you want to look up the Model S and its peers, set the range to “2013-15”, the size to “Large” and the type to “Luxury Cars”.

The site explains the rating you see the following way:

All results are stated in relative terms, with 100 representing the average for all vehicles under a given coverage type. For example, a result of 122 is 22 percent worse than average, and 96 is 4 percent better than average. The colors also indicate how each model compares with all other vehicles.

I pulled the the data on the Model S and friends an sorted it on two categories.  The first was “Personal Injury Protection” which is defined as “[c]overs injuries to you and your passengers, regardless of fault.”

Screen Shot 2017-07-23 at 8.51.30 PM
(source: IIHS.org)

As the IIHS data shows, the Tesla Model S 2WD has the highest rating, was ~60% better than average and ~15% better than the next car. As I write this, the site reports there was not enough data on the “D” versions for a rating, but given they have essentially the same body, I would not expect a wildly different result.

The second sort I did was on “Medical Payment”, defined as “[c]overs injuries to you and your passengers if you are at fault”:

Screen Shot 2017-07-23 at 8.52.23 PM
(source: IIHS.org)

Again, the Model S comes out on top with a score more than 50% better than average.  So, may point is this, if you are going to look at the test data (what we think will happen), I think you also should also look at the real-word results (what actually happened) get a truly accurate picture.

One final thought.  as autonomous driving becomes reality, I really think all these testing protocols (NHTSA, IIHS, Euro NCAP, etc) need to be revisited to take the quality and capability of those systems into account.  The NHTSA investigation into the accident in Florida showed Tesla accident rates dropped 40% for cars with AutoPilot.  I would think, if your car is smart enough to avoid an accident in the first place, that should count for something when you are making Top Picks for safety.

 

 

Leave a Reply