This is the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition – commonly known as the DSM-5. It is the newest edition of the DSM and was just updated from the 4th edition in 2013.
This manual serves two purposes. First, it provides a sort of dictionary for mental disorders. By that I mean that it takes collections of symptoms and gives them a name. This was one of the first purposes of the DSM. It was handy to make sure that people that were researching different constellations of symptoms were calling those the same thing. Without that common language, it’s very difficult to make progress in research on mental disorders. The second purpose of this document is to provide insurance codes for clinicians so they can get reimbursed by insurance companies for their services. If you are treating someone for a mental disorder, ostensibly they have something. If you want an insurance company to give you money for a treatment, you have to tell the company what the patient has.
You might not think it, but the DSM-5 and the NFL Combine have a lot in common.
You see, the DSM-5 as a resource actually kind of sucks. You don’t have to take my word for that. The Director of the National Institute of Mental Health, the primary government funding body for mental health research, recently made a statement saying that “…patients with mental disorders deserve better [than DSM-5].” You can read the full post, but to summarize, the director is saying that what is stated in the DSM-5 does not hold much water when we subject the assumptions and statements of fact contained in the book to serious, empirical, systematic testing.
In the same vein, the NFL Combine kind of sucks. We drag 300 very large men to Indianapolis every year to collect a large amount of largely useless data. In some cases, like the 40-yard dash, that data is useless because it doesn’t predict anything important that decision makers care about. In other cases, like hand size for quarterbacks, we see a number, can come up with endless theories about what that number means, but really have no idea what to do with that number.
The DSM-5 and the NFL Combine both suck for the same reason – epistemology. Epistemology is the notion of how we know what we think we know. In other words, epistemology is the study of the criteria and factors we use to determine truth. In the case of both the DSM-5 and the NFL Combine, we have decided to determine truth on the basis of authority.
In the case of the DSM-5, the book says there is a particular disorder called, for instance, borderline personality disorder and that disorder has a certain set of symptoms. And the reason the book says that is because a high ranking and highly influential psychiatrist said so. If you look in section three of the DSM-5 (the section on new and emerging trends) you will see a dramatically different picture of personality disorders based on evidence and collected data. You can still get to a disorder called borderline personality disorder but the data paint a much different picture of the disorder and what to do about it than the “official” section of the book that clinicians must use to get reimbursed for their services.
In the case of the NFL Combine, we have decided that a small subset of tests will be the ones used at the Combine. It doesn’t really matter that most everyone recognizes that the tests are largely worthless. In modern times, even NFL teams don’t really use the 40-yard dash to evaluate wide receivers. The correlation between draft position and 40-yard dash time for wide receivers since 2011 is 0.25 for relative draft position and 0.28 for absolute draft position. Statistically significant correlations, but accounting for such a small percentage of the decision (about 5%) that they’re barely worth talking about.
So why do we continue this way? Well, because authority tells us that this is the way things are. Most NFL teams know that the Combine drills are useless, but the NFL itself wants to maintain a sense of authority. And so everyone continues to talk about 3-cone drill times and hand sizes, and Wonderlic scores and all sorts of other useless data that should make no difference to anyone trying to find the best football player. Change is unlikely to happen because that would mean the people that put on the event would have to admit that their publicly stated authority isn’t as correct and proper as they have indicated.
However, the data definitely show that the Combine is a large waste of everyone’s time. Sometimes the wrong data is collected, sometimes the right data is used in the wrong way, and sometimes teams have no idea how to use the data they get. In all cases, this amounts to useless data and incorrect evaluations. And until the authority figures can admit that what we knew 30-40 years ago is different from what we know now, we will continue to get poor outcomes with both mental health treatment and evaluation of NFL players.