On Race, ADHD, and the Frustration of Missing Cross-tabs
/This week in 150 Seconds, Attention Deficit Hyperactivity Disorder and Racial Disparities. Spicy stuff, but like an all-you-can-eat restaurant that doesn’t serve dessert, this study may have promised more than it delivered.
The anticipation...
Appearing in the Journal Pediatrics, researchers wanted to answer what seems like a straightforward question.
Many studies have shown that more white kids are diagnosed with ADHD than black kids. Is this because black kids are underdiagnosed? Or are white kids over-diagnosed?
There could be actual differences by race as well, btw.
How would you conduct such a study? To me, it doesn’t seem too hard. Take a big group of kids and perform some standardized test for ADHD on them. Then see how many who had ADHD were actually diagnosed with ADHD. Stratify by race, compare, and you’re done.
And the data the researchers looked at basically had all of these elements. Big group of children? Check – 4,297 from a variety of locations, followed in grades 5, 7, and 10. Standardized ADHD test? More or less – they used a survey that asked a series of questions that mirror the diagnostic criteria in the DSM-V.
Diagnoses of ADHD? Check – at least according to parental self-report.
Now I’ll take the data they provide and we’ll see where things go off the rails.
The overall percentage of likely ADHD, defined by their standardized test, was 7% in white kids, 12% in black kids. So far so good. The overall diagnosis rate was 16% in white kids, 9% in black kids. But… wait… that’s the overall rate. What about the diagnosis rate in the kids who had likely ADHD based on the questionnaire? Where’s that data?
It’s never given. And it’s super important, because it could change the entire conclusion of the study.
Take a look at this Venn diagram. In the best case scenario, everyone who had likely ADHD got diagnosed with ADHD and you have something that looks like this:
We owe a great debt of gratitude to John Venn.
To my eye, those results would suggest that we're doing pretty well diagnosing black children, and definitely overdiagnosing white children.
Of course, the data could also look like this:
Uh... guys, we are REALLY bad at diagnosing ADHD.
In which case we're missing the boat entirely on everyone. Obviously, that second picture is unlikely. But the point is, without the information about overlap we are totally in the dark.
To me this is the only way to figure out who is getting over- or under-diagnosed. The authors argue that, since medication use was lower in black kids, under-diagnosis is happening in that population. But they also state “…we are unable to address this question of overdiagnosis and overtreatment”.
Compare this to the quote at the top of the post. Jeez.
Actually, I think they can address the question. The data is there.
But look, this is an important issue, and to be fair it’s more difficult to assess than I’m making it sound. Recall bias in addition to any number of unmeasured factors could influence whether parents report that their kid was diagnosed with ADHD. But telling us the percent of white kids and percent of black kids diagnosed with ADHD, without comparing it to some standardized screening method doesn't actually tell us much at all.
There's more to this story, it would seem. And when we hear about, you'll hear about in 150 seconds.