For the video version, click here.

Did the rate of autism diagnoses increase 400% between 2007 and 2011? No, they did not. But you might not agree with me if you read this article, appearing in the journal Pediatrics.

The paper meant to examine how poverty affects the increase in 3 chronic childhood diseases: asthma, ADHD, and autism. But the interesting results, showing that the poor have borne the brunt of increase in asthma and ADHD diagnoses, is overshadowed by this statement, appearing right in the abstract and several times in the paper itself:

The lifetime prevalence of ASD rose almost 400%.  This is an error.

How this error occurred should be a lesson to all of us who read papers like this to guide our practice.

The study used data from the National Survey of Children’s Health, a telephone-based survey conducted in 2003, 2007, and 2011-2012 that asked parents about a host of their children’s health conditions. It's a reasonable way to get prevalence numbers, and when it comes to those with Asthma, ADHD, and autism spectrum disorder, it’s pretty clear that the prevalence is on the rise:

Autism spectrum disorder is the blue line on the bottom there, and yes, it's going up. How do you get to 400%?

Well, the authors get a 400% increase by comparing the rate of autism in 2003 (0.5%) to 2011 (2.3%).  That’s a four-fold increase. 400%.

But the survey used a different question in 2003 than in 2007 or 2011. 

In 2003, the question asked only about “autism”.

Has a doctor or health professional ever told you that [CHILD] has any of the following conditions]?

Autism?

In the later years this was changed to be much more inclusive, as seen here:

Please tell me if a doctor or other health care provider ever told you that [S.C.] had the condition, even if [he/she] does not have the condition now.

Autism, Asperger's Disorder, pervasive developmental disorder, or other autism spectrum disorder?

In fact, the authors note this problem and state:

“Important methodological differences in how the questions were asked for children with ASD limited examining trends between the 2007 and 2011–2012 administrations.”

But somehow, that 2003 rate – an invalid comparison – apples to the oranges of the later years was used to calculate the 400% increase.  In fact, the authors erroneously assign the 0.5% prevalence of autism to the high-quality 2007 data, when it came from the 2003 data.  In fact, according to the corresponding author, Dr. Amy Houtrow, the rate in 2007 was 1.9%. Using that number, we get an increase in autism prevalence of 20%, a far cry from 400% and much more similar to the increase in the rates of asthma and ADHD that were also reported in the study. 

If anything, this study should remind us to be really careful when we look at statistics in manuscripts and maybe to push for more open data sharing. Or, perhaps even more simply, we should remember that when a number doesn’t seem quite right, it’s usually not. We've reached out to the journal Pediatrics and have been told that the error will be corrected.