Researchers at Ohio State University refuse further comment on CrossFit study amidst allegations of inaccurate data from the study’s own coordinator.
A few weeks ago, I learned that researchers affiliated with the American College of Sports Medicine (ACSM) had published a study on CrossFit’s efficacy in the National Strength and Conditioning Association’s Journal of Strength and Conditioning Research.
The study—CrossFit-Based High-Intensity Power Training Improves Maximal Aerobic Fitness and Body Composition—was conducted by Michael Smith, Ph.D., a then-Ph.D. candidate at Ohio State University working under Steven Devor, Ph.D. and Fellow of the ACSM.
Much of the study struck me as odd. Curious, I turned to CrossFit’s chief scientist, Dr. Jeff Glassman, who wrote a formal, comprehensive response to the study. The response, which makes numerous claims about the validity and rigor of Dr. Smith’s paper, focuses heavily on this particular section:
“Out of the original 54 participants, a total of 43 (23 males, 20 females) fully completed the training program and returned for follow up testing. Of the 11 subjects who dropped out of the training program, two cited time concerns with the remaining nine subjects (16% of total recruited subjects) citing overuse or injury for failing to complete the program and finish follow up testing.”
What is “overuse or injury”? The study does not define what it means by the term “overuse.” The study also does not detail what specific cases of “overuse or injury” the subjects cited, what caused them, whether the cases were pre-existing conditions, or how long the subjects experienced “overuse or injury.”
Furthermore, the study was “blind,” meaning the researchers in the lab were only able to identify participants by a single number. If the 11 subjects who failed to show up for the test-out were de-identified in this way (and obviously not present at the Ohio State lab), how could Dr. Smith collect any data on the reason for their absence?
Chelsea Rankin, a member of CrossFit 614, volunteered to be the study coordinator for Dr. Smith. During our conversation, I asked Rankin how Dr. Smith could have gathered data on why the 11 didn’t show up to the lab.
Rankin gave me her own opinion on Dr. Smith’s work: “I did all the data collection for the study, and I know every person who didn’t re-test. It was easy to figure out they weren’t injured. This data is inaccurate. Those individuals were not injured, and that wasn’t the reason they didn't test out. To me this questions the validity of the research.”
Luckily, the corresponding author, Dr. Devor, consented to a recorded phone interview, the full transcript of which is available here. More problems became apparent during our conversation. When I asked him about the collection of data from the 11 participants who did not re-test, Dr. Devor did not seem confident in answering.
At the end of our interview, Dr. Devor suggested that I speak with Dr. Smith, whom he insisted would be able to answer all my questions. He even offered to help put me in touch with him. To my surprise, I received an email from Dr. Devor two days later. It contained the following line: “I have spoken with Dr. Smith at Gonzaga University. We will have no further comment on our Journal of Strength and Conditioning Research (JSCR) CrossFit publication.”
In two days, Dr. Devor had gone from conceding that I had a legitimate question and assuring me that he would help answer it to defending his apparently fraudulent data solely on the merit and authority of the journal in which it was published.