Elite South African runner Caster Semenya at the 2012 London Olympics. (Credit: La Sud Africaine)
Should high testosterone levels disqualify elite female athletes from competing among women?
That will be the question at hand in a Swiss court this month when South African sprinter Caster Semenya faces off against the world governing body for track and field over controversial new rules requiring women with high levels to medically lower them in order to compete.
The rules, established in 2018, hinge on a previous study concluding that elevated testosterone gives women a significant advantage in at least five events.
But a new paper led by CU Boulder suggests that research is “fatally flawed.” The authors, who will appear as expert witnesses in the case, are now calling for a retraction of the original research and asking the International Association of Athletics Federations to reconsider the rule change.
“In almost any other setting of science, errors of this magnitude would lead to a paper being retracted,” said lead author Roger Pielke Jr., director of the Center for Sports Governance at CU Boulder. “And it certainly would not be the basis for broad regulations that have a profound impact on people’s lives.”
Testosterone and competitive advantage
In April 2018, the IAAF announced new regulations requiring certain female athletes with naturally high testosterone levels to take testosterone-lowering hormones if they want to continue to compete in the women’s category for the 400m, the 400m hurdles, the 800m, the 1500m and the one mile.
The rule, which applies to IAAF-sanctioned international competitions, requires that they maintain serum testosterone levels below 5 nanomoles per liter (nmol/L) for at least six months prior to competition. Most females have testosterone levels ranging from 1.12 to 1.79 nmol/L while the normal adult male range is 7.7 – 29.4 nmol/L. About 7 in every 1,000 elite female athletes have high testosterone levels, according to IAAF.
Any agency, in sport or beyond, should be expected to produce science that can withstand scrutiny and which actually supports the justification for proposed regulations. That simply did not happen here."
-Roger Pielke Jr.
The association had attempted to put forth similar regulations in 2011, but that rule was thrown out when the Swiss-based Court of Arbitration for Sport (CAS) – the highest court for international sport – concluded in 2015 that there was a lack of evidence linking high testosterone to “a real competitive advantage” in women.
“While a 10% difference in athletic performance certainly justifies having separate male and female categories, a 1% difference may not justify a separation between athletes in the female category, given the many other relevant variables that also legitimately affect athletic performance,” the CAS panel concluded, calling for more research.
In 2017, the IAAF came back with that research, publishing a paper in the British Journal of Sports Medicine (BJSM) which claimed that elite women runners with the highest testosterone levels performed as much as 3 percent better than those with the lowest levels.
Pielke and co-authors Erik Boye, a professor emeritus of molecular biology at the University of Oslo, and Ross Tucker, a University of Cape Town exercise physiologist, challenge those results.
“We found problematic data throughout the study and consequently, the conclusions can’t be seen as reliable,” Pielke said.
Fatal flaws in data collection
When the three tried to replicate the original findings using data from the study’s authors and publicly available results from four of the races included, they uncovered “significant anomalies and errors.”
For instance, they found performance times that were erroneously duplicated and “phantom times” that did not exist in official IAAF competition results. In addition, some athletes disqualified for doping were included in the study dataset – a fact that could confound the results.
In all, from 17 to 32 percent of the data used in the study was found to be in error.
“The IAAF should retract their paper containing faulty data, and the new rule based on this paper should be removed,” said Boye.
The researchers also note that IAAF researchers themselves conducted the BJSM study and have declined to share the majority of their research data.
“We would not find it appropriate for cigarette companies to provide the scientific bases for the regulation of smoking, or oil companies to provide the scientific bases for regulation of fossil fuels. Sport regulation should be held to the same high standards,” they write.
The IAAF researchers did correct what they characterized as “data capture errors” and re-ran their analysis in a subsequent letter to the journal. But flaws remain, Pielke said.
The IAAF “remains very confident of the legal, scientific, and ethical bases for the regulations,” according to a statement on its website.
Scientific integrity at play
Roger Pielke Jr.
The research will be at issue in February when Pielke and Tucker serve as expert witnesses at the Court of Arbitration for Sport, where Semenya and Athletics South Africa have brought a case against the IAAF.
Under the new regulations, those who decline to medically reduce their testosterone levels must relinquish their right to compete as females.
Semenya, 28, a two-time Olympic champion in the 800 meters, has said the rules stigmatize women who do not conform to perceived notions of femininity and called them “discriminatory, irrational and unjustifiable.”
Originally set to take effect in November, 2018 implementation of the rules has been postponed until after the outcome of the case.
“Fundamentally, the issues that we raise with our paper are about the integrity of science in regulation,” said Pielke. “Any agency, in sport or beyond, should be expected to produce science that can withstand scrutiny and which actually supports the justification for proposed regulations. That simply did not happen here.”
