Considering and reporting sex as an experimental variable II: An update on progress in the British Journal of Pharmacology

The escalating concern over the use of only males in biomedical research (e.g., Heidari et al., 2016) prompted the Senior Editorial Board (SEB), under the leadership of the Editor-in-Chief (EiC) of the British Journal of Pharmacology (BJP), to publish an editorial that addressed this issue (Docherty et al., 2019). That article highlighted some sex differences in vulnerability to, and expression of, diverse medical disorders, as well as the efficacy and risk of harmful side effects of treatments in women. Even those handful of examples provided compelling evidence to justify a decision that, in future, manuscripts submitted for publication in the BJP should be based on studies that include both sexes in the experimental design unless there was a clear justification for using only one sex. An exemption from this policy would be when the subject of the research is confined to one sex, such as investigations focused on the uterus or the prostate as examples. We also recognised that there would be a time lag between the publication of our editorial with this new guidance and the subsequent publication of manuscripts that adhered to advice detailed in that article. For this reason, we continued to consider for publication all reports of studies that had been carried out before the publication of our guidance, even if they had used only one sex (either male or female). However, in such instances, authors were expected to include a plausible scientific justification for not using both sexes. Also, when the use of a single sex was accepted by the editors, we asked that the manuscript should include a statement in the discussion, to acknowledge that the research findings might not generalise across both sexes and to consider what the findings might mean for the other sex. Our intervention was timely. Since 2019, there has been an explosion of interest in addressing diversity issues within the biomedical sector. This awareness has included high-profile calls to action in the academic literature across the specialities (e.g., the cardiovascular and neurological fields [Vogel et al., 2021; Waters et al., 2021]) as well as the formal recognition by national governments of the issues, and a commitment to develop policies and strategies to improve the balance in research studies (e.g., in the UK, https://www.gov.uk/government/consultations/womens-health-strategy-call-for-evidence/womens-health-strategy-call-for-evidence). Three years later, we now wish to determine whether or not our interventions have had any impact. This current editorial reports the findings from an audit of recent articles published in the BJP, and was carried out to determine whether or not the introduction of our 2019 guidance, coupled with a number of initiatives to raise the profile of the issues, has led to any improvement in this matter. The primary objective was to ascertain if the proportion of publications reporting studies that used both sexes had increased since 2019 and, for those that used only one sex, whether (and, if so, how) authors justified that decision. From the perspective of the BJP, two key milestones contributed to the effort to persuade researchers to study both sexes in their experiments. The first was in 2016, when the Sex and Gender Equity in Research (SAGER) guidelines (Heidari et al., 2016) and a policy statement from the National Institutes of Health in the United States (NIH, 2016) were published. The latter explained why, in the future, the research funded by the NIH should include both males and females in the experimental design. The results of our audit, which are presented and discussed in this editorial, start from BJP publications in 2014 in order to have some information on the ‘baseline situation’ before the NIH activated their abovementioned policy, in addition to the impact of our own interventions in 2019. We realised that there would be a time lag between the initiation of a policy and its percolation into the literature, but some increase in the use of both sexes might be expected, if only because of a heightened awareness of the problem. After the announcement of the new NIH policy in 2016, the next milestone was the publication of the BJP editorial in November 2019. With the anticipation of a gradual change in scientific mindset, we expected the number of publications, between 2020 and 2022, that reported the use of both sexes, would show an upward trajectory. This is not least because referees and editors of the BJP would examine authors' compliance with the journal's policy on the incorporation of both sexes into the experimental design, as a factor to consider when deciding to recommend a manuscript for publication. The audit sampled publications from all volumes of the BJP published between January 2014 and December 2022. Two issues were sampled as a matter of routine from each volume (calendar year); these were issue 4 (i.e., February) and issue 20 (i.e., October). Because themed issues comprising mainly reviews sometimes appeared in issues 4 and 20, with few original research reports therein, it was necessary on occasion to include additional issues in each volume (3 and 11 and, subsequently, consecutive issues), to ensure that the audit included at least 20 papers per volume (year). Studies using fish, embryos, and haploid cells (oocytes) were excluded because identifying their sex is not straightforward. For the same reason, immortal cell lines were also excluded because, with few exceptions, the sex of the donor is not documented. Apart from these exclusions, all original research papers that qualified for the audit in each issue were included in our sampling process to avoid any risk of sampling bias. These articles reported experiments regardless of whether they had used animals of any species (including humans), tissue samples or primary cultures, or if experiments were conducted in vivo, ex vivo, or in vitro. Consequently, the number of papers audited in different issues varied slightly and ranged from 20 to 28 per volume (year), giving a total of 168 in all. Publications that reported the use of the same sex(es) for all experiments in the series reported in the manuscript were given a single score in the audit. However, some studies involved the use of different sexes in different experiments in a series. In such cases, the use of males and/or females and/or both was scored for each individual experiment within each publication. The main point of interest was to determine if there was any increase in the proportion of studies that had used both sexes. However, that question gave rise to several others that were equally important. All the points that were included in the audit are listed in full in Table 1. As is evident from the bar chart in Figure 1, there has been little change in the exclusive use of males (>60%) since 2014. A promising decline between 2018 and 2019 was reversed in the following two years and so the proportion of studies in 2022 (21%) that used both sexes is similar to that in 2014 (19%). Despite increasing awareness of the need to study both sexes, only two papers (one each in 2020 and 2021) discussed the limitations of using only one sex or implied that the findings might not generalise from one sex to the other. This was despite some authors including text in the discussion that identified other limitations of their research. A surprisingly large proportion of studies (22%) did not specify the sex used in the experiments, even though authors are required to sign a ‘Declaration of compliance’ with the ARRIVE 2.0 guidelines (Percie du Sert et al., 2020) and the BJP Instructions for Authors, both of which require this information. This omission even applied to studies of humans and other large mammals (e.g., pigs). Across all years, a large majority of studies used only one sex without providing any justification for that decision (Figure 2). However, there is an encouraging increase in the proportion of studies that now justify the use of only one sex: from 0% (2014–2016) to nearly 30% in 2022. The nature of the justifications differed considerably. Some authors opted to use the sex that expresses the higher incidence of the disorder in humans: for example, eating disorders (females), resistance to metabolic syndrome with high fat diet (females), aggression (males), lower pain threshold (females) or where a gene linkage was traced to the Y chromosome (males). The most common justification for using only one sex (males) was the higher variability of female response. Given the well-documented evidence for progesterone metabolites as allosteric modulators of the GABAA receptor, for example (Bäckström et al., 2022), it would be surprising if there were not slow cyclical fluctuations in the physiology and behaviours of females. However, a growing number of studies challenge this suggestion, with several assessments/studies providing evidence that increased variability of females is not a justifiable reason for studying exclusively males, a view supported by recent meta-analyses (Becker et al., 2016; Kaluve et al., 2022). Whether or not the variability of male and female rodents is similar across all fields of preclinical research remains to be seen, but it is worth bearing in mind that studies of such questions, even in humans, are neither straightforward nor conclusive (e.g., Davis et al., 1999). Regarding studies that used both sexes, it was striking that this decision was rarely considered as a relevant factor in the experimental design. Only seven publications specified that ‘sex’ was considered for randomisation of the experimental units to different treatments (Figure 3). By contrast, 14 studies did not randomise (or counterbalance) sex across the different treatments and 18 did not mention this point at all. In some studies (even of humans), the numbers of the two sexes were not balanced at the start of the experiment. For example, in one study, the ratio of males to females was 4:1. In such cases, neither randomisation of the samples from both sexes to different experimental treatments nor meaningful interrogation of the data for sex differences would have been feasible. One study used both sexes, but the experiments were carried out independently (to avoid ‘confounds’). Yet, a sex difference in the response of interest was reported, despite the lack of randomisation (or counterbalancing) of the study. In respect of statistical analysis of the data, only 9 of the 40 studies that had used both sexes included ‘sex’ as a factor in the statistical analysis of which 5 reported a sex difference as a finding. However, a common feature of studies of the remainder was to pool the data from both sexes without first validating that step: That is, there was no primary factorial analysis to test whether there was an interaction between the factor, sex, and others in the design. Yet, only if it is confirmed that the effect of the experimental intervention does not differ in males and females would such pooling of the data produce a safe conclusion. Finally, only four papers specified the sex of the subject in the title of the publication. Many more papers (36) specified the sex of the subjects in the abstract, but this information did not increase during the period covered by the audit (Figure 4). Although some authors included text in the discussion dealing with limitations of the study, only two papers included the use of only one sex or acknowledged that the findings might not generalise to the other sex (both published after the first BJP editorial on this topic). Overall, our audit indicates that progress in persuading researchers to study both sexes has been disappointing. Moreover, it is evident that manuscripts are being accepted for publication in the BJP that do not comply with our recommendation that both sexes should be integral to the study design, unless there is a clear justification for not doing so. It would not be desirable to impose a strict rule that papers will not be accepted for publication unless they are compliant with our guidelines on this point. This is not least because the animals have been used for research already and it would be unethical do deny dissemination of the findings. Also, the audit makes it clear that unless there is a profound change in research practice, then only a small handful of papers each year would qualify for publication in the BJP. What is needed for the BJP, and other journals, is to find a way to convince researchers that the study of both sexes is fundamentally important for scientific discovery and the equitable development of effective therapies: Women's health simply cannot be marginalised. But, as our audit shows, despite the increased awareness of sex bias in biomedical research, more needs to be done to convince researchers to act on this awareness. Perhaps the reluctance to adopt practices that are essential for delivering inclusive research relates to a perception of economic pressures. However, as highlighted in the previous editorial (Docherty et al., 2019), it is essential to correct the misconception that using both sexes inevitably increases the total number of animals needed for the experiments. As was explained in our 2019 article, an appropriately powered study that includes sex as an experimental factor can use the same number of animals, regardless of whether only one or both sexes are used. The announcement, in September 2022, that the United Kingdom Research and Innovation (UKRI, 2022) now requires the research they fund to study both sexes, aligning their policy with that of the NIH (NIH, 2022), will surely be helpful. It will be interesting to repeat the audit in 4 years' time, to confirm whether or not this policy has had any effect on manuscripts submitted to the BJP, or if researchers continue to assert that their experiments qualify for an exemption. Meanwhile, referees and editors are well placed to maximise compliance, within the constraints of the experiments that were carried out. For instance, if only one sex were used, then they can ensure that (1) the justification has scientific merit; (2) the title and abstract specify the sex of the subjects that were used; and (3) the limitations of using only one sex are discussed fully. The authors declare no conflicts of interest.