Grey Literature citation and inclusion rates in gambling review articles: Opportunities for improvement

Commercial gambling is expanding in many countries worldwide, and researchers and governments are increasingly approaching gambling as a public health issue. Grey literature is a popular avenue for disseminating findings from gambling studies, accounting for over 20% of gambling research publications. As evidence-based policy decisions increasingly rely on systematic and scoping reviews, it is important that grey literature evidence is included in these reviews. To date, two umbrella review has assessed systematic reviews for gambling-related interventions, and using the AMSTAR 2 critical appraisal tool, both found the overall quality of the reviews to be low. One of the AMSTAR 2 criteria is “a comprehensive literature search strategy”, which includes grey literature where relevant. The goal of this study is to assess the extent to which grey literature is cited in gambling related review articles, how often grey literature searches are included in reviews with systematized search strategies, and how grey literature is discussed (or not). Using the multidisciplinary scholarly database Web of Science, a broad keyword search for “gambl*” was performed, limited to documents of type “Review” published from 2016 to 2020. After screening for articles unrelated to gambling, 174 reviews were included. The references section of each article was reviewed and all grey references were catalogued. For articles with a formal search strategy, the proportion of grey items included in the systematic search results was also recorded. To determine if and how grey literature is discussed, the full text was searched for grey literature related terms. Of the 174 included studies, 100 had systematized search strategies and 74 did not. In systematized reviews, grey literature sources were included in just over half of reviews (n=54). Of the 46 systematized reviews that excluded grey literature, only one provided a methodologically sound reason for doing so and only eight acknowledged grey literature exclusion as a limitation of the review, while many did not mention grey literature at all. Across all review types, grey literature represented an average of 9.2% of works cited. Compared to similar reviews in other domains, it appears that grey literature is underutilized in gambling reviews. Efforts to improve grey literature uptake in gambling reviews may be most effective if focused on increasing awareness of grey literature and providing information or training resources to gambling researchers, peer reviewers and journal editors. “One of the problems with systematic reviews is to get it published in a regular gambling journal, you'll get reviewers like me who know the psychology but don't really know... What is there to actually critique in a systematic review? It kind of is what it is. [...] Maybe that's my naive take on something that I do very little in research. Maybe I just don't understand the nuances.” Aaron, Male researcher based in Canada who studies gambling from a psychological perspective Insights and Issues that Challenge and Demonstrate the Role of GL Baxter 68 Gambling is a quickly expanding multinational industry. The global gambling industry grew from $250 billion USD in 2003 to $450 billion in 2013 (The Economist, 2014), to a projected $635 billion by 2022 (Morgan Stanley 2015, as cited in Cassidy, 2020). Commercial gambling is heavily marketed as a legitimate and harmless leisure activity, but has been called a “dangerous commodity” similar to tobacco and alcohol because of the addiction and other social harms is presents (Markham & Young, 2015). Many people who gamble do not experience harm from gambling; clinically diagnosed problem gambling affects around 2-3% of the global population and has even decreased to 0.6% in Canada (Williams et al., 2020; Williams, Volberg, et al., 2012), but governments and gambling researchers are increasingly approaching gambling as a public health issue as problem gambling is only a small part of the picture. Firstly, for each person with problem gambling, an average of six other people are harmed by that person’s gambling, usually their closest family and friends (Goodwin et al., 2017). Secondly, although few people gamble at a level that qualifies as clinical problem gambling, many more gamble at what is called a lowor moderate-risk level and still experience harms from that (Browne et al., 2016; Langham et al., 2016). Thirdly, harms from gambling disproportionately fall onto people in poorer and racialized communities (Abbott et al., 2004), thus the harms from gambling pose an equity issue related to social determinants of health, although the gambling industry’s conflict with public health has been implicated as a “commercial determinant of health” (Ndebele et al., 2020). Finally, there is evidence for a “total consumption model” of gambling harm, meaning there is a strong positive association between the total gambling in a population and harmful gambling in that population (Rossow, 2019; Sulkunen et al., 2019). In light of the gambling industry’s growth over the past 20 years, this means we can expect there has been a concomitant growth in overall gambling-related harms. Governments can use gambling research to inform gambling policies to reduce gambling harm, but there are some limitations to the gambling evidence base. The gambling research funded and conducted in any given country will be shaped by that country’s existing gambling policies. For example, countries that have a history of approaching gambling as a public health issue (e.g., New Zealand) or strong public health research programmes generally (e.g., Nordic countries) produce more public health research on gambling harm prevention, whereas nations like Canada that have a history of viewing gambling as an individual addiction primarily produce research focused on individuals who have gambling problems and how to treat them (Baxter et al., 2019; Nordmyr & Forsman, 2018). Overall, research on the characteristics of individual gamblers is much more common than research on the broader social, cultural, and economic aspects of gambling (Baxter et al., 2019; Hilbrecht & Baxter, 2021). In addition to this, the gambling industry funds a fair amount of the available gambling research, either by sponsoring research directly or through funding “independent” gambling research organizations. Research funding from the gambling industry has risks of conflict of interest in common with funding from tobacco and alcohol industries (P. J. Adams, 2007; Peter J. Adams, 2016). The 2013 report “Fair Game: Producing gambling research” found that gambling research is behind studies of tobacco, alcohol and other drugs in terms of methods used and dealing with conflicts of interest, and funding Insights and Issues that Challenge and Demonstrate the Role of GL Baxter 69 structures tends to focus on “safe” research questions unlikely to lead to further gambling regulation (Cassidy et al., 2013). Two other studies found industry funding had no biasing effect on gambling research, but these studies were themselves funded by the gambling industry and must be approached with caution (Ladouceur et al., 2019; Shaffer et al., 2019). Finally, for governments and policymakers to utilize gambling research to make evidenceinformed decisions, the research must also be summarized and brought to their attention. A common method for summarizing research from multiple research articles on a topic are knowledge syntheses such as systematic reviews and scoping reviews. Unfortunately, the results of the two umbrella reviews evaluating gambling review articles are not promising: one found that primary research articles and systematic reviews on gambling were of low methodological quality in general (McMahon et al., 2019), and the other avoided doing a quality assessment altogether as “most of the studies would have been rated of weak quality and eliminated” (Velasco et al., 2021). The limitations discussed above largely refer to gambling research and reviews published as primary academic literature, but can be addressed in part by gambling grey literature. Gambling has a sizable and unique body of research published as grey literature: a systematic search of gambling grey literature published in Australia, Canada, New Zealand, the United Kingdom and the United States found that grey literature makes up over 26% of gambling research publications (Baxter et al., 2021b), and a thematic analysis of recent gambling research found that studies published as grey literature tended to focus on broader population health and well-being aspects of gambling, whereas studies published as journal articles tended to focus on the individual characteristics of people who gamble (Baxter et al., 2021a). Grey literature has other general benefits, such as being more up-to-date and detailed than what is available in the primary literature, as well as providing a “unique global perspective” (Bonato, 2018, p. 28). The latter is especially important in gambling with local research being constrained by the existing policies of the jurisdiction, as context-specific research from different jurisdictions with different gambling policies is more likely to be published as grey literature. For example, the Australian Productivity Commission’s two inquiry reports on gambling (Productivity Commission, 1999, 2010) are highly regarded and well-cited internationally despite their focus on Australian gambling, as they present highly detailed investigative research and analysis that could not be published in an academic journal or book. Somewhat paradoxically, the academic literature aims to make generalized/universal claims, but in doing so loses attention to specific contexts that may inform policies in other contexts. Grey literature such as the Productivity Commission reports achieves this. Although much of gambling grey research is government-funded and the resulting reports may be consulted