Media-Prominent Scientists Dominate Climate Policy, Rather Than Authors Producing Highest Quality Research
Basil Mahfouz
The 28th United Nations Climate Change Conference (COP28) in Dubai has highlighted the need to embed high-quality scientific insights into global climate policy.
My PhD research at UCL’s Department of Science, Technology, Engineering, and Public Policy, is focused on developing new methods for measuring the capability of policy makers to utilise high-quality scholarly research. Since the start of 2023, I’ve been working closely with supervisors Professor Sir Geoff Mulgan, Professor Licia Capra, and Elsevier’s International Centre for the Study of Research (ICSR) to analyse whether policymakers are systematically using high-quality climate research in their decision-making processes.
Initial findings suggest policymakers may be disproportionately influenced by the opinions of scientists who feature heavily in mainstream media, rather than a broader spectrum of experts.
In the Elsevier SDG mapping initiative, almost 30 million research publications have been categorized as tackling one of the 17 United Nations Sustainable Development Goals.
Utilizing this classification, we identified 331,907 research articles related to SDG 13: Climate Action, published between 2005 and 2021. Approximately 22% of these articles were cited in policy documents indexed in the Overton database.
In the first test, we wanted to check if the articles that were cited in policy also have high academic impact. To do this, we examined a series of bibliometric indicators commonly associated with research excellence, such as citation counts, journal impact factors, and author h-indices. Our hypothesis was that papers with high scores in academic impact indicators would be those most frequently cited in policy.
To test this hypothesis, we employed machine learning methods to analyse the extent to which these academic impact indicators predict a paper’s citation in policy documents. Contrary to our expectations, the results showed that our model was a poor fit, correctly predicting 10% of articles that were cited in policy. This finding suggests that there is no observable relationship between research quality and climate policy impact. Rather, other, non-academic factors may be playing a significant role in determining which papers are being cited in policy.
Despite the model’s poor overall fit, it revealed that media mentions are 22 times more influential than journal metrics, twice as influential as author level metrics, and 30% more influential than article-level metrics at explaining why a paper is cited in policy. The data suggests that policymakers may be relying on the media as a knowledge broker for identifying climate evidence. Indeed, less than 10% of authors account for over 90% of all climate policy citations and 79% of all climate research coverage in the media. Yet these influential authors are not necessarily academic leaders in their fields. Only one-third of these highly cited authors in policy are among the top 20% most academically cited researchers in climate research. Further analysis revealed a moderate positive correlation (r=0.54) between the frequency of an author’s research mentions in the media and their policy citation count. This pattern raises questions about the criteria for policy citations and the role of media visibility in shaping policy-relevant research landscapes.
The current system, with its overreliance on a limited set of researchers, particularly those with high media visibility, may not be serving the global need for high quality scientific insights. The media is not an adequate knowledge broker between the scientific and policy domains. Such an approach not only risks compromising the quality of research influencing policy decisions but also leads to a strong geographical bias. Although publishing slightly fewer papers than China, the United States alone accounts for 24.55% of all policy citations. The disparity in citation ratios across countries, especially between the global north and south is stark, even when controlling for research quality. For instance, while the Netherlands enjoys a high citation ratio of 43.13%, countries like Malaysia and the Russian Federation experience substantially lower ratios, at 6.41% and 7.48%, respectively. This pattern reveals a troubling underrepresentation of research from the Global South, potentially side-lining valuable perspectives and contributions in global climate discourse.
In conclusion, our study highlights the urgent need to revaluate how climate policymakers access scientific research. The data points to a system of over reliance on media to identify key researchers, which risks side-lining crucial perspectives, particularly from underrepresented regions like the Global South, and compromises the quality and comprehensiveness of scientific inputs. In light of the complex, global nature of climate change, it’s imperative to establish more inclusive and equitable approaches, which ensure that high-quality research from across the globe, irrespective of media presence, is recognized and effectively harnessed in climate policy-making.