How do you know what decision-makers want, and give them what they need, during the pandemic? IPPO’s learning journey
‘We’re demand-led’ is a phrase often used to describe the International Public Policy Observatory’s approach in helping to address the social impacts of COVID-19. IPPO’s Principal Investigator, Joanna Chataway, explores the realities – and challenges – of trying to be led by the needs of policymakers
Getting closer to the needs of our users
The International Public Policy Observatory (IPPO) is designed to bridge evidence and policy so as to mitigate the negative social impacts of COVID-19. In being ‘demand-led’, we aim to be guided not by the instincts and experience of researchers on which topic area to focus on, but by the needs of decision- and policy-makers.
The scale of the demand for COVID-related policy responses, and the conditions of stress the pandemic has created, means evidence-gathering resources must be targeted carefully to ensure they meet policymakers’ needs. We therefore designed IPPO to be demand-led because we know from work done by consortium partners (and many others) that efforts to simply push completed research in the direction of policy- and decision-makers is, by and large, not effective if the aim is to inform their policy and decisions with evidence.
Significant research exists supporting the idea that brokerage and iteration is fundamental to effective knowledge-sharing and to the use of research in policy- and decision-making. A recent review article from Peter Gluckman and colleagues endorses this approach, offering 10 recommendations for researchers seeking to make their research relevant to policy. All of these recommendations were considered in the design of IPPO; the first relates directly to the issue of working with policymakers to identify their demand and then frame the research question around that need.
But what does this mean for the way IPPO is working in practice – and what are we learning about the realities of being demand-led? This diagram outlines the process on which our approach is based:
Note that the diagram does not start with a research output but with policy demand – and then works back to the research via iteration mechanisms. Our ‘virtual roundtable’ events with policymakers, decision-makers and experts are a vital way of generating the conversations that can lead to the second and third parts of the process.
Understanding and constructing demand
A few months into IPPO’s lifespan, however, we have already made changes to the way we work. While the diagram hasn’t changed in its fundamentals, making the process work requires some additional steps and is less neat.
To explain how our approach has developed, let me use the example of the first areas of work for the IPPO team: education and mental health. It is obvious that education was a sector put under huge strain by the virus, with multiple policy decisions needing to be made under tight time constraints.
Early conversations with policymakers confirmed this. Beyond questions about the timing of children’s return to school, there were numerous uncertainties: how to cope with ‘catch-up’, the mental health of children, access to resources, how to plan for the next year, and whether to keep an element of distance learning in the future were all issues raised in our early conversations. Thus, initial conversations laid out a very broad terrain of policy needs and concerns.
Mindful of the need for IPPO to deliver policy-relevant evidence in a short timeframe, instead of jumping straight into a roundtable to consider this very wide range of issues, we first developed and published a topic snapshot and a global policy scan to inform the more detailed discussions ahead. These reviews confirmed the importance of an issue which had emerged in early conversations with policy people and experts alike: the catch-up process had to be viewed in broader terms as an opportunity for children to reset, in light of the very difficult social circumstances that many have had to deal with over the past year.
While we kept the discussion at our first roundtable event open, it was loosely guided both by what we had learned in early bilateral conversations, and by these scans of evidence and policy landscapes. Thus, our thinking about the pressing responses was already being informed by evidence and activity – not simply by the thoughts of policymakers. It is important to acknowledge this because the reality of being demand-led is that defining the questions, and reflecting on what is possible to address with evidence, is very much an iteration from the outset.
Based on the additional insights brought to the roundtable by the range of policy- and decision-makers present, we progressed the issue of balancing academic and wellbeing in summer catch-up plans by commissioning a more policy-focused, evidence-based ‘rapid answer’ (and two supporting blogs), and then, most recently, with a second workshop to further give definition to possible policy interventions.
The importance of dialogue
IPPO’s early work, and the nature of our conversations and roundtable discussions, has certainly made us aware of the importance of dialogue. We have ruled out the idea that we might rely on surveys of decision-makers, or ask for written answers to questions. Conversation and dialogue has allowed us to refine ideas, and to interrogate and iterate between the demand for, and supply of, evidence. It allows us to frame questions and issues, and is a key component in addressing policy needs within the timeframes we are working with, and on the basis of sometimes patchy evidence and rapidly evolving policy agendas.
Put simply, gauging the demands of decision-makers is complex and – in the context of IPPO’s agenda to link COVID-19 research to policy under short timescales and conditions of stress – not one that can be thought of in purist terms. We are not working to a demand model that allows an ideal of how we might generate evidence in the medium and longer terms. Rather, we have found it useful to think in terms of trying to generate ‘effective demand’ that reflects and is based on real-time assessments of the existing landscape of evidence and policy.
Our IPPO process diagram has not fundamentally changed – it was always based on iteration – but the wording in the first semi-circle might be better expressed as: ‘Identifying key policymaker needs and pressing issues, and iterating after an initial review of policy and evidence landscape.’ In future, our work in these early stages of iteration between supply and demand will be facilitated by our recently launched IPPO Living Map, which will allow roundtable participants to see at a glance where the main bodies of evidence relating to social impact reside.
Turning latent demand into effective demand
In economics, the concepts of effective and latent demand are used to differentiate between types of demand, and to point to the kind of resources that are needed to fulfil them. Demand or need may well exist and may even be critical – but without the requisite resources or knowledge of what is on offer, it may still not be satisfied. In a well functioning market underpinned by sufficient economic resources, latent demand becomes effective demand for goods that either exist or can be made. Things happen and needs get met.
In the IPPO context, our supply and demand relationship – our ‘market’ – is not underpinned by money and finance (we are thankful to the ESRC for enabling us to conduct this work without additional money from policy- and decision-makers for the supply of research), but on knowledge flows and a clear articulation of what is effective demand.
However, especially in the context of the pandemic, it is still not a simple endeavour to ask the question: ‘What evidence do decision-makers need to make good decisions?’ Rather, this is a difficult, collective and sometimes messy process that, in the best cases, leads to clarity and a transition from latent to effective demand. In this process, as Geoff Mulgan and Nesta colleagues have pointed out, effective demand is intimately connected to effective supply: the generation of forms of knowledge that are designed to inform policy. This is very much what we are working towards at IPPO.
All of these processes – the iteration between demand for and supply of knowledge, the collective sense of making what is actually needed, how that need can be framed in relation to available evidence, and what must be acknowledged as beyond scope – allows for knowledge to be cohered, organised and presented in appropriate forms. It also, however, means that we have had to move away from the idea of ‘purity’ in our demand-driven approach.
Professor Joanna Chataway is Head of Department for Science, Technology, Engineering and Public Policy (STEaPP) at University College London, and Principal Investigator of IPPO.