By Senior Manager, Alexandra Ellinson
The Politics of Evidence: From evidence-based policy to the good governance of evidence, Justin Parkhurst, Routledge Studies in Governance and Public Policy, 2017.
As evaluators, we want to generate credible evidence that is relevant and useful for informing public policy decisions that improve social outcomes. So, it’s unsurprising that Justin Parkhurst’s recently released book—or more specifically, it’s subtitle, ‘From evidence-based policy to the good governance of evidence’—caught my eye.
This book is a helpful reminder of the imperative that evidence-based policy making addresses the reality of politics head-on; and that this can (and should) be done without giving up on rigorous research. As Associate Professor at the London School of Economics and Political Science, Parkhurst shares in the concerns of both those who champion evidence in the face of the politicisation of science, and those who critique the de-politicisation of policy making that can occur when social values are obscured through the acceptance or promotion of only limited evidence sources and methods. In what follows, I briefly outline the key points and conceptual moves that Parkhurst makes in this book, and highlight what I take to be most relevant to policy makers and evaluators alike: his argument that there is a need for advisory systems that normatively embed the good use of evidence in policy making.
Part I opens with an exploration of what is meant by the oft-touted phrase, ‘evidence-based policy’. Parkhurst clearly unpacks the usefulness and limitations of framing policy making simply around the aim of ‘doing what works’. Starting from the premise that evidence matters a great deal for good policy, Parkhurst gives examples of what goes wrong when there is a lack of information or poorly used evidence e.g. the public health advice that babies should sleep on their fronts, which continued for decades despite mounting data about the dangers of this practice. Yet he also acknowledges the limits of technocratic cries for ‘more evidence!’ to inform policy decisions that are essentially principle or rights-based e.g. access to reproductive health choices or same-sex marriage provisions.
Parkhurst goes on to discuss the value of evidence from well-designed research methods with nous, and displays an appreciation of methodological pluralism when it comes to appropriately understanding social issues. This includes an important, but by no means preeminent, place for randomised controlled trials. He also gives a quick nod to realist evaluation by challenging readers to think about more than just ‘what works’, but also consider what works ‘for whom’ and ‘where’. While devotees of realist approaches might take issue with how he employs the term ‘mechanism’ in this discussion (given its very precise meaning in the field), they are likely to endorse the spirit of his argument.
Interestingly, Parkhurst doesn’t discuss or appear to make a distinction between evidence-based and evidence-informed policy: he primarily uses the first term, although at times the latter. While this seems unusual insofar as evidence-informed policy is often put forward as a more pragmatic and politically tuned-in alternative, I think this is also a smart move on his part. It works to avoid definitional debates and micro-arguments around the degree of political influence before something is described as ‘based in’ or ‘informed by’ evidence, but it also allows him to locate his concerns as part of a bigger picture that applies across cases.
Part II contains an interesting exploration of the role and functioning of various forms of bias that impact not only on what and how evidence is used by policy makers, but also the evidence that is funded and generated by researchers in the first place. Parkhurst discusses overt biases in pursuit of political interests as well as the subtle politics of cognitive biases. Perhaps most useful is a distinction that Parkhurst makes between ‘technical biases’ in the use of evidence and ‘issue biases’ in how evidence is deployed to inform political debates. In relation to technical biases, Parkhurst serves readers well by including both the invalid use of individual methods (i.e. poor scientific practice) and the failure to appropriately include relevant information from multiple sources. In relation to issue biases, he thoughtfully highlights questions around political legitimacy and representative politics in relation to how evidence is deployed to shift debates towards or away from issues/ areas of inquiry, often in a non-transparent way.
On an initial reading, this discussion about biases in Part II is interesting but not obviously essential in contributing to Parkhurst’s argument. On a closer reading, however, it becomes clear that the inherent risk of biases—and hence the need to develop a systematic response to mitigate these dynamics—provides the rationale for why, in Part III, Parkhurst advocates for a governance approach to improving how evidence is used in policy making.
Part III is the most intriguing part of this book. Although it feels somewhat abstract and leaves the reader in want of more concrete examples, I suspect this is an artefact of his principles-based approach to institutional change—one that favours ‘guided evolutionary incrementalism’ rather than programmatic planning. The strength of the concluding chapters is how they challenge policy makers and evaluators to think critically and in new ways about how institutions, within and outside of governments, operate systemically to shape the evidence that is gathered and in turn how it is used in decision making.
Parkhurst starts by looking for principles that constitute good evidence beyond the well-worn technical hierarchies, and constructs a framework of appropriateness through which policy-relevant evidence might be considered. In doing so, Parkhurst’s defines ‘appropriate evidence’ as that which speaks to multiple concerns at stake in a policy decision, is constructed in ways that are most useful to achieve policy goals, and is applicable in the local policy context. In turn, ‘good evidence for policy’ comes to be defined as evidence that meets the aforementioned appropriateness criteria, as well as high-quality research standards.
Before presenting conclusions around the good governance principles for the use of evidence, Parkhurst next turns to a discussion on the factors necessary to ensure the democratic legitimacy of the ‘evidence advisory system’. While a vital discussion to be had, I’m unsure whether Parkhurst succeeds in making a new contribution to the long-standing debates and voluminous literature on the topic of whether (or to what extent) we can accept irrationality as the cost of democracy. With this issue parked, the book concludes by outlining key principles for the good governance of evidence.
Taking a broad understanding of good governance as the ‘art’ of systems and processes through which collective decisions are made, he argues that governance needs to be thought of in terms of both the processes and outcomes that are relevant to the use of evaluation with a policy process.
To do this, eight good governance principles for using evidence in policy making are elaborated
These principles appear sensible and provide a comprehensive response to the issues/ concerns raised throughout the book. Yet I can’t help but wonder whether Parkhurst has gone far enough and fully achieved a description of principles in a governance framework around the use of evidence: to my mind, the principles seem closer to what might underpin a quality assurance checklist. Parkhurst makes a brief reference to the use of governance in the corporate sphere but does not pursue this enquiry—arguably, this might be where some of the key lessons and fresh insights lie. This governance perspective would throw into focus a slightly different set of issues, such as the delegation of decision making (e.g. around the evidence that is gathered and how it is used); the strategies for managing risks (e.g. around the lack or inappropriate use of evidence); the duties owed to those with a ‘stake’ in the evidence (e.g. to the public or potential program beneficiaries, that decisions are made in their interests); and expectations around transparency and reporting (e.g. ‘rules’ safeguarding quality information and communication flows in deliberative processes).
Overall, the strength of Parkhurst’s argument is the explicit engagement with the political questions in determining what constitutes the better use of evidence. Doing this allows him to recognise that improvement requires building institutional arrangements (however incrementally this may be) that can address forms of bias while incorporating democratic representation. Readers may still want to know more about the ‘how’, and in the Australian and especially Canadian evaluation contexts one can’t help but be reminded of growing calls for an Evaluator-General to oversight the better use of evaluation evidence. These live debates highlight the importance of Parkhurst’s book in contributing to new ways of thinking about the twin problems of evaluation utilisation and the use of reason in democratic politics.