Evidence and persuasion

Aricle Image for Evidence and persuasion

March 2018

By Manager, Katherine Rich

Why do economic arguments hold sway in public debate? I recently attended the thought provoking Melbourne School of Government’s conference:  A Crisis of Expertise? Legitimacy and the Challenge of Policymaking. In a panel discussion, Economist Richard Denniss spoke about the disproportionate weighting given to economic evidence and its persuasive power in public debate. It got me thinking about why this is so.

In their simplest form, economic arguments appear easy to understand and are compelling. To illustrate this, Denniss related a story of his son asking him if he would take him to Disneyland. When Dad said ‘no’ and used an economic argument – ‘it’s too expensive, we can’t afford it’ – his son innately understood and, for the most part, accepted the decision. However, as Denniss pointed out, the concept of Disneyland not being affordable is really a value judgment rather than an objective fact. The reason Denniss’ family didn’t go to Disneyland was because they had other priorities to spend their money on.

Economic arguments, such as those made through the use of cost-benefit analyses (CBAs), can seem objective and easy to understand even though they are not – with values concealed behind a veneer of expertise and a language that not everyone understands. I agree with Denniss’ suggestion that rather than pretending a cost-benefit analysis was value neutral, advocates of particular causes should start from their value position and then make an economic case for their argument.

So what does all of this mean for evaluators when evaluative arguments are complex and can be difficult for non-evaluators to follow? We could leverage some of the same power of economic argument – make our evaluative judgments appear value neutral. However, in trying to make a holistic judgement about the merit and worth of a program, it would be problematic to use only one quantifiable metric like a cost-benefit ratio.

What we can do is bring together diverse stakeholders to first understand their perspectives and then develop a comprehensive set of criteria to assess value (see my recent post on this – a balanced approach to valuing in evaluation) and develop a logic model to clearly communicate what success will look like and how it will be measured. We can strengthen the persuasive power of these models, by drawing on social science research to develop and refine them.

When it comes to making economic arguments in evaluation, we can also look more broadly than cost-benefit analysis. Julian King’s recent publication OPM’s approach to assessing Value for Money sets out an approach to measuring value for money (VfM) that goes beyond using blunt, readily quantifiable measures like CBA, and acknowledges that some of the most valuable outcomes can be the hardest to quantify. It claims that good VfM assessments are clear about the value judgments being made. The approach uses an equity lens to capture not only the economy, effectiveness, cost effectiveness and efficiency of an intervention but reach to those most disadvantaged, acknowledging this may be costlier than reaching moderately disadvantaged people but can have greater impact.   

Economic arguments have power not because they are objective, but because they appear value neutral. As evaluators we can advocate for greater transparency of economic metrics and more nuanced approaches to VfM, and we can be explicit about how stakeholder values influence criteria and, thus, evaluative judgements.