By Director, Jade Maloney
An evaluator might weep at the plethora of literature that shows their work going unused. But my recent research with Australasian Evaluation Society members shows evaluators have had success in overcoming at least some of the obstacles they have encountered to use.
Disinterest in, or more active resistance to, evaluation: Some evaluators I interviewed had been able to overcome this by: selling the value of evaluation; engaging stakeholders where they are at and from a ‘what’s in it for me?’ perspective; and involving stakeholders in a dialogue as data is collected to answer their questions.
Fear of judgement: In some cases, resistance to evaluation was related to a fear of being judged and a misconception about evaluation. Some evaluators had overcome this obstacle by convincing stakeholders they were there to help them with their work, using their interpersonal skills to make connections with stakeholders.
In some cases, fear related to evaluation construed as an accountability or compliance mechanism rather than a learning exercise. To address this, some evaluators emphasised learning and strengths-based approaches. Others, however, suggested a need to combine accountability and learning purposes or saw accountability in a more positive light.
Lack of purpose: when they encountered evaluations undertaken as a ‘tick-a-box’ compliance activity or without a clear purpose, some evaluators had been able to help stakeholders find a purpose. They helped them work through the questions they wanted to ask, and who would use the evaluation.
Negative findings: Given evidence of the human tendency to accept information that confirms our preconceptions and to refute information that challenges them (Oswald & Grosjean, 2004), it is unsurprising that evaluators had encountered more resistance to negative findings (and positive findings about programs that were slated to be discontinued). To overcome this obstacle, evaluators prepared stakeholders for what they might not expect from the outset, surfacing biases that needed to be overcome. When negative findings eventuated, they shifted resistance by socialising findings as they emerged, use the positive sandwich approach to frame findings and/or focus on lessons learned and solutions, as identified in the following quote. However, it was noted that these strategies can fail when working with organisations that lack a learning culture and when findings are politically unpalatable.
Next time, a look at the common brick walls to evaluation use and whether evaluators have alternative routes around them.