Reflections on a year of evaluation for design and development
While ARTD has a 29-year history in evaluation, more and more of our work is in policy and program design and development. A trend toward increasing evaluator involvement in design was also evident in my research with Australasian Evaluation Society (AES) members.
So what have I learned from my time in design in the social services this year?
It’s important to develop a deep understanding of the problem you are trying to address so you don’t set your sights on the wrong target. What are the causes and consequences of the problem? Who is affected? If your problem is complex and multi-faceted, a root cause analysis can help you trace back beyond surface level indicators, identify interrelationships between causes and determine effective points for intervention.
But, in focusing on the problem, don’t forget about the individual and community strengths and capabilities you can build on. Or you risk a deficits-based approach that perpetuates problems.
To get to grips on what matters and find the foundations you can build on, seek out the consumer perspective. Then think about the big picture goal.
Creative tools can open up possibilities. Ask stakeholders to build a LEGO model of the end goal or select a photo that communicates how their program works. But respect the choices of those who want to opt out of ‘play’ and share their views another way.
When you think you’ve got an idea of what you’re going to do, reality check what’s feasible and map the ecosystem in which you’ll be operating. By identifying who else is playing in this space and where things are at, you can avoid duplication and harness synergies. You might even be able to find ways of turning factors in the external system that would otherwise have been stumbling blocks into a connected program component.
It can take more steps to get to a shared vision when you involve stakeholders with diverse perspectives and positions in the system, but the value this brings is worth the investment. Over several sessions in a room together, people see from others’ position more clearly and find ways to integrate their views.
While program logic is the ‘go to’ tool of evaluators, and it has a lot of strengths, sometimes you also need a different kind of picture. For example, one that shows how the components of a program connect and interact with external parts of the system.
And, potentially the most important thing, every process will be different. What works will depend on the people and the dynamics.
