Part 3: Make it a process not just a product
Have you ever spent all day in the kitchen cooking a meal, only to find yourself not particularly hungry when you sit down to eat? Often, we’re so focused on creating a Masterchef moment at the end, that we miss the enjoyment of shopping for and preparing a meal to be shared with loved ones.
The same can be true for evaluations. It’s easy to fixate on The Final Report as if we were preparing a degustation dinner party. We miss the memo that it’s not just the products of evaluation that create value. Evaluation processes are also enormously valuable: for our clients, their stakeholders and our own practice.
I firmly believe the opportunity for evaluation processes to create value exists in every evaluation, regardless of its nature, size, scope or methodology. Consistent with Michael Quinn Patton’s work on utilisation-focussed evaluation, our intention is always to work towards our evaluation products being used.[1] This is far more likely if we’ve also focussed on using evaluation processes as strategically as we can.
For example, project start-up meetings are a chance for all evaluation stakeholders to be in the same place, at the same time. We recently began a project in regional New South Wales, where our start-up meeting was the first time the commissioning agency had met the leaders of the contracted service provider. Through conversation at the meeting, practitioners and stakeholders deepened their understanding of each other’s strategic objectives and restrictions. Program design logic workshops, and best practice showcases are also a chance for practitioners to network, share information and identify potential collaborative opportunities.
In our experience, the distinction between process and product is especially important when working with commissioned service providers. It’s common for them to be anxious about how their performance will be judged by the evaluation, and they’re conscious of the evaluation’s role in determining future funding decisions. When we shift the focus to using our evaluation processes for learning and improvement, some of those concerns subside.
Some years ago, Jade and I led an evaluation of a suite of early intervention programs for children with autism spectrum disorders. Our evaluation drew on service providers’ data which, until that point, was held in paper copy only. As part of the evaluation, we developed a data entry portal for service providers to use beyond the evaluation. This was a big win for service providers, whose own learning loop was shortened by having access to up-to-date, easy-to-access data. And the advantage for us was having access to a complete, consistent dataset.
Viewing evaluation processes as equally, if not more, important than their products is also an opportunity for our own professional practice and reputation. We are far more likely to deliver a meaningful, nuanced and relevant evaluation product when we have used every possible evaluation process to create connection with our clients and their stakeholders. One of our regular practices is to present our indicative findings at a workshop with our clients. We’ve found that when clients can help us interpret the findings, there’s a stronger sense of ownership and connection with the results. For us, of course, this also reduces the chance our evaluation product won’t meet our clients’ needs.
Here’s our thoughts on opportunities to leverage evaluation processes.
- Know your audience(s). In our last article in this series, Jade wrote about how to identify your evaluation audience (remembering there’s usually more than one.) Work with your clients to develop a communication map, which outlines each groups’ information needs. We find this helps us keep our end users in mind, and also helps clients realise the potential of the evaluation.
- Ask questions. Evaluation work is typically won or lost through public tender. By identifying ways your evaluation processes can create additional value beyond the RFQ, you can position yourself as a preferred supplier.
- Collaborate and co-design. Encourage your clients to move from a ‘done for you’ to a ‘done with you’ evaluation model. If your clients are used to a hands-off approach to evaluation, help them understand why a focus on collaborative processes can improve the evaluation outcomes.
In our next article, Jade will zoom in on evaluation products and how to make them more usable through careful structuring.
