Discover

Effect use of evidence is crucial to inform the development of effective and appropriately targeted policies and programs, as well as ongoing program management and strategic decision-making. We provide tailored training and capacity building projects to meet the specific needs of individual clients.

Our services include:

  • development of organisation-wide evaluation policies and plans
  • advice on evaluation and monitoring
  • tailored training and workshops in program logic development, evaluation concepts and performance monitoring
  • mentoring for individuals and teams delivering internal evaluation projects
  • development of evaluation toolkits and guides
  • peer review of evaluations.

Our senior staff use their specialist skills and expertise, along with an understanding of effective adult learning techniques and change management processes to ensure the knowledge gained through our training and capacity building projects is used in the workplace.

Capacity building services can be delivered as a stand-alone project or as part of the delivery of a larger evaluation or monitoring project. We recently used our evaluation expertise working with Patricia Rogers of RMIT University to develop the NSW Evaluation Toolkit.


Project examples

Click on any of the following projects for more information.

Advice on development of Evaluation Readiness Service

(Department of Social Services, 2017)

Advice on development of Evaluation Readiness Service

(Department of Social Services, 2017)

The Department of Social Services (DSS) is establishing a new ‘evaluation readiness service’ that aims to ensure a program has the mechanisms in place to monitor and report on program performance across its lifecycle and be able to evaluate program outcomes. DSS contracted ARTD to assist with developing and refining guidance and templates to be used by DSS staff to deliver the evaluation readiness service. ARTD conducted a document review of the evaluation readiness content and interviewed key DSS staff in order to further our understanding of the service and the content and skills necessary for delivery. Once the evaluation readiness service materials have been finalised, a workshop will be held with DSS staff to orientate them to the service process. This will equip DSS staff with the knowledge to deliver an effective and efficient service to internal and external stakeholders seeking evaluation readiness services.

Literature review on impact evaluation methods

(Department of Industry and Science, 2015)

Literature review on impact evaluation methods

(Department of Industry and Science, 2015)

Impact evaluation is a broad topic that includes descriptive, underlying and evaluative questions. The appropriate approaches and designs will depend on the nature of the program, the questions being asked, and the time and resources available. This project was conducted in conjunction with Professor Patricia Rogers of RMIT and betterevalation.org to provide the department with advice on selecting impact evaluation methods. The report was provided in the context of loud calls to apply randomised controlled trials (RCTs) to improve the rigour of the evidence base for public policy. The project identifies a key role for RCTs but one that is subordinate to the overarching goal of evaluation for generating reliable and valid evidence to inform decision making. The report was provided to the department and presented in a panel format including discussion of different perspectives on impact evaluation. The report has not only stimulated discussion with in the department, but provided advice and on different approaches for conducting impact evaluation depending on the nature of the program, the questions being asked and the resources available. The report has been published on the Department’s website and can be found http://www.industry.gov.au/Office-of-the-Chief-Economist/Publications/Documents/Impact-evaluation-report.pdf

Program logic and evaluation training for Department of the Environment

(Department of the Environment, 2015)

Program logic and evaluation training for Department of the Environment

(Department of the Environment, 2015)

We conducted workshops on program logic with Departmental staff. We first introduced the concept and approaches using case studies, then supported staff to work on logics for their own programmes. The workshops further explored how program logic is used to design monitoring and evaluation plans.

WorkCover Evaluation Capacity Building

(WorkCover Authority NSW, 2015)

WorkCover Evaluation Capacity Building

(WorkCover Authority NSW, 2015)

We were engaged to assist WorkCover to increase the return on investment they obtain from monitoring and evaluation. They have traditionally relied heavily on survey data collection and were experiencing ‘survey fatigue' and low response rates. This problem required an analysis of contemporary approaches to evaluation to identify cost-effective approaches to fit the WorkCover context. We provided a report in two parts. The first outlined different purposes and approaches to evaluation, highlighting the importance of performance monitoring and formative evaluation for new projects and the need to delay sophisticated impact evaluation until an intervention is sufficiently definable, significant and mature. The second brought together the latest research on achieving high-quality survey data through high response rates across 18 different survey delivery modes. The report has been used to inform the evaluation design for new projects and make immediate improvements to survey techniques.

Design an online evaluation toolkit for the NSW Government

(RMIT University, 2014)

Design an online evaluation toolkit for the NSW Government

(RMIT University, 2014)

The NSW Government introduced a sector-wide Evaluation Framework in 2013. ARTD, in conjunction with Professor Patricia Rogers of RMIT University, designed, wrote and piloted an online Evaluation Toolkit to support the Framework. It covers the core elements of a significant program evaluation, with a focus on a rigorous outcome evaluation. The toolkit is linked to the BetterEvaluation website, and is used as a key reference in the NSW public sector.

Mentoring for managers doing program evaluations

(Department of Immigration and Border Protection, 2014)

Mentoring for managers doing program evaluations

(Department of Immigration and Border Protection, 2014)

ARTD advised and supported programme managers to design and conduct evaluations of their programmes, including a programme to support refugee youth in Australia, the settings for a visa programme, and English language training for staff at overseas posts. ARTD also supplied technical services for surveys, interviews and data analysis. The mentoring substantially progressed the evaluations and gave the Department’s staff knowledge and skills in the practical experience of conducting an evaluation project.