Governments are increasingly using the internet to support or deliver social policy and programs and it is important that we as evaluators have the knowledge and tools to effectively evaluate websites or online services. Our Senior Consultant Andrew Hawkins has managed several evaluations of online services, including evaluations of Gambling Help Online (Victorian Department of Justice), Lifeline Online Crisis Support Chat Trial(Lifeline Australia) and the SageCentre (Ageing, Disability and Home Care). Andrew is presenting a paper on the evaluation of websites and online services at the 2011 American Evaluation Association annual conference in Los Angeles, California. His Seehttp://www.eval.org/eval2011/
ARTD have developed an evaluation framework, strategy and monitoring system for the NSW Department of Family and Community Services Community Builders program, funding community organisations for activities that build community strength and capacity. The Community Builders program has been developed in response to an emergent body of evidence suggesting that community strengthening is an effective way of reducing inequality and disadvantage.
The evaluation framework provides a basis for evaluation and monitoring. It is shaped by the program’s aims and objectives, informed by evidence about constitutes a strong community, and structured by program logic for Community Builders. The evaluation strategy aims to generate an evidence base for the program that can inform decisions by stakeholders at different levels. It sets out important strategic issues, key questions for evaluation, and feasible methods to address them.
ARTD have worked in collaboration with Community Services and the Local Community Services Association (LSCA) in developing the framework and monitoring system. Principal Consultant, Sue Leahy recently co-presented with these partners to the 2001 LCSA conference on the approach to and progress with this work.
ARTD recently worked with Camp Quality on a major national research project to find out more about the needs of children with cancer, their siblings and parents throughout their cancer journey, and how Camp Quality and other cancer support organisations could better meet these needs.
The project involved focus groups with Camp Quality children with cancer, their siblings and parents at 18 camps across Australia, a survey of Camp Quality families, an online survey of relevant health professionals, and a literature review. Camp Quality is using the findings of the research to inform further development of the programs they deliver to support families around Australia.
ARTD Consultants Margaret Thomas and Narelle Ong recently worked with the Clinical Education and Training Institute, Rural Division (previously the NSW Institute of Rural Clinical Services and Teaching) to evaluate the Rural Research Capacity Building Program. The NSW Health Framework for Capacity Building—which highlights elements of workforce development, organisational development, resource allocation, leadership and partnership—was used to guide both the development and implementation of the program and the program evaluation. As part of the Institute’s overall evaluation, ARTD interviewed health professionals in the Program, and analysed the data to assess the Program’s effectiveness in improving research capacity among rural health professionals and the impact in their workplaces. The evaluation also identified barriers to research capacity building in rural NSW health services and possible improvements to the Program. The paper they co-authored with our clients, Dr Emma Webster and Ms Linda Cutler, has now been published in the Australian Journal of Primary Health, Volume 17, No 1 2011 p107–113.
ARTD completed the five-yearly legislative review of the Medicare Provider Number legislation on behalf of the Department of Health and Ageing in 2010. The review involved extensive consultation with the sector, in particular medical colleges, professional bodies and relevant government officers. The review report was tabled in Federal parliament in February and is available on the Department of Health website.
We take our writing seriously at ARTD. We know our clients need reports that clearly and concisely answer their key questions. Our staff have trained with Mark Tredinnick, author of The Little Black Book of Business Writing. You can find out more about Mark’s writing and consulting work on his website.
We also have internal processes to ensure the quality of our written documents. ARTD’s Principal Consultants sign off on all reports, and we have an experienced editor on staff.
Cost-effectiveness and cost-benefit analysis are an increasingly important part of our work in evaluation. To build on our existing skills (and make sure we’re on top of new methods and issues), one of our Senior Consultants, Andrew Hawkins, recently attended a short course on Advanced Methods of Cost-Effectiveness Analysis run by the Health Economics Research Centre of the Department of Public Health, Oxford University. The course covered both the theoretical concepts and practical exercises in analysing health outcomes data, collecting cost data, discounting costs, dealing with missing data, bootstrapping sample data, constructing decision trees, Markov modelling, and calculating and interpreting incremental cost-effectiveness ratios (ICERs) and the net benefits of health care interventions. You can find out more at www.herc.ox.ac.uk/courses/advancedmethodsmainpage.