Disability and Ageing
Autism Connect – National Autism Information Line
AMAZE
THE CHALLENGE
Since 2008, Amaze has provided the Autism Advisor Service as a gateway into the Helping Children with Autism Package and an Information Support line to provide free, independent, evidence-based, accessible autism-specific information and support for autistic people, their families and carers. With the transition to the National Disability Insurance Scheme (NDIS), the operating environment for these services changed, but there was an ongoing need for autism-specific information for those navigating a diagnosis and beyond.
THE OBJECTIVE
Amaze contacted ARTD to support the development of a service model suited to the new operating context, and then to strengthen the evidence base for the service to achieve ongoing funding.
OUR APPROACH
Transformation Project
In 2017, ARTD worked with Amaze staff to:
- conduct an analysis of strengths, weaknesses, opportunities and threats (SWOT) for the Autism Advisor Service and Autism Information Line in the NDIS ecosystem
- map a place for a new integrated Autism Information and Support Service in the NDIS ecosystem
- develop a theory of change and logic model for the Service
- develop a monitoring and evaluation framework for the new service, with links to the NDIS ILC Outcomes Framework and NDIS Participant Outcomes Framework, and pilot new data collection measures
- conduct an initial review of the Service covering December 2017 to February 2018.
This project supported Amaze with a successful NDIS Information, Linkages and Capacity (ILC) Building Grant to implement the revised model.
The Integrated Autism Information and Support Service pilot
ARTD worked with Amaze to establish a consumer advisory group to guide implementation and review of the new multi-channel service. We also supported Amaze to strengthen ongoing monitoring data collection, supplemented by additional evaluation data collection to build the evidence base for the service and identify potential improvements to the service. This process was enabled through sense-making sessions with staff and consumers.
This project supported Amaze’s successful application for an NDIS ILC Building National Information Program Grant to take the service national.
The National Initiative
Since 2019, ARTD have been supporting Amaze with the evaluation of the national service. This has involved working closely with Amaze staff and the Autism Connect Consumer Advisory Group to:
- monitor service user outcomes
- conduct initial and follow-up interviews with service users to understand satisfaction with the service, immediate outcomes and longer-term needs and outcomes
- analyse and interpret data and implications for service improvement
- create Tableau dashboards to enable automated monitoring and reporting beyond the evaluation.
IMPACT
The evidence of need and impact provided through monitoring and evaluation supported Amaze’s funding applications and scale-up to a national service, and continues to inform ongoing improvements to service delivery to ensure the service is reaching and meeting the needs of autistic people and their families nationwide.
Dementia-Friendly Communities evaluation
Dementia Australia, 2016–19
THE CHALLENGE
As of 2019, there are an estimated 447,115 people living with dementia in Australia. Without a major medical breakthrough, this is projected to increase to 1,076,129 by 2058[1]. Most people with dementia (about 70%) live in the community, many with support from family and friends[2]. While health and specialist services play an important role in meeting the needs of people with dementia, there is also a need to engage and inform the wider community to reduce stigma and increase social inclusion[3]. A dementia-friendly communities movement has emerged to address this need. The evidence base for dementia-friendly is still emerging – it is primarily descriptive and qualitative[4].
THE OBJECTIVE
ARTD worked with Dementia Australia to iteratively develop and evaluate the Department of Health-funded national Dementia-Friendly Communities Program, which included an online resource hub, a Dementia Friends awareness and education program and community grants.
OUR APPROACH
We co-designed the evaluation with a Steering Committee and Dementia Advisory Group (made up of people with dementia and their carers). The overarching approach for the evaluation was developmental. We proposed this approach to:
- best support the iterative development of the Program
- enable us to adapt our methods to support iterative development and capture the
value of the evolving Program rather than judging against pre-determined outcomes - align with the values of dementia-friendly communities
- suit the nature of community development, which shares the characteristics of
innovation in complex, adaptive systems.
The developmental approach involved building the program team’s capacity for evaluation; facilitating regular data-based discussions about implications for design and delivery; and delivering more formal six-monthly progress reports, focusing on interpreting the evidence in context to support decision-making.
We were also informed by empowerment evaluation, when engaging with funded communities, and principles-focused evaluation when assessing the design process and the Program design.
We evolved our methods over time to best suit ongoing program development. For example, when engaging with funded communities, we realised that their stories could not only inform the evaluation, but support broader public engagement with the program – by helping other communities understand what ‘dementia friendly’ might mean in practice. So, we updated our case study approach to include the production of videos that capture stories of change over a 12-month period.
IMPACT
The project supported the co-design of the initiative. Regular reporting supported data-informed decision-making to strengthen the ongoing development of the initiative at a national level, while site visits and webinars supported reflection and development of community-level initiatives. We were also able to amplify the program through video case studies (making communities’ key learnings available to other communities in a digestible format) shared through the DFC Hub.
SECTOR IMPLICATIONS
Grounded in principles of inclusion, self-determination and empowerment, community development initiatives are driven by community members. When you begin, it is not clear what it will look like, how it will be delivered, or even what outcomes will be achieved. Projects continuously evolve. Given this, there are significant concerns about traditional approaches to evaluation among community development theorists and practitioners[5].
Based on our learnings from applying developmental[6], empowerment [7] and principles-focused [8] evaluation to this community development initiative, we published a journal article –‘Evaluating Community Development’ in Social Work and Policy Studies: Social Justice, Practice and Theory – using the project as a best-practice case study. More specifically, the evaluation has supported dementia-friendly community development and evidence building by collating learnings from funded communities about success factors to support the development of future dementia-friendly communities. We presented our findings at the DFC Research Roundtable ahead of the NHMRC National Institute for Dementia Research Australian Dementia Forum in Hobart (13–14 June 2019), and our findings inform discussion at international conferences.
References:
[1] Dementia Australia. (2019). Dementia in Australia: Prevalence estimates 2019–2058.[2] Alzheimer’s Australia. (2014). Living with Dementia in The Community: Challenges and Opportunities. A Report of National Survey Findings Alzheimer’s Australia
[3] Alzheimer’s Disease International. (undated). Dementia Friendly Communities: Key principles
[4] Hebert, C., Scales, K. (2017). Dementia friendly initiatives: A state of the science review. Dementia, vol 0, pp1–38; Lin, S. (2017). “Dementia-friendly communities” and being dementia friendly in healthcare settings. Current Opinion in Psychiatry, 30(2), pp145–150.
[5] Craig, G. (2002). Towards the Measurement of Empowerment: The Evaluation of Community Development.Community Development, 33(1), pp. 124-146; Kenny, S. (2002). Evaluation and Community Development: Mantras, challenges and dilemmas. Paper presented at the 2002 Australasian Evaluation Society International Conference; Liket, K., Rey-Garcia, M., & Maas, K E. (2014). Why Aren’t Evaluations Working and What to Do About It: A Framework for Negotiating Meaningful Evaluation in Non-profits. American Journal of Evaluation, 35(2), pp. 171-18; Rawsthorne & Howard, 2011; Wadsworth Y. (1991). Everyday Evaluation on the Run. Action Research Issues Associated Inc. Melbourne.
[6] Patton, M.Q. (2007). Developmental Evaluation: Evaluation for the Way We Work. The Nonprofit Quarterly, pp. 28-33.
[7] Fetterman, D., Kaftarian, S.J., & Wandersman, A. (2015). Empowerment evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability, Second Edition. SAGE Publications Inc., USA; Fetterman, D. (2017). Transformative Empowerment Evaluation and Freirean Pedagogy: Alignment with an Emancipatory Tradition. New Directions for Evaluation (155): 111–126.
[8] Patton, M.Q. (2017). Principles-Focused Evaluation: The GUIDE. Guilford Publications, USA.
[9] Department of Social Services. 2012. https://www.dss.gov.au/our-responsibilities/disability- and-carers/publications-articles/policy-research/shut-out-the-experience-of-people-with- disabilities-and-their-families-in-australia
_______
Community development for inclusion
St Vincent de Paul Society NSW for Ability Links NSW, 2018
THE CHALLENGE
About one in five Australians have a disability. While most people with disability are no longer ‘shut in’ – hidden away in large institutions – many are still ‘shut out’ of buildings, homes, schools, employment, businesses, sports and community groups. [9] People with disability can face barriers to being included in their communities, gaining employment, and accessing information and services, including financial, geographical, informational, social and cultural. People with disability from culturally and linguistically diverse backgrounds, people from regional and remote areas, Aboriginal and Torres Strait Islander people, young people, and people identifying as LGBTQIA+ can face additional barriers. Ability Links NSW was designed to support the inclusion of people with disability.
THE OBJECTIVE
We worked with Ability Links NSW providers to produce a toolkit to celebrate their successes, guide future community development initiatives, and help sustain the impact on inclusion.
OUR APPROACH
ARTD worked collaboratively with providers to ensure the toolkit collated practice-based evidence and was accessible. We ran three workshops across NSW to gather Linker’s learnings and top tips for working with different types of organisations – from schools and community centres to local businesses and councils – and different community groups, recognising the intersectional barriers to inclusion. We collated case studies of successful inclusion projects across NSW, and strategies, guidelines and tools that Linkers used to support their work. We also included guidance on monitoring and evaluating community development initiatives of all scales to support organisations to demonstrate the benefits of, and continuously improve their initiatives.
To support implementation of the toolkit, we hosted a webinar for Linkers and developed a video in partnership with Linkers and Crux Media. You can access the toolkit
IMPACT
The toolkit and a video distilling it’s core messages are available to community organisations here.
SECTOR IMPLICATIONS
The toolkit provides guidance on effective design, management and evaluation of community development for inclusion initiatives that may be useful for a range of community organisations supporting inclusion, including organisations funded through the National Disability Insurance Scheme’s Information linkages and Capacity Building projects. We have presented the guidance on monitoring and evaluation through other forums such as the NSW NGO Research Forum.
_______
NDIS Quality and Safeguarding Framework
Department of Social Services (DSS), 2015–16.
THE CHALLENGE
The Productivity Commission’s inquiry report into Disability Care and Support recommended the National Disability Insurance Scheme (NDIS) replace existing disability support systems, which were underfunded, unfair, fragmented and inefficient, and which gave people with disability little choice and no certainty of access to appropriate supports.
In the NDIS market, participants choose their providers, rather than providers being contracted by government agencies. This means that many of the previous quality and safeguarding measures – managed through funding agreements – no longer apply. A new system was needed to replace these measures. As this represented a significant regulatory policy, with the potential to impact individuals, businesses and community organisations, a Regulation Impact Statement was needed.
THE OBJECTIVE
ARTD Consultants worked with governments to consult consumers, businesses and community organisations on options for the National Quality and Safeguarding Framework to inform a Regulation Impact Statement.
OUR APPROACH
We supported the DSS, the National Disability Insurance Association (NDIA) and state and territory governments to consult with people with disability, their families and carers, as well as service providers and peak bodies around the country to identify appropriate quality and safeguarding settings. This included facilitating 16 public meetings in capital cities and regional locations in each state and territory and 7 provider meetings in locations around Australia, as well as reviewing data from 6 workshops with specific stakeholder groups, 220 submissions, 585 questionnaire responses and an online discussion forum. We used Auslan interpreters and closed captioning, as well as accessible venues, and supported submissions through multiple channels to ensure the process was accessible and inclusive.
We captured what people told us in the consultation report. We then worked with the government agencies to agree on the Framework design and roles and responsibilities, and helped edit the final framework.
IMPACT
Our work supported the development of a new regulatory system for the NDIS. This Framework provides a nationally consistent approach that advances the rights of individuals with disabilities and their families by providing them with more information and tools to exercise choice and control and make informed decisions about the quality and suitability of providers.
RELATED PROJECTS
ARTD has continued to support the evolution of the Quality and Safeguarding system, and related NDIS market projects, including the following.
- Code of Conduct review (DSS, 2018): A key element of the Framework is the NDIS Code of Conduct for workers and providers of disability services, which is a legislative instrument that sits under the NDIS Act (2013) (Cth). We revised the accompanying guidelines based on written feedback from peak bodies, then facilitated a full-day workshop with key stakeholders to finalise the guidelines and develop scenarios to support implementation.
- NDIS Auditor Training (NDIS Quality and Safeguards Commission, 2018–19): Auditor training was essential to ensuring a trained contingent of Auditors available from rollout of the new quality and safeguarding system in 2018. We have supported our partners Floyd and Engels with three rounds of auditor training, managing communications and logistics and managing the online examination process.
- Sector Development Fund and Jobs and Market Fund (Department of Social Services, 2017 and 2019): The Sector Development Fund (SDF) was established to support providers and consumers transition to the NDIS. We conducted a mid-term review to assess progress and produce communications materials on the projects. We subsequently conducted a final review of the SDF, in partnership with AlphaBeta, to assess achievements and inform the design of the new Jobs and Market Fund (JMF). We reviewed documentation and consulted with more than 55 government, industry and consumer peak stakeholders through two workshops, and individual or group interviews, and interviewed 22 stakeholders implementing six selected projects. We then worked with the Department to develop a monitoring and evaluation framework for the JMF. SDF project profiles have been shared on the NDIS website to ensure resources and learnings are accessible.
