Summary of Impact Research New Zealand Ltd Projects 2015 and previous
Family Works Resolution Service: Results Based Accountability Pilot
Impact Research NZ was contracted by Family Works Resolution Service to develop, trial and evaluate a survey mechanism to gain evaluative feedback about the provision of Family Works Resolution Service. A combination of survey and interview data is being collected from Family Works Resolution Service clients, utilising a Results Based Accountability (RBA) approach.
In addition, survey and interview data is also being collected from key stakeholders, including call centre staff, contracted mediators and preparatory counsellors, and managerial personnel. The client survey mechanism is aligned with the Ministry of Justice and Family Works Resolution Service’s Key Performance Indicator (KPI) requirements, as part of monitoring and reporting on the provision of services, including the production of management and funder reports. KPI and RBA reporting, when combined, will form a comprehensive evaluative framework.
Developing NGO Capacity to Undertake Evaluation
2015 – Current
Impact Research NZ has been working with a large NGO (that offers a broad range of social services for families in South Auckland) to undertake an organisational review of their tools used to evaluate their programme and services. Workshops have been conducted on developing an organisation and programme evaluation plans linked to their Annual Plan and Strategic Plan. Other workshops focused on developing and implementing RBA and innovative models of evaluations. The NGO was also interested in its organisational responsiveness to Maori and Pasifika and both hui and fono were held facilitated by Impact Research NZ and reports provided to the NGO for follow-up.
Community Initiative – Process Evaluation
Impact Research NZ, in partnership with David Turner Research Ltd, undertook a process evaluation of a community initiative pilot project funded by MSD currently being trailed in two Auckland districts: Manukau and Waitakere. The pilot project is targeted to assist low income individuals and families to improve their circumstances. A process evaluation assesses whether a program suits the context for which it was designed, if it has been appropriately implemented, and if it has reached those persons for whom it would provide the most benefit.
The evaluation methodology utilises a combination of interviews and document analysis to achieve the evaluation objective. The evaluation focuses on the quality and quantity of services delivered through the programme, both from a programme perspective and from the client perspective. The process evaluation was based on the programme’s intervention logic, which describes the problem to be addressed as well as identifying programme inputs, outputs, and intended outcomes. The evaluation l was used to inform future ongoing monitoring of the programme.
Turner, D., Weir, A., McEwan, B., & Holmes, J. (2015). Community Finance Pilot: Process Evaluation 2015. Ministry of Social Development: Wellington.
Social Workers in Schools RBA Evaluation
Presbyterian Support Northern commissioned Impact Research NZ to undertake the Social Workers in Schools (SWiS) RBA Evaluation. 126 schools in the upper North Island participated in the survey. The number of SWiS in schools = 71. The number of children in SWiS service = 1653. In 2015 the service evaluation included responses from more than 650 responses. Children and their parents/caregivers participated in a paper-based survey and the school staff in an online survey designed to find out how well the service is doing, suggestions for improvements and what are the outcomes of the service for children, their parents/caregivers and the school staff. The ten regional sites each received a report to share with their schools and an overall report provides information on how well the services is doing in reaching its intended outcomes and what services improvements are to be made.
Family Works New Zealand RBA Pilot Survey
The Ministry of Social Development funded the establishment of a common outcome measurement framework, including conducting a baseline RBA pilot survey of clients, across seven Presbyterian Support regions throughout New Zealand. Impact Research NZ was commissioned to undertake the development and implementation of the RBA pilot surveys of children, youth and their parents/ caregivers. The development of the survey instruments was done in collaboration with the seven regions. Each region received a report on their survey findings including client demographics, quantitative and qualitative data, as well as an overall report looking at the Family Works NZ service RBA survey results as a whole.
An External Review of the Response Made by The Salvation Army to the two Canterbury Earthquakes of September 2010 and February 2011
Impact Research NZ was commissioned by The Salvation Army to undertake an external review of its response to the two Christchurch earthquakes of September 2010 and February 2011; with the aim of discovering what could be done to ensure that The Salvation Army’s response to future disasters was appropriate, effective and consistent with its Mission Statement. The researchers worked in collaboration with senior Salvation Army staff to formulate the overall research design, and to identify the necessary research documentation and interview participants. A literature review was undertaken to identify key research findings in relationship to best-practice disaster management.
The research project undertook two stages of data collection. In stage one, a document review and analysis was undertaken assessing a range of earthquake-response documents including: Salvation Army-based daily field reports, email communications between staff, quarterly and annual Board reports, financial reports related to earthquake fundraising and expenditure, website press releases, and two low-level review reports completed by the Emergency Services. In stage two, a mixture of face-to-face and telephone interviews were undertaken with senior Salvation Army staff; and key informants selected from government, local government and NGO personnel involved in the Canterbury earthquakes.
The research findings, in conjunction with the ‘best practice’ disaster-response research literature, identified a number of organisational opportunities for The Salvation Army to consider when reviewing its Disaster Response Policies, Protocols and Practices.
Impact Research NZ has been contracted by Family Works Resolution Service to develop, trial and evaluate a survey mechanism to gain evaluative feedback about the provision of Family Works Resolution Service. A combination of survey and interview data is being collected from Family Works Resolution Service clients, utilising a Results Based Accountability
This evaluation of day programmes for people living with dementia, aims to identify the elements of an effective client-focussed programme, identify the methods employed by organisations to measure the quality of outcomes of programmes, and lastly to determine the efficacy of the methods adopted.
In Collaboration with the Royal New Zealand Plunket Society and Family Works Northern, Impact Research NZ have designed and implemented an evaluation plan, ultising a Results Based Accountability (RBA) framework, to access the outcomes of an early intervention partnership pilot. This evaluation aligns with finding and recommendation from an earlier evaluation on the pilot service conducted by Impact Research NZ, which identified that the ongoing collection of robust outcomes-focused data would support the success of the service.
Ⓒ 'Dancing with Data' Programme
This practical evaluation programme was offered in partnership with Auckland Communities Foundation and funded by SKYCITY Auckland Community Trust Board. Three cohorts of participants from social service NGOs completed the programme, which comprised three workshops of two hours duration, with an expectation that development activities would be completed between the workshops; an overall commitment of approximately 12 hours over the duration of programme.
Impact Research NZ have were commission in 2013 and 2014 to undertake qualitative thematic analysis on data collected from multiple sites from clients on their experiences and outcomes of engaging with Family Works Northern’s social support services.
Enliven Stakeholder Survey
March 2014 - March 2013
An online survey of stakeholders, including District Health Boards and Needs Assessment Agencies, was designed and conducted on behalf of Enliven in 2013 and 2014. The survey gathered both quantitative and qualitative data in order to gain an understanding of the Enliven service’s performance from the perspective of its stakeholders.
Impact Research NZ was commissioned by ACC Home and Community Services to undertake a scoping exercise in order to determine the how the six lead providers of services currently collect and use client data to support the quality of their service provision. Multiple data collection methods were used to gather data from a range of stakeholders.
This evaluation of Enliven’s home support services for clients of ACC Home and Community Support Services designed an piloted a Results Based Accountability (RBA) survey, in order to measure outcomes of the service.
The Royal New Zealand Plunket Society and Family Works Northern collaborated to pilot an innovative model of early intervention social service provision in two areas of high socio-economic over eighteen months. Impact Research NZ was commissioned to conduct an evaluation of the early intervention pilot utilising a participatory approach that engaged key stakeholders and used a range of data sources. The evaluation addressed the effectiveness of the service in terms of meeting its goals and objectives, identified factors that supported effective service delivery, and made recommendations for service delivery going forward.Read more
Waitakere Family Start is a collaborative service delivered by West Fono Health Trust and Presbyterian Support Northern. The service supports families/whanau through the early years of a child’s life, with the child at the centre of service delivery. Impact Research NZ conducted a pilot evaluation of the service, utilising two different approaches to the collection of feedback from clients, in order to assess outcomes for clients of the service, and whether there were advantages to either data collection method, in terms of supporting service outcomes and quality.
Presbyterian Support Northern, Whakamanhia Wahine Tauranga Women’s Programme Evaluation
June 2013 – November 2013
Impact Research NZ was commissioned by Presbyterian Support Northern to conduct an evaluation of their pilot programme for women offenders, funded by Department of Corrections as part of their Innovation fund aimed at reducing reoffending. The evaluation aimed to explore how effective the programme was in achieving its intended aims and included reviewing the programme implementation and how well it was received by participants and key stakeholders.
The goal of Child Youth and Family (CYF) is for children in care to be safe, happy and part of a family. If a child cannot return to their own family, permanent care is required. Care does not always address emotional and behavioural difficulties; this can lead to a cycle of failure and aggression. Read more
Inner City Interconnectedness
Connectedness refers to the relationships people have with people around them and their surroundings. It includes relationships with family, friends, neighbours and people they have met at clubs or through voluntary work for the community. Connectedness also encompasses the way people feel about the built environment: if there are good services available, if they can walk around their neighbourhood, if the public transport is easy to use and if they feel safe.Read more
Kidpower Healthy Relationships
In 2010 Impact Research evaluated the Kidpower, Teenpower, Fullpower, Healthy Relationships programme. The programme provided training in violence prevention, personal safety and self defence. It was designed for all ages and abilities, for individuals, staff and professionals. It included risk reduction and successful communication techniques. Read more
Kiribati – Environmental Migrants
Kiribati (pronounced “Keer-ah-bhass”) comprises three groups of islands in the South West Pacific. The people of Kiribati are not Kiribatese, but I-Kiribati (pronounced “Eee-Keer-ah-bhass”). Kiribati is expected to be the first country in which all land territory disappears due to global climate change. New Zealand has an immigration agreement with the government of Kiribati to relocate the first few environmental refugees.Read more
Midlands - Results Based Assessment
In 2010 Family and Community Services (FACS) commissioned Presbyterian Support Northern to conduct an outcomes-focused evaluation of Midland family and budgeting services. Impact Research (of PSN) conducted a Results Based Assessment (RBA) pilot and recruited 26 providers to complete three RBA surveys over a two year period. Read more
National Social Workers in Schools (SWiS)
This one year pilot was commissioned by Child Youth and Family (CYF) to evaluate the effectiveness of a Results Based Assessment (RBA) framework for measuring changes in SWiS services. Read more
This pilot-programme is trialling an early intervention that involves Plunket and Family Works Northern staff working together to support clients. It is expected that the two organisations will achieve more for clients through working together, than they could working independently. Read more
Results Based Assessment
Results Based Assessment (RBA) is a type of evaluation that measures outcomes. Government departments, not-for-profit organisations and educational services are increasingly relying on RBA to provide evidence for the success and quality of their practices. PSN has been at the forefront of RBA development since 2009. Using specially designed survey tools, PSN collects information from Family Works, Enliven, Coactive and Budgeting Services. Read more
Refugee Catch-Up Class
The refugee student catch-up class began in 2005 after Somali community members voiced concern about the under-achievement of their college-age, school children. The catch-up class is a collaborative effort between families and community. The majority of students are Somali and Eritrean. Read more
Safer Homes in New Zealand Everyday (Shine)
Shine delivers training on domestic abuse to a wide range of professionals who work in the social services or criminal justice system. A small number of these training programmes are open to the public. Read more
Social Workers in Schools
The Social Workers in Schools (SWiS) service is funded by Child, Youth and Family (CYF) under the Ministry of Social Development (MSD). The service was established in response to the impact social and family problems were having on students. Read more
The Theatre in Education Trust (THETA) Sexwise Programme
The New Zealand Health Curriculum states that the purpose of sexuality education is to provide students with the knowledge, understanding and skills to develop positive attitudes towards sexuality, to take care of their sexual health and to enhance interpersonal relationships. Read more
Waitakere Process Evaluation
In 2010 Presbyterian Support Northern (PSN) began a results based assessment (RBA) pilot with Social Workers in Schools (SWiS) based in Waitakere. Read more
Jun 2013 - Effective Client-Focussed Dementia Day Programmes in the Community: Improving the Quality of Life for People Living with Dementia and their Families
Nov 2012 - Evaluation of Safer Dance Parties and Large Events
Nov 2012 - Safer Parties Together: One Voice
Jan 2011 - Evaluation of the Kidpower Teenpower Fullpower Healthy Relationships Programme
Dec 2010 - Review of North Shore Family Violence Crisis and Advocacy Service Requirements