Continuous Practice Improvement
An approach by the Parenting Research Centre
Leading organisations partner with us to better understand their progress towards creating impact and ensuring positive outcomes for children and their families.
Through designing real-time practice data feedback systems that give agencies and practitioners information on programs and practice frameworks, we ensure they are on track to deliver effective services and make a difference in the lives of the families they work with.
What is Continuous Practice Improvement?
Continuous Practice Improvement (CPI) provides community service organisations with a practical way to realise the vision of evidence-based practice. CPI views evidence-based practice as a process rather than a product; it begins with current practice and uses data to drive iterative improvement in the quality of practice. The foundations of the CPI approach are built on Continuous Quality Improvement processes and Improvement Science.
How does it work?
CPI is a data-driven approach to improving the quality and effectiveness of practice. CPI is a system that has two key elements:
- Data transformed into meaningful practice intelligence; and
- Team-based continuous improvement processes that ensure learnings from the data are translated into real changes on the ground.
A CPI system can answer typical program evaluation questions, but differs from traditional evaluation in that it is continuous — rather than a slice in time — and supports an ongoing and dynamic process of practice development and adaptation. This offers a significant advantage over evaluation approaches that require consistent and uniform program delivery throughout the evaluation period, and extended periods of time before program improvement recommendations are received.
What are the benefits?
In addition to informing decisions about program implementation, CPI data systems can be used to provide feedback to multiple audiences (for example implementation teams, senior managers, funders, policy makers, program developers, evaluators, program leadership, communities and clients).
Adopting a CPI approach:
- Provides a compelling demonstration of the impact of your programs and services
- Underpins practice governance by creating a line of sight between practice and client outcomes for the whole organisation
- Illuminates the relationship between what you do and your client outcomes
- Is a tangible way of valuing local knowledge creation
- Puts real-time data in the hands of the people best placed to use it, enabling true data-driven decision-making
- Involves your staff in a systematic process of learning and improvement powering genuine and sustainable improvement driven from the bottom up.
Developing an effective CPI system
An effective CPI system has a number of important working parts (see image above). We can help you to build any or all of the following elements:
Theory of change
Building a CPI system begins with developing or refining your program’s theory of change. There are various ways to depict a theory of change, or program logic, however, the most useful approaches endeavour to make the program’s mechanisms of change explicit. For this reason, we recommend ‘outcomes mapping’ as way of representing and road testing your program’s theory of change.
A well-defined practice framework or model
The next step is identifying and describing the practices needed to bring your theory of change to life. Developing a practice framework involves articulating your vision, designing core aspects of your practice architecture (guides and tools that will help your staff operationalise the practice strategy), and describing the key elements of the practice approach in a way that enables practitioners to effectively implement it.
Building a data set
There are three types of data that are needed for an effective and fully operational CPI data system:
- Outcome measures: your choices here are guided by the program’s theory of change. A range of cascading outcomes need to be tracked including intended outcomes (the results we want for children and families); mediating outcomes (what needs to change to achieve intended results); engagement outcomes (quality and nature of client engagement with change processes); and implementation outcomes (immediate results of practices).
- Process measures: these track the implementation of key actions and activities. The choices here are guided by your practice framework.
- Progress measures: involve continuous measurement of early indicators of a good outcome or clients’ progress. For example, you might track the client’s perception of their relationship with the practitioner and/or the clients growing sense of self-efficacy.
Identifying indicators
It’s important to focus on the outcomes that matter. Rather than choose outcomes purely because they are measurable, we recommend finding the best possible measures for the outcomes that are important.
It’s also rarely possible to measure every outcome included in a theory of change, so choices must be made about what information will be collected. Useful data can be drawn from all sorts of sources including administrative databases. Where formal measures are needed, tools shown by research to be valid and reliable are preferred. It’s also important that any assessment tools used with families are useful to them, as well as providing important data for the CPI system. Another important consideration is the time cost associated with data collection for both families and practitioners.
Data capture
It is crucial that data collection processes are simple, efficient and involve minimal additional processing by practitioners. Any process that creates additional data collection or manual data processing imposes unnecessary burden and creates friction in the process that ultimately degrades data completeness and quality. Fortunately, we now have access to a range of affordable online data collection tools that easy to use.
Data dashboards
To be useful, raw data needs to be transformed into practice intelligence and be readily accessible in real-time to staff who are driving improvement initiatives. A fully functioning system should process the raw data automatically in the background, and not involve manual reporting or analysis by intermediaries.
Continuous improvement processes
The final key component of a CPI system is the establishment of team processes to ensure that the data drives real improvement. This generally involves the establishment of a Practice Improvement Team responsible for overseeing quality and performance of the CPI Data System and conducting rapid Plan-Do-Study-Act cycles to drive improvement.
CPI and rapid cycle evaluation
CPI systems generally aspire to create comprehensive and long-term solutions for programs. However, it is also possible to deploy Plan-Do-Study-Act cycles in targeted improvement areas. In this case, data collection is typically limited to the measurement of the specific improvement in focus. This approach gives the organisation the capacity for rapid cycle development and testing of program improvements.
Data collection software
Our clients can access PRC’s own online data collection software, PRISE. This tool enables convenient and efficient data collection without the costly licensing fees associated with commercial products. PRISE is also compatible with Power BI, providing for the rapid development and testing of CPI data system prototypes prior to commissioning bespoke software systems, or longer-term data solutions.
Think we can help?
Through our PracticeWorks™ co-design methodology, we can provide support at the level you need for the development and deployment of your CPI system, including helping you refine your program’s theory of change, designing or refining practice strategies, or designing, building and supporting a CPI data system. We can also provide rapid cycle evaluation support.
Contact us
We blend real-world practice expertise with research rigour and a highly developed collaborative process refined over 20 years of working in partnership. This powerful combination helps our clients uncover solutions that are both practical and sustainable.
We’d love to hear from you.
Dr Catherine Wade, Principal Research Specialist
P: +61 3 8660 3500
E: info@parentingrc.org.au
W: https://www.parentingrc.org.au/contact-us/