Lessons Learned

Lessons for Future Program Evaluation

The research project studied numerous aspects of quality of care, accountability and outcomes over the two years. In the second year of the project, both Community Care Access Centres (CCACs) began to consider how they would address program evaluation of this new service model once the research project was completed. During interviews in the second year of the project, numerous ideas were identified.

  • Mechanisms for determining accountability, quality and outcomes have been identified and need to be addressed in future program evaluation.
  • A workload approach to tracking therapists’ activities and time use fits better with the P4C model than the traditional caseload management approach.

Quality, Accountability and Outcome

Current literature about program evaluation within healthcare services focuses on three broad areas: quality, accountability and outcome (Perrin, 2015). During interviews in Year 2 of the project, key stakeholders identified components of these three areas of program evaluation that they felt were relevant to school based, integrated rehabilitation services and needed to be included in future program evaluation activities:

Quality

Quality includes: equitable access to care; timely access to care; access to occupational therapists; good relationships; consistency of service provision.

I think if the waitlist has decreased, so the kids are getting served in real-time, like problems identified and they’re serviced right away — I think that’s a strong indicator of whether the program is working.

— Community Care Access Centre

…when we’re promoting our Partnering for Change model that we are in fact promoting a model that is consistent across multiple agencies and multiple therapists and our schools have been appreciative of that.

— Community Care Access Centre

Accountability

Accountability includes: doing what you say you will do; fidelity; cost effectiveness; knowledge translation.

It’s about using the resources — the right resources in the right way; so, achieving the highest quality at the lowest cost is always the most important thing.

— Community Care Access Centre

Accountability for me means that we’re delivering a service that we are expected to do, that our staff that are in the schools are providing that support in the school setting that they should be.

— Community Care Access Centre

If the OT is a consistent presence at the school, the teacher gets it! The child’s seating is correct every day, not just when the OT is there.

— occupational therapist

Outcomes

Outcomes includes: self-confidence in children; participation; satisfaction; capacity building; early identification.

One recommendation was to move tracking and reporting processes from the traditional ‘caseload’ approach, which uses a system of individual billings for each child served, to a ‘workload’ approach that takes into account the complexity of the school context and the variety of intervention strategies being used by the occupational therapists.

If a child has some input by an OT there she’s not just going to look at that one child, she’s going to look at the whole environment…so it’s like not just dealing with that child, it’s dealing with the whole thing…it’s the whole class approach.

— Community Care Access Centre

Workload is reflective of requirements in an educational setting and includes assessment and interventions as well as ongoing collaboration with education staff, communication with the children’s parents, and participation in school and district-level committees.

— occupational therapist

An evidence-based research project completed by two Master’s Level occupational therapy students with support from the P4C team and CW CCAC partners, proposed several additional recommendations to facilitate evaluation of the quality, accountability and outcomes of the P4C service.

Footnotes

  • Perrin, K. M. (2015). Essentials of planning and evaluation for public health. Burlington, Massachusetts: Jones & Bartlett Learning. ↩