Measurement

Performance improvement ensures the system is working in an efficient and effective manner. It is not a one step process, but is multi component spanning all phases of EBP implementation and creating a feedback loop that allows the information to be utilized for continual improvement.

Fidelity

[Note: The material on this page and the linked sub-pages that follow was provided by Gary R. Bond, Ph.D., Indiana University]

Fidelity refers to the degree of implementation of an evidence-based practice (EBP); A scale that measures fidelity is called a fidelity scale (Bond, Evans, Salyers, Williams & Kim, 2000). Such scales have been developed for each of the six EBPs included in the Implementing EBP Project (“Toolkit Project”):

• Assertive Community Treatment (ACT)
• Supported Employment (SE)
• Illness Management and Recovery (IMR)
• Family Psychoeducation (FPE)
• Integrated Dual Disorder Treatment (IDDT), and
• Medication Management According to Protocol (MedMAP).

Each scale consists of approximately 15-30 items assessing the critical ingredients of the EBP, based on the underlying principles and methods of the EBP. The scale items provide concrete indications that the practice is being implemented as intended.

Why measure fidelity?

One key use of fidelity scales is for monitoring programs over the course of their development. Programs that are not monitored may erode over time in their fidelity of implementation. Routine use of fidelity scales provides an objective, structured way to give feedback about program development. Experience also suggests that this is an excellent method to diagnose program weaknesses, while helping to clarify program strengths. Fidelity scales also provide a comparative framework for evaluating statewide trends and outliers. The strategic use of repeated evaluations of programs using fidelity scales, either for an individual program or on a statewide level, is based on the general principle that whatever is attended to is more likely to be improved.

Important Fidelity Links:

To read more on the subject of fidelity scales, see:

Introduction to Evidence-Based Practice Fidelity Scales

Links to Protocols and Scales for Each EBP

References to Literature on Fidelity Scales

Issues in measuring fidelity:

• Measuring fidelity must be ongoing
• Refinement of fidelity measures based on current practice and a focus on what are the key elements?
• Measure should be both quantitative and qualitative
• Exploring who is in charge of measuring, who gets the information and for what purpose?

Outcomes

Outcomes are a way to measure the impact of a process or program. Only collecting the outcomes are not enough, they must be fed back into the program for quality improvement. Some areas to help facilitate outcome collection:

• Standard outcomes – consumer based - include outcome monitoring in quality care
• Ongoing availability of TA
• Reliable use of quality measure by quality staff
• Assess GOI at state level as well as local.

Information Technology helps to build the infrastructure needed for sustained administration and implementation of EBPs.

Back