Criteria for Defining Evidence-Based Practices

More than thirty years ago, Archie Cochrane wrote a seminal textbook, Effectiveness and Efficiency: Random Reflections on Health Services, outlining the need to implement healthcare programs or interventions that have an evidence base and the importance of using randomized control trials (RTCs) to evaluate the treatment. His call launched the modern day focus on EBP. However, outlining the finer details of what constitutes an EBP has proven challenging due to the difficulties of reaching a consensus across researchers, stakeholders and states regarding: how much evidence is needed; accounting for the natural drawbacks of social science research (resources, ethics, real world vs. clinical trials, homogenous sample pool); and allowing for programs that are used in the field currently to develop an evidence base. Accounting for these issues when reviewing programs is important; most criteria assess programs along a continuum. Terms like model program, well-established, promising, etc., reflect the amount of evidence that supports the practice with the definitions and terminology differing from one criteria to the next. Below are links to different criteria sets for evidence-based practices that have been developed to guide the review of programs.

• The American Psychological Association's Task Force on Psychological Interventions and the Task Force on Promotion and Dissemination of Psychological Procedures criteria created two categories: Well-Established Treatments and probably Efficacious Treatments. The APA initially focused on treatments that show positive outcomes (efficacy) in well-controlled studies and have standardization (manuals available), replication, and generalizability (client characteristics identified). Effectiveness studies are the other area, which is how well an efficacious treatment is transported to a real world setting. The APA mentioned this issue as a focus for a later time.

• SAMHSA’s Center for Substance Abuse Prevention created the National Registry of Effective Programs review and identify effective substance abuse prevention and mental health programs. Programs must be nominated for inclusion. Based on 18 criteria, programs are defined as Promising (i.e., some promising outcomes), Effective (i.e., consistently positive outcomes, strongly implemented and evaluated), or Model Programs (i.e., effective, plus available for dissemination, technical assistance available from developer, and fidelity monitoring). These criteria also include cultural competence and consumer involvement. Program descriptions and contact information are available via the online registry.

• Cochrane Collaboration - Systematic reviews of the effects of health care interventions, including mental health interventions.

• Campbell Collaboration - Systematically prepares, maintains, and disseminates reviews of the effects of social, behavioral, and educational interventions.

• The Society for Prevention Research (SPR) is an international organization comprised of scientists, practitioners, advocates, administrators, and policy makers. They are focused on preventing social, physical and mental health problems, and the advancement of science-based programs that promote health, safety, and well-being of individuals and their respective communities. In 2003, the SPR appointed a committee to determine criteria for preventive interventions that are recognized to be efficacious, tested and effective. The result, is the booklet, The Standards of Evidence, which was approved by their Board of Directors on April 12, 2004.

• Center for the Study and Prevention of Violence - Information source about programs demonstrated to be effective in preventing and intervening in youth violence, with applications to mental health.

• The Hawaii Department of Health - Child and Adolescent Mental Health Division adapted the APA criteria for the review of interventions for youth with behavioral and emotional needs. The two categories were expanded to five levels of "Evidence-Based Services" ranging from Best Support (highest) to Known Risks (lowest). The criteria are based on the methodology design, number of trials, replication, standardization, and generalizability.

• The Oregon State Legislature passed an EBP bill in September 2003. The legislation asked the SMHA to define EBPs and ensure that 25% of funds for 05/07 budget will go towards EBPs for populations receiving emergency treatments, youth at risk for juvenile justice involvement or adults at risk for criminal justice involvement and that percentage increases to 50% for 07/09 and 75% for 09/11. The Oregon EBP Model has five levels with the first three defining Evidence-Based practices. Beyond the typical criteria outlined in APA and others (study design, replicable, generalizable, etc.) the highest level of evidence requires the treatment be shown effective in both scientifically controlled and routine care settings as well as having a fidelity tool.