An educational outcomes measurement plan is critical to demonstrating the value of continuing education (CE) activities. Below are three keys to ensure your educational outcomes measurement efforts are successful.
- Begin your outcomes measurement planning early:
While it may seem counterintuitive, selecting and designing your outcomes measurement plan EARLY will produce a CE initiative that effectively moves participant practice from the current state to the desired state. When planning is initiated:
- Select the outcomes level you want to achieve;
- Use the intended outcomes to align your gaps, objectives, and educational content; and
- Write your outcomes assessment questions while authoring the content.
- Select a realistic outcomes measurement based on the data you will be able to collect:
Accredited continuing education activities must be designed to promote a change in practice, namely competence, performance or patient outcomes. According to the ACCME, the majority of continuing medical education (CME) activities are designed to improve competence:
Desired Outcomes of CME Activities
|Desired Outcome Level*||Frequency of CME activities in 2019**|
|Level 1: Participation||—|
|Level 2: Satisfaction||—|
|Level 3: Knowledge – “Knows/Knows How”||—|
|Level 4: Competence – “Shows How”||98%|
|Level 5: Performance – “Does”||58%|
|Level 6: Patient Outcomes||29%|
* Moore’s Framework
** Data from the ACCME® Data Report 2019
Data sources for measuring competence include educational observations (objective) or participant self-report of competence or intent to change practice (subjective). These sources are easily obtained without additional resources for participant follow-up, which likely accounts for the ubiquitous focus on competence.
For higher levels of outcomes, CE planners must utilize additional data sources and/or participant follow-up. Sources of data to show that participants can “Do” what the activity intended them to do include case based scenarios, participant self-report of intent to change performance, and follow-up surveys asking participants what changes they actually implemented (subjective) and clinical observation of performance, patient charts, and/or administrative databases (objective). At level 6, data to demonstrate improved patient outcomes can be obtained from patient self-report of health status (subjective) or patient charts/administrative databases (objective measures).
- Don’t lose sight of the desired outcome:
While educational outcomes measurement is essential for demonstrating the value of CE, don’t lose site of the planning process needed to produce quality accredited continuing education. An outcomes measurement plan cannot measure improvements that did not take place because the gaps, educational objectives, educational design, and desired outcomes were not aligned. In 2018, Donald E. Moore and his coauthors published an article presenting “a more actionable description of the components of the outcomes frame-work published in 2009.” This was necessary because, in the authors’ view, “…assessment of outcomes became the focus at the expense of planning learning activities that would produce desired outcomes.”
- Accreditation Council for Continuing Medical Education. 2020. ACCME Data Report: Steady Growth in Accredited Continuing Medical Education – 2019. www.accme.org/2019datareport.
- Moore DE Jr, Chappell K, Sherman L, Vinayaga-Pavan M. A conceptual framework for planning and assessing learning in continuing education activities designed for clinicians in one profession and/or clinical teams. Med Teach. 2018;40(9):904-913. doi: 10.1080/0142159X.2018.1483578.
- Moore DE Jr, Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1-15. doi: 10.1002/chp.20001.
Deborah C. Clark, Ph. D.
VP, Professional Development
AKH Inc., Advancing Knowledge in Healthcare
Let’s connect! Do you need help with Educational Outcomes Measurement? AKH’s outcomes team includes educational, clinical, and statistical experts who can boost the success of your educational programming. Contact us today at [email protected].