Criterion 11 from the ACCME’s Updated Accreditation Criteria (September 2006) states the following: “The provider analyzes changes in learners (competence, performance, or patient outcomes) achieved as a result of the overall program’s activities/educational interventions.”

Several misconceptions regarding ACCME expectations of CME providers with respect to educational outcomes measurement are still in circulation. One of the most common misconceptions is discussed here. Subsequent articles will deal with other misconceptions.

Misconception: CME providers don’t have to measure every CME activity.

This statement is incorrect. The following quote is from the Ask ACCME section of the ACCME website (http://www.accme.org/index.cfm/fa/faq.detail/category_id/f270fb8d-50ba-437e-926b-3dd408e2ffab.cfm):

Question: “Is an assessment of changes in knowledge, practice or patient outcomes required for each CME activity?

Answer: Yes……at this point an assessment of change is required for each CME activity. The information is then analyzed by the CME provider in the context of the overall program’s effectiveness.  Criteria #11 and #12 require the provider to have knowledge of the effectiveness of their Program of CME in the context of changes in competence, performance, or patient outcomes.”

Also, in the recently distributed document, “ACCME’s Accreditation Criteria, Notes and Extracted  Examples of Compliance” (November 2008, page 3), relative to Criterion 11, the ACCME states: “Provider seeks to understand the net, or overall, changes in competence, performance, or patient outcomes facilitated by their CME program using data and information from each and every CME activity…” (emphasis added).

However, keep in mind these pearls:

• You don’t have to measure every participant.

• For Regularly Scheduled Series (RSS), the full set of weekly/monthly meetings are most often collectively considered one CME activity and therefore you wouldn’t necessarily have to conduct outcomes measurement for each meeting in the series (although some providers choose to do this). Some providers use a sampling method to measure an RSS (measure a certain number of meetings during the year).

• Likewise, with an annual conference, it is usually considered one large/long CME activity worth many credits, and you wouldn’t have to measure every single presentation that makes up the entire conference.

• In general, physician course directors and faculty aren’t up to speed on the new outcomes measurement requirements (or many of the other newer ACCME criteria). You’ll need to spend time explaining the criteria, the important and changing role of course directors and faculty with respect to measurement, and your outcomes measurement strategies. The term “outcomes” means something quite different to a physician than it does to a CME professional.

So the jury is out—CME providers need to measure each and every CME activity to be in compliance with the ACCME’s Updated Accreditation Criteria. The question is, how are you doing on your progress towards using evidence-based methods for measuring, analyzing, and reporting results from each CME activity, and then translating those results into improvements in your CME activities and overall CME program?

Stay tuned for future articles on misperceptions regarding:

• Outcomes measurement and Level 3 accreditation status

• Measuring changes in knowledge versus competence

• The use of objective performance and patient outcomes data versus physician self-reported data.