Kathleen McCartney, PharmD
Clinical Instructor
University of Colorado
School of Pharmacy
Coordinator for Continuing Pharmacy Education
Pharmacy
C238-P15
12700 E. 19th Avenue, Suite 4100
Aurora, CO 80045
303-724-3549
Kathleen.McCartney@UCDenver.edu
a. What is the central question, issue, or problem you plan to explore in your proposed work? Does the act of prospective reflection on an educational topic positively affect a participant’s objective learning and subjective response to the educational experience in the context of continuing professional education?
b. Why is your central question, issue, or problem important, to you and to others who might benefit from or build on your findings? In December 2009, the Institute of Medicine released a report titled Redesigning Continuing Education in the Health Professions, which summarized the conclusions of a group of leaders in health care and continuing professional education. The committee called for a wide restructuring of the design and implementation of continuing professional education that would encourage learners to identify their own knowledge gaps, promote lifelong learning toward the end of improving patient care, incorporate the principles of continuing professional development, and increase the study of the science supporting continuing education.1
Prior to the 2009 IOM report, the accrediting body for pharmacy education, the Accreditation Council for Pharmacy Education (ACPE), had responded to various inputs to revise the educational standards for pharmacy students and for continuing pharmacy education. The 2007 standards for the doctor of pharmacy curriculum and the 2009 standards for continuing pharmacy education emphasized the following:
• Individuals should identify their own learning needs
• Educational activities should be designed to promote effective professional practice with the goal of improving patient outcomes
• Educational activities should relate the knowledge, skills, and attitudes learned in class to practice in the real world
• Pharmacists should be lifelong learners, preferably using the model of continuing professional development.2,3
Continuing professional development (CPD) is an “ongoing, self-directed, structured, outcomes-focused cycle of learning and personal improvement” to improve professional competence.4 CPD incorporates the steps reflect, plan, act, and evaluate. In this context, reflection is the identification of personal learning needs, planning is the development of individual learning outcomes and a plan accomplishing them, acting is the implementation of the educational plan, and evaluation is the act of assessing achievement of the learning goals.
CPD has been criticized as being a lofty, yet unwieldy process that lacks evidence of effectiveness. While the process appears to have value, more research is needed on how best to implement each step, and to study the relationship between reflection and learning. My project will increase the knowledge base related to the reflection step, testing one process for reflecting on education need and documenting its effect on learning.
c. How do you plan to conduct your investigation? What sources of evidence do you plan to examine? What methods might you employ to gather and make sense of this evidence? Study subjects will participate in a structured reflection step before attending an educational activity. In the reflection step, subjects will reflect on an educational topic and the publicized learning outcomes, assessing their perceived level of achievement of the learning outcomes; they will develop at least one personal learning outcome for the activity. Achievement of the learning outcomes will be assessed using pre- and post-tests at the time of the activity and another post-test approximately 3 months later. After the educational activity, all subjects will answer questions about satisfaction and perceived achievement of the learning outcomes at the time of the program, and 3 months later.
The steps of my project include:
1. Conduct literature search to find definitions of reflection and locate published processes for conducting a personal knowledge gap analysis.
2. Create a simple and standardized process for a structured reflection. The process will incorporate a set of questions that study subjects can ask themselves about what they need to learn in the context of planning to participate in a specific CE program.
3. Determine a study population and randomize the group into study subjects and controls. Investigator(s) will be blinded as to group assignments.
4. The study group will use the reflection tool to produce some reflective writing, including at least one specific learning outcome for themselves related to the educational program they are going to attend. The study group will receive instruction on writing SMART learning objectives (Specific, Measurable, Achievable, Relevant, Timed).
5. All participants (study and control) will take a pre-test to assess baseline knowledge before the program.
6. All participants will an ACPE-accredited continuing pharmacy education program.
7. All participants (study and control) will take a post-test to assess achievement of learning outcomes after the program.
8. All participants will do a post-program reflection to assess their level of satisfaction with the program and their perceived level of achievement of the learning outcomes. Participants will not know the results of their post-tests.
9. To assess longer-term retention of content, all participants will take a post-test approximately 3 months after the educational activity.
10. All participants will also do a post-program reflection at about 3 months after the program to assess their level of satisfaction and perceived level of achievement of the learning outcomes. Participants will not know the results of their post-tests.
My sources of evidence will include:
1. Pre- and post-test scores for objective assessment of learning.
2. Reflective writings for subjective gap analysis/assessment of learning. The reflective writings can be analyzed using qualitative research methods.
3. Self-assessments by the study subjects will include quantitative measures such as the degree of accomplishment of learning objective (% accomplished or Likert scale); and qualitative answers, such as what the study subject believes what actions would be necessary to accomplish a specific learning outcome.
4. We will assess whether study subjects who do prospective reflection
a. Perform better on the post-test learning assessment
b. Have a greater degree of satisfaction with the educational program.
d. How might you make your work available to others in ways that facilitate scholarly critique and review, and that contribute to thought and practice beyond the local?
I have access to several groups who can provide scholarly critique and review:
1. As a member of the School of Pharmacy faculty, I can ask appropriate members of the faculty, in addition to my mentor, to review and comment on various aspects of the project.
2. I am a member of the School of Pharmacy’s Continuing Pharmacy Education Advisory Committee, which comprises pharmacists from various practice settings. These members would be glad to assist in the development of a reflection tool, and perhaps participate in pilot testing.
3. I work closely with the CU Office of Continuing Medical Education and am a member of the CME Advisory Committee. Dr. Ronald Gibbs, associate dean for Continuing Medical Education, has spearheaded a number of innovative practices for CME and will be glad to assist in design and evaluation of my project.
4. As chair-elect designate of the Continuing Professional Education Section of the American Association of Colleges of Pharmacy, I will have access to other academic continuing education faculty for advice and pilot testing.
e. Include a literature review of the theory and effective teaching practice of the subject of your inquiry in order to locate your research in the literature preceding it.
I conducted a limited literature search in PubMed using the search terms “reflection” and “learning.” The results included several recent publications that evaluated barriers related to achieving self-directed learning goals, best methods to promote students’ clinical reasoning skills, the relationship between reflection and mastery of skills, and a reflective learning framework within the context of continuing medical education.
1. Li ST, Paterniti DA, West DC. Successful self-directed lifelong learning in medicine: a conceptual model derived from qualitative analysis of a national survey of pediatric residents. Acad Med. 2010 Jul;85(7):1229-36.
2. Richmawati E, Wiechula R. Education strategies to foster health professional students’ clinical reasoning skills. Nurs Health Sci. 2010 Jun;12(2):244-50.
3. Branch WT Jr. The road to professionalism: reflective practice and reflective learning. Patient Educ Couns. 2010 June 4 [Epub ahead of print].
4. Leung KH, Pluye P, Grad R, Weston C. A reflective learning framework to evaluate CME effects on practice reflection. J Contin Educ Health Prof. 2010. Spring;30(2):78-88.
5. Stuart E, Sectish TC, Huffman LC. Are residents ready for self-directed learning? A pilot program of individualized learning plans in continuity clinic. Ambul Pediatr. 2005 Sep-Oct;5(5):298-301.
f. What is your record of innovation in teaching and/or the assessment of learning? As CE coordinator, I am responsible for the ACPE accreditation of the continuing education program, which involves development of learning assessment tools and processes. I have attended several ACPE training sessions related to continuing professional development. I have taught an online educational rotation several times and conducted a formal educational needs assessment with Dr. Carol Balmer, the former assistant dean for postgraduate professional education. I have seen a number of opportunities to conduct scholarly work related to effectiveness in pharmacy education, but have not previously had the opportunity or guidance to pursue such work. I would look forward to further developing my investigative skills related to educational methods and effectiveness.
g. Are you able to attend the required meetings as specified the sections titled, “What are the Benefits?” and “What commitments are expected of participants?”
I will be able to attend the meetings on September 16, 2010 and May 16, 2011, as well as meeting regularly with my mentor.
h. Can you suggest an appropriate coach/mentor for your project? I would like to work with Dr. Kari Franson, associate dean for professional education at the School of Pharmacy. In addition to her PharmD degree, Dr. Franson has a PhD in medical education and is experienced in educational research and design.
i. If your project is selected, are you willing to serve as a coach in PTLC in a future year? Yes, I would be happy to serve as a mentor.
References
1. Committee on Planning a Continuing Health Professional Education Institute. Institute of Medicine. Redesigning continuing education in the health professions. Report Brief, December 2009. http://www.iom.edu/Reports/2009/Redesigning-Continuing-Education-in-the-Health-Professions/Report-Brief-Redesigning-Continuing-Education-in-the-Health-Professions.aspx (accessed July 8, 2010).
2. Accreditation Standards and Guidelines for the Professional Program in Pharmacy Leading to the Doctor of Pharmacy Degree. Accreditation Council for Pharmacy Education. July 1, 2007. http://www.acpe-accredit.org/standards/default.asp (accessed July 9, 2010).
3. Accreditation Standards for Continuing Pharmacy Education. Accreditation Council for Pharmacy Education. January 1, 2009. http://www.acpe-accredit.org/ceproviders/standards.asp (accessed July 9, 2010).
4. Rouse MJ. Continuing professional development in pharmacy. Am J Health-Syst Pharm. 2004;61:2069-76.
