2014 Research Projects
Darcy Solanyk, MS, PA-C
University of Colorado Denver
School of Medicine
Department of Pediatrics
Child Health Associate/Physician Assistant Program
Mail Stop F543
13001 East 17th Place
Aurora, CO 80045
Development of a Valid and Reliable Evidence-Based Practice Curriculum Assessment Tool
3a. What is the central question, issue, or problem you plan to explore in your proposed work?
Do the attitudes, knowledge and skills of PA students in the area of evidence-based practice (EBP) progress after taking a one semester Evidence Based Medicine (EBM) Course? Do the students also progress in these areas from the point of matriculation to completion of the three years of the Child Health Associate/Physician Assistant (CHA/PA) program?
b. Why is your central question, issue, or problem important to you and to others who might benefit from or build on your findings?
As core faculty within the CHA/PA program and co-course director of its Evidence Based Medicine course, it is important to know if the information conveyed results in effective learning and retention of key concepts and skills in the area of evidence-based practice. Not only is it important to know if the class is effective, but being proficient at EBP is essential to being a high quality healthcare provider. The CHA/PA program needs to know if our students are competent healthcare providers and if the curriculum we teach is equipping them adequately to practice in the current healthcare system. Evidence, or Practice-Based Medicine is one of the key competencies in medical training, and therefore an important program outcome measure.
c. How do you plan to conduct your investigation? What sources of evidence do you plan to examine? What methods might you employ to gather and make sense of this evidence? What literature have you reviewed on your topic?
In order to ascertain whether or not the EBM course and the CHA/PA curriculum improve attitudes, knowledge and skills around evidence-based practice, I plan to take the following approach: I will consult with an expert in developing a reliable and valid questionnaire. There are currently two well-tested questionnaires in use - one questionnaire is used to measure thoughts and attitudes about EBP. The second questionnaire, the Fresno Test for Evidence Based Medicine, has been found to be valid and reliable when measuring knowledge and skills around EBP . By effectively combining the two questionnaires, I can get at an overall picture of the attitudes, knowledge and skills in EBP of the PA students. There is currently no single valid tool in use that measures each of these important components.
There have been several instruments developed to evaluate the attitudes around EBP. There are two under consideration for merger with the Fresno Test. The Knowledge, Attitudes, Access and Confidence Evaluation (KACE) and the Evidence-Based Practice Attitude Scale (EBPAS-50) have demonstrated validity in assessing self-efficacy (student’s judgments regarding their ability to perform a certain EBP activity) and attitudes around EBP. KACE has been used with dental students and faculty. EBPAS-50 has been used with mental healthcare workers and social service providers. My goal is to identify which components of the current tools might be combined to produce a valid new tool that would be best suited for use in measuring attitudes, knowledge and skills of PA students...
The Fresno Test is one of only two questionnaires that fully, reliably and validly evaluates the knowledge and skills of all areas of EBP. Rarely has this test been utilized to evaluate the effectiveness of a longitudinal curriculum. The few times it has been utilized, it has been found to be valid and reliable. Although it adequately assesses the knowledge and skills of students, it does not evaluate the attitudes surrounding EBP, and as the literature shows, attitude frequently drives behavior.
Once I have created a questionnaire that can evaluate the attitudes, knowledge and skills of PA students in regard to EBP, I plan to administer it at four different times throughout the three year curriculum. I will administer the questionnaire at matriculation, as a pre-test to the EBM course, as a post-test for the EBM course and at the end of the three years prior to graduation. By establishing multiple points of evaluation within the curriculum, I can more adequately evaluate the impact of the EBM course itself and the impact of the PA curriculum as a whole in teaching our students how to conduct EBP. If I can see a progression of the attitude, knowledge and skills of the PA students in the utilization of EBP while in our program, then they are more apt to retain those behaviors and skills once they are out in clinical practice.2 Retaining such skill sets will make them more effective clinicians that will better serve their patients and communities.2
d. How might you make your work available to others in ways that facilitate scholarly critique and review, and that contribute to thought and practice beyond the local? (Keep in mind that coaching will be available to invite you to develop these aspects of your proposal, so you need not feel you must present a finished project design at this time.)
Once I have tested the newly developed questionnaire and found it to be valid and reliable, I would be interested in sharing it with other programs on the Anschutz Medical Campus who utilize EBP in a longitudinal fashion within their curriculum. I would also like to present the data from the questionnaire to the national PA Education Association for other PA programs to evaluate and critique. Hopefully other programs would be willing to further evaluate the questionnaire’s validity and reliability within their own settings. There is currently no tool in use within other PA programs. Because Practice-Based Medicine is a competency measured by most health care training programs, a valid tool could be useful across multiple disciplines.
e. Include a literature review of the theory and effective teaching practice of the subject of your inquiry in order to locate your research in the literature preceding it. (The website http://www.colorado.edu/ptsp/ptlc/libraryresources.html offers expert advice on how to conduct a relevant literature review.)
Evidence-based practice may be defined as the integration of the best research evidence with patients’ values and clinical circumstances in clinical decision-making.5 EBP is an essential quality in a good provider not to mention it involves concepts that many accrediting bodies require being taught to health professions students. There are over 104 unique assessment strategies for evaluating EBP in learners.5 Most of these assessment tools are utilized in medical students (undergraduate and postgraduate trainees) or practicing physicians. Few (13 out of 104) are utilized in non-physician professions5 and only a small number (27 out of 104) evaluate the attitudes around EBP.5 Only two effectively and objectively evaluate the knowledge and skills involved in EBP—the Fresno Test and the Berlin Questionnaire.5 No questionnaire or survey has been found to assess the attitudes, knowledge and skills of future healthcare providers, especially non-physician trainees.
f. What is your record of innovation in teaching and/or the assessment of learning?
My record of innovation in teaching over the past three years has been mostly in regard to the Evidence Based Medicine course. I have expanded the course curriculum to not only teach EBP concepts but to also include sessions on Information Mastery that help students learn to navigate the literature effectively at the point of care. I have also implemented interactive sessions with students within the course including case-studies from the Centers for Disease Control and Team Based Learning sessions to enhance concepts.
g. Are you able to attend the required meetings as specified in the sections titled, “What are the Benefits?” and “What commitments are expected of participants?”
Yes, I am able and excited to attend the required meetings and complete the expectations as specified in the application. I have the full support of the CHA/PA Program Director as well as an identified mentor.
Name and contact information for a mentor/Coach for your project?
My mentor will be Cathy Ruff, MS, PA-C. Ms Ruff is the Associate Director of Curriculum within the CHA/PA Program and has over 10 years of experience in PA education. Her expertise is in curriculum development and assessment. Email: cathy.ruff @ucdenver.edu; Phone: 303-724-1345.
If you are selected, we ask you to agree to serve as a Coach in PTLC in a future year.
I would be happy to serve in this capacity in the future.
1. Ramos KD, Schafer S, Tracz SM. (2003). Validation of the Fresno Test of competence in evidence based medicine. BMJ. 326 (7384), 319-321.
2. Tilson JM, Kaplan SL, Harris, JL, Hutchinson A, Ilic D, Niederman R, ….&Zwolsman SE. (2011). Sicily statement on classification and development of evidence –based practice learning assessment tools. BMC education, 11(1), 78.
3. Hendricson WD, Rugh JD, Hatch JP, Stark DL, Deahl T, Wallmann ER. (2011) Validation of an instrument to assess evidence-based practice knowledge, attitudes, access and confidence in the dental environment. JDentalEduc, 75 (2): 131-144.
4. Aarons GA, Cafri G, Lugo L, Sawitzky A. (2010) Expanding the Domains of Attitudes Towards Evidence-Based Practice: the Evidence Based Practice Attitude Scale-50. Adm Policy Ment Health 1-10.
5. Shaneyfelt T, Baum, KD, Bell D, Feldstein D, Houston T…& Green M. (2006) Instruments for Evaluating Education in Evidence-Based Practice: A Systematic Review. JAMA 296 (9), 1116-1127.
6. West CP, Jaeger TM, McDonald FS. (2011) Extended Evaluation of a Longitudinal Medical School Evidence-Based Medicine Curriculum. J Gen Intern Med 26 (6): 611-15.