Implementation

Course Re-design to Address Assessment Alignment after Course Scale-up – Summer 2018

The Discovery Labs aim to provide students with an authentic research experience that incorporates design, presentation, and defense of experiments and resulting data. The initial goals of the course were ambitious for first-year students, and after scaling up the course, I decided that focusing on experimental design, data analysis, and scientific communication would best serve the diverse cohort of students enrolling in The Discovery Labs. Significant challenges associated with assessing the research gains of students led to refocusing the curriculum to better align with the course goals. I redesigned Discovery Labs I and II in summer 2018 with the guidance of a University of Colorado Faculty Teaching Excellence Program (FTEP) Assessment Summer Institute Assessment. I aimed to begin preparing the students for the CURE Symposium earlier in the semester by formally assigning components of the project with feedback provided for each step to guide students through developing, presenting, and defending their work (Table 1):

1. Proposal & Plan – Groups submit a description of the proposed research using a rubric. The proposal describes a plan for students’ independent projects that uses the experimental approach utilized in the drug screen performed earlier in the semester.
2. Peer Review – Groups submit reviews of two other projects (author names removed) and receive peer review from another group. Peer reviews are completed using guidelines.
3. Instructor Meeting - Group members meet with instructor to discuss the details of the research plan. The peer reviews becomes part of the discussion in instructor meetings with the groups. All groups receive written feedback from the instructor and copies of their peer group as reference and to provide ideas. Additional detail is developed during the meeting using a rubric.
4. Introduction Section of Poster Draft – Groups submit a draft of the introduction section of their posters. Drafts are produced using guidelines and are edited by instructors.  Members of the group are emailed the edits along with feedback about the progress of the Introduction section.
5. Methods Section of the Poster Draft - Groups submit a draft of methods section of their posters. Drafts are produced using guidelines and are edited by instructors.  Members of the group are emailed the edits along with feedback about the progress of the Introduction section.
6. Data Reporting & Interpretation – One recitation section is dedicated to data reporting ethics and presentation along with a module quiz submitted through Canvas.
7. Future Directions – What should be done immediately to get reliable answers (achievable in Discovery Lab)?  What should be done in the longer term to move the compound toward drug development (not achievable in Discovery Lab)? An optional draft can be submitted for feedback. Guidelines are loosely provided to encourage independent thought and creativity.
8. Poster Design – A PowerPoint tutorial is provided in recitation and in lab and checklist for final design of the poster.
9. Poster Oral Presentation – Demonstrations are provided by former Discovery Lab students during the recitation section.

PART I. Oral Presentation Peer Evaluations

Scientists are required to effectively communicate and defend their work to a variety of audiences. As such, the semester culminates in the CURE Symposium during which students present their work to small groups in a poster session that is attended by nearly 800 and includes instructors, teaching assistants, MCDB professors, and student peers. While the research is completed in groups of 3-4 students, posters are presented individually so all students are responsible for understanding the content. Posters are presented in one of two poster sessions that are 40 minutes each.

Posters are evaluated by 2-3 instructors or teaching assistants and members of 3-4 posters from other courses. An average of 11 reviews for each poster are received from electronic submission of a Qualtrics Survey. This anonymous link can be accessed multiple times from the same machine (laptop, mobile, tablet). Students are required to complete all fields (10 total), which include indicating the poster number and last name of the first author on the poster evaluated and their own name and poster number.

1. Spoke clearly, confidently, comfortably, made eye contact (0-4)
2. Presentation tailored for the audience (0-4)
3. The research aims were made clear in the context of previous research (0-4)
4. Expected outcomes were discussed (0-4)
5. Methods were clearly explained (0-4)
6. Results were clearly explained (0-4)
7. Conclusion was discussed including potential impact of the project (0-4)
8. Overall assessment (0-4)
9. General comments (required – 50 character minimum)
10. Suggestions for improving poster and presentation (required – 50 character minimum)

PART II.  Student Reflection on the CURE Symposium

In spring 2019, forty-nine students were asked to reflect on their experiences at the CURE Symposium. They were asked to respond to the following prompts: 1) What challenges did you encounter while preparing for the CURE Symposium? 2) Describe your general experience at the CURE Symposium, and 3) Describe the value of the CURE Symposium in terms of your development as a scientist. These prompts were chosen to gain information important to various stakeholders in the Symposium including the students themselves and funding sources. Submissions were one-page, double-spaced documents that were completed in the three days following the Symposium. Responses to questions 2 and 3 were groups for evaluation because students tended to answer these two questions together.  After reviewing all submissions, categories were created that captured common themes and then blindly quantified:

Question 1 - Challenges:

1. Not enough time or resources to complete the desired experiments,
2. Not enough time to think about the results or significance of the project in the context of previous work,
3. Project plan changed or progressed in an unexpected manner,
4. Inconclusive or unexpected data were obtained,
5. Felt nervous/stressed/anxious about possible questions and not being prepared to answer,
6. Problems associated with group work. 

Questions 2 & 3 - Experience at the CURE Symposium:

1. Gained experience in public speaking and presentation of scientific data,
2. Learned about other classes or research that could be pursued,
3. Professional, inviting, or comfortable environment,
4. Feelings of ownership over project and pride in completion or presentation,
5. Gained transferrable skills like making and presenting a scientific poster and communicating science to the public,
6. Evaluation of others’ work helped with understanding their own project,
7. Presentation of poster helped with understand their own project.

Back (Background)           Next (Student Work)