Ecology and Evolutionary Biology
Course Context: General Biology Lab for science majors, freshman/sophomore level, 800 students, 24 graduate-student TAs. Laboratory exercises utilizing an investigative, hands-on, active-learning approach emphasizing the process of science — hypothesis testing through experiments with data and analysis.
Target of Improvement: In lab, students often resort to procedure-following protocols and often do not get the big threshold concepts because they focus too much on details. The big question for this investigation is -- Can reflection help improve student learning and/or lead to better lab design?
Methods (What was done?): In General Biology Lab II, 24 TAs teach each teach 2 sections of 18 students. In spring 2010, I chose two labs to use for the reflection assignment: vertebrate dissection and exercise physiology. Each TA chose one of their two lab sections to do the reflection assignment for the vertebrate dissection lab and with the other lab section they did the reflection assignment for the exercise physiology lab. The reflection assignment was done by students outside class on CU learn and consisted of the question -- Describe the three most important concepts you learned from the Exercise Physiology/Vertebrate Dissection Lab and explain your choice. I assessed outcomes in two ways: an attitude survey (Appendix A) and an evaluation of the reflection assignment. To evaluate the reflection assignment, I examined student answers and developed a table with content categories. I then examined student responses and tallied responses in the table. Each student had three responses.
Results (What Happened?):
512 out of 825 enrolled students completed the end-of-semester survey. I compared student ratings from the categories of overall rating, how much the lab helped with lecture material and how much the students learned for the time they invested, for students who had the reflection assignment incorporated vs. those who did not. A t-test demonstrated no significant differences were present between students who did the reflection assignment vs. those who did not do the reflection assignment for any of the rating categories in both lab categories (P> 0.05; Table 1).
Table 1: Average student ratings (one standard error) on a 10-point scale with 10 as the highest. Students were asked to: provide their overall rating of the lab (rating), rate the lab on how much it helped with lecture (lecture help), and rate the lab on how much they learned for the time they invested (time efficiency). Refl. = students who did the reflection exercise and No Refl. = students who did not participate in the reflection exercise. Two labs were compared: (1) Vertebrate Dissection and Exercise Physiology. Sample size is as follows: dissection -- reflection = 184, no reflection = 328, physiology reflection = 182, no reflection = 326. No significant differences were present between the reflection and no reflection treatments in any category (P > 0.05).
|Refl.||No Refl.||Refl.||No Refl.||Refl.||No Refl.|
|Vert Diss.||7.99 (.12)||7.90 (.10)||6.74 (.18)||6.82 (.14)||7.61 (.13)||7.42 (.11)|
|Exercise Phys.||6.70 (.16)||6.84 (.10)||5.73 (.19)||5.71 (.14)||5.86 (.18)||6.08 (.12)|
The end-of-semester survey was also evaluated to see whether students thought the reflection had a direct positive influence on their learning experience. For this part of the survey, four questions were included that utilized a 5-point, Likert-type scale, ranging from strongly agree, agree, neutral, disagree to strongly disagree. A one-sample t-test with 3 as the comparison mean indicated that students significantly agreed the reflection assignment positively impacted their learning (Figure 1). I finally combined the reflection and no reflection ratings for each category and compared overall ratings of the combined Vertebrate Dissection labs vs. the Exercise Physiology lab with a t-test. Students rated the combined Vertebrate Dissection labs significantly higher than the Exercise Physiology labs in every category (Table 1, P < 0.001).
The analysis of student answers on the reflection assignment revealed several categories constituting students’ perceptions of their own meaningful learning (Table 2, 3). To evaluate how well the two lab-styles improved conceptualization of big picture themes (higher-order cognition) vs. knowledge and comprehension (lower-order cognition), I categorized answers students provided in the reflection assignment into various levels of learning according to Bloom’s Taxonomy (Bloom et. al. 1956). I then compared the number of higher-order answers to lower-order answers in the Exercise Physiology Lab vs. the combined Vertebrate Dissection labs with a chi-squared test (Table 4). The chi-squared test indicated no significant differences were present in higher-order vs. lower-order answers between the types of labs (P > 0.05).
Figure 1. Average net agreement for the 4 survey questions associated with the reflection assignment. Students were asked to indicate whether they strongly agreed, agreed, were neutral, disagreed, or strongly disagreed. On the figure, 3 = neutral, < 3 = students agreed the reflection was a valuable learning experience. The 4 questions were. (1) By reflecting on the 3 most important concepts I learned, I was able to improve my overall learning experience in the lab. (2) The reflective assignment was busy-work and did not improve my learning. (3) I would like a reflection assignment like this as a graded component of several labs during the semester. (4) Pondering about the lab I just completed does not help me integrate the information or clarify the big-picture concepts in the lab. A one-sample t-test with 3 as the comparison mean indicated students significantly agreed that the reflection was a positive learning experience (P < 0.001). Error bars represent one standard error.
Table 2. A distribution of student answers for the following question. Describe the three most important concepts you learned from the Exercise Physiology Lab.
|Level of Learning||Category||Number of Students||% of Students|
|Higher-Order||Conservation of energy.||23||10|
|Lower-Order||Long-term effects of exersise.||38||17|
|Lower-Order||What is blood pressure? How is is calculated? How is it influenced by exercise? How is blood pressure measured?||43||19|
|Lower-Order||High individual variability.||3||1|
|Application (lab skill)||Measuring blood pressure.||45||20|
|Application (lab skill)||Use of Excel to run statistics.||7||3|
|Application||Physiology is mathematically described.||7||3|
|Miscellaneous||Interesting tidbit of information.||8||4|
Table 3. A distribution of student answers for the following question. Describe the three most important concepts you learned from the vertebrate dissection labs.
|Level of Learning||Category||Number of Students||% of Students|
|Higher-Order||Form follows function.||14||8|
|Higher-Order||Systems are comprised of integrated components.||15||8|
|Higher-Order||Different types of organism have analogous systems.||11||6|
|Higher-Order||Different types of organism have special adaptations.||12||7|
|Higher-Order||Conservation of Energy.||4||2|
|Higher-Order||Variations in individual anatomy.||1||1|
|Lower-Order||Heart and Circulation.||31||17|
|Application||Seeing and feeling helos cognition.||28||16|
|Application (Lab Skill)||Dissection Techniques.||7||4|
|Miscellaneous||Interesting Tidbit of information.||7||4|
Table 4. Number of higher-order vs. lower-order student answers for the three most important aspects learned in either the Exercise Physiology lab or the combined Vertebrate Dissection labs. A chi-squared test revealed no signifitcat differences were present betweent he two types of labs (chi-squared cal = .001, P>.05)
|Level of Learning||Exercise Physiology||Vertebrate Dissection|
Discussion (What was learned?)
When students evaluated the reflective assignment relative to differences between labs (content and style), the reflective assignment did not impact their overall learning experience (rating), their learning of lecture material (lecture help), or their overall learning efficiency (time efficiency; Table 1). However, when students evaluated the reflective assignment by itself, they significantly agreed that the reflection improved their learning and helped them integrate the big picture ideas of the lab (Figure 2). The 3 most important concepts learned were variable and ranged from integrated themes (higher-order cognition), to knowledge and conceptualization (lower-order cognition) to applied learning (lab skills; Tables 2 and 3). Two lab styles were utilized: problem-based and expository observational. Theoretically, the problem-based lab style should focus more on higher-order cognition than the expository observational (Domin 1999). Therefore, I expected to see a higher percentage of students indicating higher-order cognition compared with lower-order cognition for their three most important concepts learned in the Exercise Physiology lab than in the combined Vertebrate Dissection labs. The Χ2 test revealed no significant differences were present between higher and lower-order cognition answers for the two lab styles (Table 4). In both lab styles, students indicated lower-order cognition more often than higher-order cognition. Therefore, overall, students may prize learning lower-order cognitive information more so than higher-order cognition. This is supported through the significantly higher overall ratings of the Vertebrate Dissection lab compared with the Exercise Physiology lab. However, again it should be noted that even though the focus was different between the problem-based and expository observational lab designs, students still indicated approximately equal ratios of lower to higher-order cognition in the reflection. Interestingly, even though the expository observational style of the Vertebrate Dissection labs focused on knowledge and conceptualization, students were integrating the information.
Extensions (What is next?)
These results indicate that reflection does have potential to improve learning. In this experiment, the reflection was added on as an additional, un-graded, post-lab assignment. The next step is to integrate reflection into the normal graded lab reports associated with the labs. Currently, learning in the Vertebrate Dissection labs is assessed only by a post quiz. The addition of a reflective assignment focusing students on integrating the information may be even more valuable than with other labs that already have write-ups. An examination of how a graded reflection assignment influences student learning and attitudes would be very interesting for future applications.
Dissemination (Sharing with colleagues?)
The collaborative nature of the assessment workshops with three other department members has opened doors for communication within our department. In addition, we currently have a 9-member committee overseeing General Biology lecture and lab. We meet monthly to discuss pedagogy and teaching improvements, I plan to discuss this project in one of our monthly meetings. In the past, our lead graduate TA has set up brown bag seminars on teaching, this project has the potential for the topic of one of the brown bag seminars. For the larger CU community, I will be attending the weekly DBER seminars next year and I may be able to discuss this project in one of the seminars.
Bloom, B.S., Engelhart, M.D., Furst, E.J., Hill W.H., and Krathwohl, D.R. (1956). Taxonomy of educational objectives: Part I, cognitive domain. (New York: David McKay Company Inc.).
Domin D.S., (1999), A review of laboratory instruction styles, Journal of Chemical Education, 76, 543-547.