PBA Home > Institutional Research & Analysis > Outcomes Assessment > Unit Summaries > Ecology & Evolutionary Biology

Department of Ecology & Evolutionary Biology
Last updated fall 2004

Knowledge and skill goals for this undergraduate degree program are recorded in the most recent CU-Boulder catalog.

In some summaries of assessment activity, goals are referred to by number (e.g., K-2 is knowledge goal 2).

Assessment in 1996-1997

EPOB Senior Outcomes Assessment Fall 2004
Prepared by Dr. John M. Basey

Introduction

The purpose of this assessment is two-fold.

• To evaluate student-learning outcomes in relation to goals outlined in the course catalogue for EPOB majors graduating in Spring 2005 (Tables 1 and 2).
• To begin development of a standardized annual outcomes assessment and methods that can be used to compare senior-learning outcomes from year to year.

Table 1. A list of learning-outcome goals for EPOB majors as listed in the school catalogue. Goal numbers are included for later identification and do not indicate any rank.

Table 2. A list of science process skills for EPOB majors as listed in the school catalogue. Goal numbers are included for later identification and do not indicate any rank.

There are several factors that limit the form of the assessment.

• EPOB (EEB) has no senior capstone class for an examination to assess students, and EPOB (EEB) has no mandatory exit examinations for students. Therefore, EPOB (EEB) can only administer optional assessments during class time or outside of class. In the recent past, senior assessments given as optional outside class assignments have exhibited poor rates of return. This semester we gave the assessment in several classes with high numbers of EPOB seniors.
• Since assessments are not graded, adequately motivating students to perform well is a key concern. Therefore, assessment time must be kept to a minimum (approximately 20 minutes) and not all learning outcomes can be addressed by a single assessment. In addition, control questions that can assess student motivation in answering questions are also required.
• Since the assessment has not been standardized, a standards-based approach is not yet possible, and as an optional assignment, a standards-based approach may not be appropriate. In the long run, we need to develop an optional standards-based assessment to test learning outcomes. As an alternative, a relative assessment comparing learning outcomes across class levels (freshmen, sophomores, juniors, seniors) and between majors can at least provide information concerning progression of our majors toward the outlined learning goals.
• It is possible to get scores from students on national standardized exams such as the GRE. However, the GRE only has boxes to fill in for biology and molecular biology. Students in biologically related majors such as kinesiology (integrative physiology) environmental science and others are categorized with EPOB (EEB) majors so that mean scores do not necessarily represent our major. Therefore, at best, we can ask our students to voluntarily submit their GRE scores to the department.

Methods

Knowledge and Awareness of...
       There are eleven categories of knowledge and awareness goals listed in the catalogue (Table 1). Since the assessment was limited to approximately 20 min., all categories could not be addressed with a single assessment. Therefore, I chose to address a single knowledge-and-awareness learning outcome in a single class, and I assessed four total classes. For each learning goal, I selected three to four short-answer, generation level questions. (Ideally the assessment should be a combination of recognition and generation questions; however, with the lack of time I limited the assessment to a few generation level questions.) The four classes I assessed are listed in Table 3 with the breakdown in student numbers and characteristics. For each class assessed, the category of learning goals assessed was different than the goal of the class (Table 3). In addition, students in General Biology Lab I were also given various versions of the optional assessment to help increase sample size.

Table 3. A list of the five classes utilized for the “knowledge and awareness of …” assessment, the teacher of the class and the number of EPOB seniors, juniors, sophomores, and freshmen in the class. “Test” indicates the set of learning goals tested from the list in Table 1.

* Ecology of Invasive Plants concerns conservation biology and assessment #8 covers conservation biology. So this class acted as an assessment of a class dealing with the topic.

Since the assessment was optional, a key concern was student effort – Did the assessment actually represent student knowledge and awareness, or due to poor effort by students, was the assessment unrepresentative? To help validate, in two of the assessments I planted a control question. I took a question that students had seen earlier in the semester on their graded exams and placed it on the assessment. Then I compared the scores from earlier in the semester to the scores on the optional assessment.

For the remainder of the goals, I used a student self-confidence assessment. The assessment was comprised of three questions relating to each of the desired learning goals in Table 1. The questions used a Likert-type scale to address student confidence in their knowledge and awareness of the listed topics. To evaluate if student confidence was representative of actual knowledge and awareness, I examined the associated scores from the assessment with the related confidence questions.

Science Process Skills

The science process skills assessment was comprised of two scenarios (see Appendix 1). Scenario 1 provided students with an observation of some biological phenomenon. Students were then asked to derive a hypothesis stemming from the observation, describe methods to test the hypothesis, predict the results of the experiment utilizing standard hypothetico/deductive reasoning, and finally identify any experimental controls and explain why they were included. Scenario 2 provided students with a hypothesis, brief methods, and a data table. The students needed to log onto computers and run either a t-test or a regression analysis, appropriately graph the data and draw conclusions. Each Scenario 2 had several levels of understanding included within the question. To attain the highest level, students had to factor out the effects of a confounding variable.

Students in General Biology Lab I were the subjects of this assessment. Students were given the assessment during regularly scheduled class time as a graded 1- hour exam for class. General Biology Lab I was comprised mainly of freshmen with some sophomores, juniors and seniors. All skills needed to successfully complete the assessment were taught in class.

GRE Scores:

In an attempt to receive GRE scores that are specific for EPOB majors, I sent a mass e-mail to all of the EPOB seniors asking them to optionally report their scores to the department.

Results

Student Efforts on the Assessment

To test the efforts of students on the optional assessments, I planted questions in two of the assessments that were used earlier in the semester on exams in each class. Then I compared scores of students on the exam questions to scores of students on the assessment questions with a t-test. Results for students in both classes indicated no significant differences were present between scores on the exam and the assessment questions (Table 4).

Table 4. Means for a question planted in the end-of-semester optional assessment compared with means for the same question during an exam earlier in the semester. Both questions were 5-point generation-style questions.

Comparative, systematic, evolutionary and environmental aspects of botany, microbiology, and zoology (Goal 4 from Table 1):

Three general questions were used to assess this goal. Each question was graded on a 5-point scale totaling 15 points. The assessment was utilized in two classes: Tropical Marine Ecology and General Biology Lab I. I compared three groups: EPOB juniors and seniors, EPOB freshmen and sophomores, and juniors and seniors of other majors. A one-way ANOVA indicated a significant difference between groups (F = 12.32, d.f. = 2, 32, P < 0.001). A Tukeys HSD test indicated that EPOB juniors and seniors had a significant better understanding of the learning outcome than either of the other groups and scores for the other two groups were not significantly different (Table 5).

Table 5. Mean scores on the assessment examining learning goal #4 for three groups: EPOB juniors/seniors, EPOB freshmen/sophomores, and juniors/seniors in other majors. Significance was determined by a one-way ANOVA followed with a Tukeys HSD test.

* Indicates a significant difference from both of the other groups (P < 0.01)

Adaptations of organisms to the physical and biotic environment, from the molecular through the individual to the population level (Goal 5 from Table 1):

Three broad general questions were used to assess this goal (Appendix B). Each question was graded on a 5-point scale totaling 15 points. The assessment was utilized in two classes: Mammalogy and General Biology Lab I.

I compared three groups: EPOB juniors/seniors, EPOB freshmen/sophomores, and juniors/seniors of other majors. A one-way ANOVA indicated a significant difference between groups (F = 22.53, d.f. = 2, 39, P < 0.001). A Tukeys HSD test indicated that EPOB juniors/seniors had a significant better understanding of the learning outcome than either of the other groups and scores for the other two groups were not significantly different (Table 6).

Table 6. Mean scores on the assessment examining learning goal #5 for three groups: EPOB juniors/seniors, EPOB freshmen/sophomores, and juniors/seniors in other majors. Significance was determined by a one-way ANOVA followed with a Tukeys HSD test.

* Indicates a significant difference from both of the other groups (P < 0.01)

Since the assessment was based on questions derived from three different components of the learning goals: question 1 related to adaptations in a broad sense, question 2 related to the individual level and question 3 at the molecular level (Appendix B) -- I examined the performance of EPOB juniors/seniors between the three questions. A one-way ANOVA indicated a significant difference between groups (F = 6.88, d.f. = 2, 81, P = 0.002). The Tukeys HSD test indicated that students performed significantly lower on question 3 relating to molecular biology than they did on questions 1 and 2. In addition, student performance on questions 1 and 2 was not significantly different (Table 7).

Table 7. Mean scores for EPOB juniors/seniors between the three 5-point questions on the assessment for learning goal #5. The three questions were classified as adaptations in a broad sense (question 1), adaptations at the individual level (question 2) and adaptations at the molecular level (question 3). Significance determined by a one-way ANOVA followed with a Tukeys HSD test.

* Indicates a significant difference from both of the other groups (P < 0.01)

Molecular, organismic, developmental and population genetics (Goal 7 in Table 1)

Three general questions were used to assess this goal (Appendix B). Each question was graded on a 5-point scale totaling 15 points. The assessment was utilized in two classes: Plant Ecology and General Biology Lab I.

I compared three groups: EPOB juniors/seniors, EPOB freshmen/sophomores, and juniors/seniors of other majors. A one-way ANOVA indicated a significant difference between groups (F = 6.92, d.f. = 2, 38, P = 0.003). A Tukeys HSD test indicated that EPOB juniors/seniors had a significant better understanding of the learning outcome than either of the other groups and scores for the other two groups were not significantly different (Table 8).

Table 8. Mean scores on the assessment for learning goal #7 for three groups: EPOB juniors/seniors, EPOB freshmen/sophomores, and juniors/seniors in other majors. Significance was determined by a one-way ANOVA followed with a Tukeys HSD test.

* Indicates a significant difference from both of the other groups. Level of significance for the comparison with freshmen/sophomores was P < 0.01. Level of significance for the comparison with other majors was P < 0.05.

Since the assessment was based on questions derived from three different components of the learning goals: question 1 related to molecular genetics, question 2 related to population genetics and question 3 related to organismic genetics (Appendix B) -- I examined the performance of EPOB juniors/seniors between the three questions. A one-way ANOVA indicated no significant difference between groups (mean question 1 = 2.82, mean question 2 = 2.06, mean question 3 = 2.79, F = 1.06, d.f. = 2, 60, P > 0.05).

Conservation Biology (Goal 8 from Table 1):

Four broad general questions were used to assess conservation biology (Appendix B). Each question was graded on a 5-point scale totaling 20 points. The assessment was utilized in 3 classes: Evolutionary Biology, Ecology of Invasive Plants, and General Biology Lab I. Since Ecology of Invasive Plants focused on conservation biology, the assessments for that class were categorized separately from the other two classes. Assessments from the other two classes were combined for statistical analyses. A one-way ANOVA indicated a significant difference was present between the three groups (F = 19.05, d.f. = 2, 48, P < 0.01). A Tukeys HSD test indicated that students in Ecology of Invasive Plants had a significantly better understanding of conservation biology than EPOB juniors/seniors of the other classes, who had a significantly better understanding than freshmen/sophomores of General Biology Lab I (Table 9).

Table 9. Mean scores on the conservation biology assessment for three groups of EPOB (EBIO) majors: freshmen/sophomores in General Biology Lab I, juniors/seniors in Evolutionary Biology and/or General Biology Lab I, and juniors/seniors in Ecology of Invasive Plants. Significance determined by a one-way ANOVA followed with a Tukeys HSD test.

Results of a t-test comparing EPOB juniors/seniors to juniors/seniors of other majors in General Biology Lab I and/or Evolutionary Biology demonstrated that EPOB majors had a significant better understanding of conservation biology (EPOB –- mean = 15.09, N = 32; other majors -- mean = 11.14, N= 7; t = 2.57, P = 0.007). In addition, a comparison of EPOB juniors/seniors to juniors/seniors of MCDB, also indicated a significant better understanding of conservation biology by EPOB majors (MCDB – mean = 8.33, N = 3; t = 3.07, P = 0.002).

Science Process Skills

The science process skills assessment was designed to examine students’ abilities to: generate hypotheses to account for observations; design observational or experimental research to test hypotheses; reach conclusions based on data obtained from observations or experimental research; present in writing biological concepts, knowledge and methods (in this case making appropriate graphs). For this assessment, EPOB majors in General Biology Lab I were examined during the regularly scheduled class period with a 30-point examination. We compared science process skills of EPOB juniors/seniors to those of freshmen/sophomores with a t-test. The juniors/seniors demonstrated significantly better abilities with science process skills (mean frsh/soph = 20.45, mean jr/sr = 25.17, d.f. = 47, t = -3.26, P = 0.001).

A one-way ANOVA comparing the scores of EPOB seniors in relation to the 4 learning goals demonstrated no significant differences between learning goals (F = 1.43, d.f. = 3, 32, P > 0.05; Table 10).

Table 10. Mean % scores for EPOB juniors/seniors on the four categories of the science process skills assessment. A one-way ANOVA indicated no significant differences between skills (P > 0.05).

Student Confidence in Their Abilities:

In each class, students were given an assessment asking them to rate their confidence in their knowledge and awareness of all learning goals. I compared student confidence scores with their actual performance scores on the assessments with a linear regression analysis. Results indicate a significant positive relationship was present for three out of the four learning goals examined (Table 11); however, for conservation biology, the perception of students about their abilities was not significantly determined by their performance on the assessment (Table 11).

Table 11. Results of regression analyses comparing students confidence in their knowledge and awareness of the various learning goals in Table 1 with their performance on the assessment.

I used a sign test to evaluate the confidence of EPOB seniors for each goal. To do the sign test, for a given category, I summed the number of responses that were greater than 3 as negative responses (poor confidence in abilities) and responses less than 3 as positive responses (strong confidence in abilities). Values of 3 were not used in the analysis. The sign test indicated that EPOB students had a significant positive confidence in their knowledge and awareness of all learning goals except for learning goal #4 – comparative, systematic, evolutionary and environmental aspects of botany, microbiology, and zoology (Figure 1). In addition, I used an ANOVA with a Tukeys HSD test to evaluate differences in mean scores of the students on the various learning goals. Results indicate that students displayed significant differences in their confidence between the learning goals (F = 30.97, d.f. = 9, 950, P < 0.0001; Figure 2). The Tukeys HSD test indicated significant differences between groups (Figure 2). I also compared the averages of EPOB juniors and seniors to those of the four MCDB students who were assessed and students of other majors. Results indicated that on average, in all but two categories, the values for mean confidence was stronger for EPOB students than for the other two groups (Figure 3).

Figure 1. Results of the sign test comparing the number of positive responses (< 3) and negative responses (< 3) relative to their confidence in their knowledge and awareness of the various learning categories. The sample was EPOB juniors and seniors. Learning goal numbers refer to numbers listed in Table 1.
*** indicates P < 0.01.

Figure 2. Average confidence scores of EPOB juniors and seniors in relation to the learning goals listed in Table 1. Significance was determined by a Tukeys HSD test.
** indicates a significant difference from all other learning goals (P < 0.05). Other learning goals were significantly different at approximately every other bar.

Figure 3. Average confidence scores of EPOB majors, MCDB majors and other majors in relation to learning goals listed in Table 1.

GRE Scores:

Only two students reported their GRE scores. So I did not do any further analyses.

Discussion

Results of this assessment reveal several important findings.

• An optional assessment of upper division classes at the end of the semester can get adequate participation and effort from students for a proper evaluation (Table 4).
• In all areas of knowledge and awareness assessed (4 out of 10 with overlap) information indicates that classes in EPOB and the current curriculum are significantly improving student learning outcomes when compared with freshmen/sophomores in EPOB and juniors/seniors in other majors including one minor comparison with MCDB majors (Tables 5 – 9). One potential area for improvement; however, may be in molecular biology (Table 7).
• Science process skills assessed indicate significant improvements by juniors/seniors over freshmen/sophomores. Due to a small sample, no differences between science process skills were elucidated. It is possible that some science process skills may need more instructional investment such as experimental design, data analysis and graphing (Table 10).
• A confidence assessment given once per year has the potential of being a valuable tool in determining changes and deficiencies in the curriculum; however, the assessment must be verified for all 10 categories of learning goals (Table 11). The confidence assessment indicated a potential weak area in the EPOB curriculum concerns comparative, systematic, evolutionary and environmental aspects of botany, microbiology, and zoology (Figures 1 and 2). Further exploration of this potential deficiency is warranted. In addition, confidence of EPOB students relative to the sample of MCDB students and other majors is strong (Figure 3).
• The major weakness of this assessment is the lack of verification of the assessment tools with multiple internal and external reviewers. In the future, we need to develop more of a standards-based assessment tool that we can use to compare our student’s learning outcomes to those of students outside the university.

PBA: L:\ir\Outcomes\OA0405\EBIO.doc

Last revision 12/28/05