Changes Made as a Result of Assessment

The AES undergraduate committee, along with the faculty who teach core sophomore courses, identified a shortcoming in students’ understanding of fundamental experimental and computational techniques and statistical data analysis through review of lab reports and effectiveness in subsequent courses.

We determined that implementing a more direct means of lab instruction would improve students’ ability to collect and analyze data. As a consequence of this learning objective assessment outcome and through discussions at the 2011 curriculum and teaching retreat, a new class was proposed, ASEN 2012 (Experimental and Computational Methods), which was introduced into the sophomore curriculum in fall 2011.

To accommodate this additional two-credit-hour course, lab sections for ASEN 2001 and ASEN 2002 were reduced by one credit-hour per class, with the understanding that the desired data analysis and statistical methods implicitly being addressed there previously would now be more explicitly covered in ASEN 2012.

The course has now been taught five times and has gone through several iterations of content and approach based on continued evaluation by instructors. Changes have included more software development in MATLAB and added emphasis on statistical and computational methods. As implemented in fall 2012, midterm exams were replaced by project-based assignments that required the use of MATLAB to process data using the computational techniques taught in class. We established closer connection with ASEN 2001 and 2002 material by providing data collected in these labs for analysis using concepts from these two courses. Most recently, in the spring 2014 retreat, instructors voiced some concern that students perceived the lab as too much work for two credits. In response, one of three project assignments was removed in 2015 and the outcome from that change is now being assessed to find the right balance.

The course continues to evolve but is recognized as a valuable addition to the hands-on learning element of the AES undergraduate program.

Based on feedback from various engineering disciplines, particularly aerospace engineering, the department in spring 2016 added several lectures on the Laplace transform to APPM 2360 (Differential Equations with Linear Algebra). This mathematics concept was identified as an important one for later engineering courses. The department strategically placed these lectures in the final third of the semester so aerospace students would learn the concept a few weeks before encountering it again in ASEN 2003 (Introduction to Dynamics and Systems).

Example 1: Pass rates for AREN students on the Fundamentals of Engineering exam dropped below our target for a few semesters. This was a concern for our faculty, discussed by both the college's curriculum committee and the faculty at large, as well as with our Joint Evaluation Committees.  Further, when graduating seniors were asked about their perceived importance of professional licensure and the extent that they had studied for the FE exam, the responses indicated that their attitudes in this regard were likely a contributing factor to poor performance. As such, greater emphasis on the importance of professional licensure was added into the capstone design course for AREN students. In addition, students were required to take a practice FE exam as part of their grade in capstone design. In fall 2014 the department developed a website in the CU course management system, Desire2Learn, to gather review materials and example problems to help students prepare for the exam. Senior survey results have shown improved student attitudes, although performance improvements have been variable. A particular difficulty for AREN majors is that there is not a major-specific exam for them; students generally take the “other disciplines” exam, and those topics do not map well to the AREN curriculum. Discussions continue on how to improve student performance on the exam.

Example 2: After new ABET program-specific criteria for architectural engineering came into force and based on feedback from the alumni survey, senior survey, and Joint Evaluation Committees, the AREN curriculum underwent a major revision that was approved by the CEAE faculty in spring 2012. The new curriculum added a requirement for AREN students to take the first-year engineering projects course (consistent with the majority of other engineering majors at CU Boulder), removed the required Introduction to Building Construction course from the curriculum, and required students to select proficiency level courses in two sub-discipline areas of AREN (rather than requiring students to take proficiency courses in all four sub-disciplines). This new curriculum also incorporated one, 3-credit free elective and 12 credits of technical electives, consistent with the literature on the importance of student choice and autonomy based in self-determination literature. The new curriculum came into effect in Fall 2013.

One of the biggest improvements in the Department of Chemical and Biological Engineering over the last 2-3 years was the creation of a departmental Engineering Advisory Board. Recent employer survey results indicate there is significant room for improvement in areas of the curriculum. We anticipate that a strong connection between the EAB and the ChBE programs will improve areas of the curriculum, specifically those related to career preparedness.

The goal of the EAB is to "connect alumni, students, industry and faculty in a long-term relationship to support both the success of our students and the long-term success of the department." The first EAB meetings were held in spring and fall 2013. The EAB and faculty agreed that one undertaking that could have a significant positive impact is an Alumni-Student Mentor Program. Through this program, we have connected countless students with alumni, strengthening our CU ChBE community. Over the past few years, we have hosted an event each semester for alumni and students. These have ranged from a Research Symposium Series (focusing on energy, bioengineering and materials) to a Mentoring Program Meet-Up to a series of technical panels (focusing on energy and biotechnology thus far). These events have drawn many alumni and friends of the department.

Example 1: Responses to the alumni survey showed generally poor agreement that the final objective – graduates will be active in civic engagement – was appropriate and being accomplished. Feedback from the external advisory boards confirmed that this objective was vaguely stated and hard to understand. The department's curriculum committee developed revised wording for the objective. In fall 2014 the CEAE faculty debated the issue and voted to revise the objective to “graduates will be engaged in activities that provide benefit to communities.” The alumni survey responses confirm that this objective is more appropriate for our program.

Example 2: The pass rates of civil engineering students on the Fundamentals of Engineering exam has intermittently failed to meet our goal to be within 5% of the pass rates for civil engineering students nationally. The problem has appeared to worsen with the change in the FE exam to the online format with a different set of topics on the civil engineering exam. Therefore, FE review is a prominent element in a new two-credit Professional Issues course that has been added as a requirement into the civil engineering curriculum. Within this course, students take a practice FE exam, identify their weaknesses, and then review those topics with testing on quizzes. A second simulated FE exam is completed by the students and graded on performance. In all, FE exam activities comprise 25% of the course grade. In addition, there are lectures and a homework assignment that emphasize the importance of professional licensure, reinforcing the content in the first-year Introduction to Civil Engineering course. The new Professional Issues course was taught for the first time in fall 2015. In future semesters, all CVEN students are required to take the course in fall of their senior year. It is too early to tell if the curriculum change has yielded any benefits in the percentage of our students that pass the FE exam.

Example 3: A number of Joint Evaluation Committees have mentioned that they believe that the communication skills of the students are weak and the curriculum should do more to encourage improved writing skills (this includes the 2014 Construction Engineering and Management JEC and 2015 Water/Environment JEC). This topic has been discussed in CEAE faculty meetings and is now being reviewed collegewide by the Undergraduate Engineering Council. In the meantime, individual CEAE faculty are trying to increase their emphasis on written communication within their courses. This includes Professor Roseanna Neupauer using a more rigorous written communication grading rubric for laboratory reports in the required hydraulics course, Introduction to Construction, that includes a project with 30% of the grade based on writing (and includes instructor feedback on a draft of the report), and Professor Lupita Montoya including a small writing assignment within the required thermodynamics course. We hope that this infusion approach to writing-in-context may yield gains as students move through the curriculum.

Faculty observed that some computer science courses were not sufficiently teaching students basic concepts required by subsequent CS courses. To address such problems, the department created a curriculum committee, whose task was to ensure that any dependencies among CS courses are resolved by having upstream courses adequately teach key relevant concepts needed by downstream courses. For example, we noticed in CSCI 2400 (Computer Systems) that the entering students had insufficient knowledge of C pointers.  This was remedied by the curriculum committee examining the content of two preceding courses, CSCI 2270 (Data Structures) and CSCI 1300 (Introduction to Programming), and integrating sufficient teaching of pointer concepts into the curriculum prior to CSCI 2400.

Learning outcomes and student experiences from our course ECEN 4610/20 (Capstone Laboratory), were falling below expectations in 2013. This was evidenced by declining scores from the senior survey in categories of “Capstone reinforced concepts” and “Capstone prepared me for engineering career," as well as declining faculty course questionnaire scores and comments from students. The issues stemmed from poorly-defined course objectives and inconsistent mentoring from faculty members responsible for overseeing individual projects. This presented a significant concern for the ECEE department, as the Capstone Laboratory class is a critical one that provides a summative experience for students and an important assessment opportunity for faculty.

The correct this, the ECEE department assigned a new instructor to teach the course, Professor Andrew Femrite, starting in fall 2014. Professor Femrite brought a strong enthusiasm for teaching and improving student outcomes along with years of experience in industry to the course, which provided him with excellent team-building and motivational skills. These skills allow him to provide effective, broad guidance to the student teams as they work on their projects under their individual faculty mentors. The improved student motivation and overall degree of organization also helped induce faculty interest in becoming a mentor. Professor Femrite also served as an effective liaison to industry, encouraging members of the Industrial Advisory Board and other industry partners to participate in the review, and sometimes funding, of capstone projects.

The results have been very encouraging. Capstone projects are being sponsored by industry affiliates such as National Instruments; by campus organizations such as the CU Center for Environmental Technology, the CU Boulder Office of Animal Resources, and the CU RoboSub team; and by individual ECEE faculty members such as Professor Albin Gasiewski and Professor Robert McLeod. Projects undergo an extensive review at multiple stages, and well-defined objectives must be met by the students. The projects themselves are creative, interesting, and relevant. Feedback from students is now very positive, and FCQ scores for the course are strong. Finally, the Capstone Expo at the end of the academic year has quickly become a favorite among faculty and industry partners who want to interact with senior ECEE students and see the progress they have made in their projects as a culmination of their EE and ECE degree programs.

Results from the mid-curriculum and senior surveys suggested that PHYS1140 (Experimental Physics 1) required too much of students' time and was not seen as a beneficial learning experience. In addition, data collected by the physics department over the last three years indicated the course was not meeting the stated learning goals. As a result, we are beginning to completely transform the laboratory course to better match students' expectations of a one-credit course and to increase learning gains centered on experimentation. We will assess outcomes after the transformation.

At the end of each semester in each of our courses, outside facilitators conduct a course-wide, mixed-method exit group feedback session in which students work in teams to define the course strengths and suggestions for improvement. Students then individually and anonymously rate the extent to which they agree with the student-defined course strengths and suggestions for improvement. In GEEN 1010 (Engineering Explorations through Physics), qualitative investigation revealed that students were not particularly happy with the redesigned class nor with their perceived level of learning. Our process revealed 29 areas identified by students as needing improvement; privately, at least two-thirds of the students agreed with 16 of those.

For the following course implementation, many curricular and pedagogical changes were made to address the student suggestions for improvement in course flow. The goal was clear: maintain the content gains, but do so through a learning experience and process wherein students are happier about the learning experience. With myriad changes to the course, students continued to score well on the content assessment. Qualitative focus groups and interviews contrasted dramatically with those conducted during the previous offering. The end-of-semester group feedback session revealed 18 areas identified by students as needing improvement; privately, at least two-thirds of the students agreed with 11 of those — more in line with what we find for engineering courses that students appreciate. And the overwhelming feeling among students was more positive than the previous year.

Example 1: Over the years, both EVEN program alumni and graduating seniors provided significant negative feedback about the requirement to take a course in physical chemistry. However, faculty voted down a proposal to remove physical chemistry as a requirement from the environmental engineering curriculum in 2009. Going forward, the program continued to receive negative feedback on physical chemistry, in addition to calls to incorporate sustainability and life cycle assessment more prominently into the curriculum. In spring 2014 the environmental engineering faculty voted to replace the requirement for a 3-credit course in physical chemistry with a new 3-credit course in environmental sustainability. The new sustainability course contributes to meeting ABET program-specific criteria for environmental engineering. The new curriculum came in force officially in fall 2015 upon publication in the course catalog. Nearly all in-progress environmental engineering students decided to graduate on these new curriculum requirements, including a number of students in their senior year.

Example 2: When the Joint Evaluation Committee convened to review the environmental engineering program and the water/environmental engineering emphasis within the civil engineering degree in spring 2015, they determined that the environmental engineering program objectives were vague and difficult to evaluate. Subsequently, in fall 2015 the EVEN curriculum committee worked to revise the EVEN program objectives; the faculty at large voted to modify the objectives in late fall 2016. These revised objectives were presented to a new JEC in spring 2016 and found to be adequate. These new objectives will be evaluated by our alumni for the first time in summer/fall 2016. The deadline for publication in the CU catalog passed before the faculty approved the revision, therefore, the new objectives will appear for the first time in the 2017-18 catalog.

Example 3: The Joint Evaluation Committee in spring 2015 noted that having seven options in the EVEN degree may be hard to properly resource. They recommended that the number of options be contracted to four. Upon review by the EVEN curriculum committee and discussions among the faculty, it was decided to no longer require students to select an option within EVEN. Instead, the requirement to select an option with three courses in a focused area (previously: air quality, water resources and treatment, energy, chemical processing, remediation, ecology, and engineering for developing communities) was replaced with the requirement to select a design-focused environmental engineering course (from among a list of options) and two additional upper-division environmental engineering courses. This option was deemed superior as the quality of the design courses within each option varied significantly and environmental engineering career paths are rarely focused within a single “media” or concentration. This new curriculum was approved in fall 2015 prior to the catalog publication deadline, and went into force starting in fall 2016.

Data obtained from focus groups and senior surveys indicated a number of issues with the MCEN3037/4037/4047 course sequence. Specifically, students found it difficult to engage with the material covered in MCEN3037 (Data Analysis) without any means of applying it. They also noted redundancies between MCEN4037 (Measurements Lab) and MCEN4047 (Measurements 2). The department addressed those issues through a complete course redesign starting in summer 2015. The result is a four-credit course, MCEN3047, that replaces all three of the original courses and allows students to directly apply their new data analysis knowledge to the process of experimental design. While there is still room for improvement, focus groups conducted in pilot versions of the new class have been very supportive of the change. Our Industry Advisory Council echoed that support when it reviewed the course in 2016.