The Undergraduate Program Continuous Improvement Policy [1] specifies the procedures that the department follows to ensure continuous improvement of its undergraduate degree programs. The department recognizes that in order to provide a high-quality program that meets the needs of its undergraduates, it must make use of multiple forms of feedback and assessment to collect the data it needs to evaluate the current status of the program and to make sound recommendations for improvement.
To guide this process, the department has established a set of program outcomes and program objectives for the BS degree program. Outcomes are statements made about the skills and capabilities of senior BSCS majors at the time they graduate. Objectives are statements made about the skills and capabilities of Computer Science alumni approximately three to five years after they graduate. The truth of these assertions is established via the assessment and evaluation mechanisms described by this policy.
The department’s website is used to make all continuous improvement-related documents available to the public including listing the outcomes and objectives, the continuous improvement policy and the annual report. An archive of past versions of these documents is also maintained along with a summary of the basis for any changes made.
Two committees, the Undergraduate Committee and the Curriculum Committee, are charged with designing and maintaining the courses and degree requirements for the program in such a way as to ensure the program’s outcomes and objectives will be met. Duties related to the task of continuous program improvement for each of these committees is described in this document.
The department’s Continuous Improvement Policy is based on the following processes:
These process are based on the use of the following:
This information is then used by the Faculty, and the Curriculum and Undergraduate Committees to make changes in the program to address problems and improve quality.
Assessment of Continued Validity of the Program Outcomes and Objectives
Each year, during the fall semester, the Curriculum Committee reviews inputs from the following sources in order to assess the continuing validity of the Program Outcomes and Objectives:
If, after reviewing these inputs the Curriculum Committee determines there is a need to modify either or both of the outcomes and objectives they will draft proposed changes and present them to the faculty for review. The Curriculum Committee and Faculty then iterate on these until an approval is granted by the faculty at which time the revised outcomes and objectives are reposted to the department’s website.
During the final year of an ABET cycle the program outcomes and objectives are further reviewed by the department’s advisory board. Any suggested modifications are then fed into the regular department review cycle.
Assessment of Student Attainment of Program Outcomes and Objectives
The department faculty define success criteria for students to meet with respect to both the Program Outcomes and the Program Objectives. Student success is assessed annually.
Program Outcomes are assessed in terms of successful completion of specific coursework based on Program Outcome Success Criteria adopted by the faculty.
All students graduating from the BSCS program must have been exposed to course work that provides and understanding of all ABET criteria. The department’s curriculum is designed to ensure meeting this ABET requirement through the foundation courses and by requiring that all students graduating pass either CSCI 4308/4328/4338 or CSCI 3100, in that case that a non-project capstone option is taken by the student. The foundation courses require student achievement of all criteria except for the professional development related criteria 3.e and 3.g. The senior project options and CSCI 3100 provide coverage of those topics with the exception of security, which is covered in CSCI 2400.
Because of the importance of ensuring adequate coverage of topics to ensure meeting ABET criteria the content and delivery of the foundation courses, CSCI 4308/4328/4338 and CSCI 3100 are closely monitored and controlled by the Curriculum Committee. If a student fails to successfully complete one of these options they must either repeat that course or elect another capstone option and complete that during the following academic year.
For students electing to complete the CSCI 4308/4318 capstone option an additional review of their achievement of the program outcomes is conducted via a survey completed by project sponsors during the Sponsor Post-Mortem in May of each year.
In lieu of an exit exam, the Department will review the achievements of select students on assessments designed to measure student achievement of the program outcomes. The students will be selected at random from each of the capstone options in numbers proportional to the number of students taking that capstone.
After graduation each year student success in achieving the faculty defined criteria for student success in achieving the Program Outcomes is assessed based on achievements by a randomly selected subset of students.
Student success in achieving the Program Objectives is assessed via the Alumni Survey annually and the Employer Survey when available. These surveys are administered by CEAS. The Alumni Survey is run annually and sent to students 3-5 years after graduation. The Employer Survey is run every 3 years.
Upon completion of the success study each year the Curriculum Committee provides a summary report to the faculty for review, which then reviews student success and any plans for improvement. Upon approval by the faculty the results are published in the department’s annual report.
Course Assessment Policy and Process
All undergraduate Computer Science courses (with the exception of special topics courses, independent study courses, and courses for non-majors) are required to make publicly available on a semester-by-semester basis the course profile.
The Curriculum Committee will review the course profile for all foundation courses annually.
Course Profile
The course profile includes the following information:
The course profile is used to track important information about a course and represents a contract as to what is taught in the course and what the department can expect from students that successfully complete the course.
The Curriculum Committee must review any changes to a course profile before these changes become official. This review ensures that consistency across course offerings is maintained and that any changes related to course outcomes do not adversely affect courses dependent on those outcomes. All externally visible changes must be announced to the faculty of the department.
Faculty Course Questionnaires
Faculty Course Questionnaires (FCQs) are administered by the University every term for each instructor of each class taught. FCQs allow students to rate their instructors and courses and provide information to instructors for teaching improvement, to students for course selection, and to deans and chairs for management. Detailed FCQ results are available on the CU Faculty Course Questionnaire website, while summaries of FCQ results for Computer Science courses are available at Computer Science Faculty Course Questionnaires website.
The Curriculum Committee will review the FCQ ratings for all courses in October and February of each year. For any courses for which the ratings fall below a 4.0 the faculty member who taught the course will be required to submit a report documenting the reasons for the low rating and suggesting an improvement strategy.
Course Archives
To assist in the department's goal of improving the quality of its undergraduate program, the Curriculum Committee is adopting a uniform policy with respect to the collection of course materials from undergraduate courses into course repositories. This information is also useful for maintaining high-quality course dossiers that must be presented to ABET reviewers during an ABET site visit.
Course repository is maintained either on moodle or in bitbucket. It must include the following items:
These materials must be archived within 1 month of the close of the semester in which the course was taught.
Because of the different nature of the courses, Senior Thesis and Project courses follow different guidelines.
Senior Thesis
The final thesis document of an undergraduate senior thesis must be submitted to the Main Office Administrator in electronic form to be stored online.
Project Team-Based Courses
In project team-based courses, artifacts such as documents and source code often undergo multiple changes throughout the semester. Rather than collecting each version of each artifact, the course coordinator should collect only the completed project for each team for archive. Projects completed by three project teams -- representing low, medium, and high performing teams -- should be submitted.
In some sponsored project courses it is not possible to collect all project artifacts. At a minimum the archive of a project must contain sponsor evaluations, peer evaluations, team advisor evaluations, team posters, and project summaries, and class presentations.
The Curriculum Committee is responsible to ensure that materials are collected for each instance of an undergraduate course and to assist in the process of collecting material from each course that was taught during the previous semester. In particular, a reminder asking for course materials will be sent at the end of the last week of classes. A second reminder will be sent at the start of the first week of classes of the next semester. Finally, a third reminder will be sent starting on the third week of the semester after a course has been taught, with weekly reminders sent until the required materials have been acquired.
Ensuring Foundation Courses Stability
In order to ensure that all ABET criteria are satisfied by BSCS graduates stability of the content and delivery of the foundation courses, CSCI 4308/4328/4338, and CSCI 3100 must be assured from semester to semester. In order to achieve this goal, faculty members are provided with the courses syllabus two months before the beginning of each semester. If a faculty member wishes to change the course outcomes or textbook the suggested changes must be reported to the Curriculum Coordinator. If the course is a foundation course or CSCI 4308/4328/4338 or CSCI 3100 the suggested changes must be reviewed by the Curriculum Committee, which then will iterate with the faculty members until agreement is reached. When the course changes are approved the syllabus is revised, the old version is archived for future reference, and both are made publicly available on the department’s Courses web page.
Surveys
Each year, the College of Engineering and Applied Science administers a number of surveys to collect information from various college stakeholders – students, alumni and employers – which can be used to assess the quality of the College's departments and programs. These surveys include the
The Curriculum Committee will make use of the data from these surveys as one point of independent feedback concerning the quality of the BS degree program and whether or not the goals of the program as stated by the program outcomes and program objectives are being met.
In particular:
Advisory Review Board
The board's job will be to provide external evaluation and feedback on the Department's goal of continuously improving its undergraduate program.
[1] Revised Feb 2014, Apr 2015, Nov 2015