[Comments? Questions?] [CU-Boulder] [PBA] [Outcomes Assessment]
Assessment Oversight Committee
Assessment for Undergraduate Units: A Brief Guide - DRAFT
Prepared by the Assessment Oversight Committee, Spring, 2001
"It has been said that universities have three undergraduate curricula: The one that appears in the catalog, the one that professors teach, and the one that students actually learn. To what degree does the curriculum asserted on paper or imagined by deans and dons accurately portray what goes on in the minds of students? Making the three curricula visible so they can be brought into register is the business of assessment..." Through authentic assessment, Boyer's scholarships of discovery, integration, and application are transformed into the scholarship of teaching (Southern Illinois University assessment web page).
One of the major trends in undergraduate education across the U.S. is the recognition of the value of assessment activities which supplement traditional measures of student achievement such as course work and grades. Carefully constructed assessment tools, designed for specific student populations, frequently provide extremely valuable information to faculty with regard to their educational goals and objectives, information not obtainable by examining student transcripts. As a result of the 1999-2000 North Central Association (NCA) accreditation process for CU-Boulder, only one specific interim report was assigned to the campus: assessment. The Colorado Commission on Higher Education has also included assessment plans as part of the Quality Indicator System. Lastly, this campus has begun to implement the notion of 'unit merit'; unit assessment processes have the potential to be included in such evaluations.
An Assessment Oversight Committee has been given the task of critiquing, supporting and re-vitalizing unit assessment efforts and of preparing a report to the NCA on progress and plans in this area of responsibility. Our premise is that outcomes assessment should, first and foremost, be useful to CU-Boulder and should help individual academic units
- evaluate their curricula,
- plan improvements where necessary, and
- evaluate the effects of the changes.
Assessment leads to curriculum change
As past examples of student achievements illustrate, outcomes assessments help units affirm aspects of their curricula and courses that are going well. They also help identify things that are not going so well, and often suggest the kinds of changes that might be needed. About half of the units doing serious outcomes assessment (and about 40% of all units reporting through the late 1990s) have identified areas for improvement. Seniors' performance in some areas may not be up to the faculty's expectations, or other aspects of the data may suggest changes that would strengthen the unit's undergraduate program.
For example:
- On exit surveys, Chemistry/Biochemistry students reported that they valued hands-on experiences in advanced labs and independent study courses. As a result, more experiments were added in lower-division courses to provide more and earlier hands-on experiences. In addition, the faculty now more vigorously encourage students to do independent study. As a result the number doing independent studies has increased by 60%.
- Faculty reviews of Theatre and Dance senior seminar essays and results on faculty-developed diagnostic exams showed that students were not as well prepared as faculty expected them to be in theatre history or dramatic literature. The faculty revised the sequence of courses in these areas and now emphasize them in the senior seminar.
- In response to early assessment results, Classics faculty added more sight-reading exercises in introductory Latin and Greek courses. As a result, translation grades in those courses rose noticeably.
- The Department of English, disturbed at the number of minor grammatical errors in even the best of student writing, added a writing component to two introductory courses.
Differences between units must be acknowledged
The diversity of CU-Boulder units is sometimes a problem in an enterprise like this one, but it is more often a benefit.
- Units are too different, and too independent, to attempt to impose common methods. Letting units experiment with methods and discover what provides useful information for them takes time and requires unit-specific discussions, feedback, and consultation. The individual unit summaries show great variation in how units gather and use outcomes information.
- Units make the best use of approaches that fit their own, and their discipline's, style of gathering and using information.
- Units learn from each other. One unit's experiences with a method can help another unit decide to try it, or to avoid it.
Some central oversight and support is necessary
The process would not have grown as well as it has without a certain amount of central monitoring, prodding, cajoling, assistance, and constructive feedback. However, the benefits of acknowledging units' differences and giving them control of their own outcomes assessment processes would be lost if there were too much central control. Keeping a balance requires constant attention.
- It is helpful if the coordinators have, or have access to people with, skills in data gathering and interpretation. Units may want or need help with planning assessment methods, gathering information, and making effective use of it. This can be particularly true for units using surveys or student records data.
- It is important to adapt assistance and feedback to the unit's own research styles and skills. Faculty in some units are very experienced in designing surveys and using survey data, for example. Those in other units may not be, but may still find survey data useful. Some are quite skilled at holistic scoring of essays. Others are not familiar with these techniques. And so on. It is not appropriate, or useful, to treat units with different sets of skills the same way.
Example Assessment Findings Programs for Selected Academic Units
The following examples illustrate findings from a few units' most recent annual assessment reports. They are only examples--more details, and results from every unit, are in the individual unit summaries.
- In the writing skills component of the College of Arts and Sciences' general education program, internal and external raters agree that student writing at the end of the semester shows clear improvement over work early in the term.
- The Departments of Computer Science, Mathematics, and Sociology report that their majors' performance on the nationally standardized Major Field Achievement Tests (MFAT) is at or above the national norms for their areas.
- Upper-division Communication majors' average scores on a specially-constructed test of knowledge in the area are considerably higher than those of non-majors in the same upper-division courses and beginning majors in lower-division courses.
- Approximately half of the papers from senior History courses reviewed each year by a faculty committee typically receive ratings in the highest category on scales for the department's knowledge goals.
- Senior-level students in the College of Engineering and Applied Science taking the Engineer-in-Training exam of the National Council of Examiners for Engineering and Surveying consistently exceed the national passing rate.
- Internship evaluations for Journalism seniors are consistently high, and the raters' narrative comments are quite positive.
Designing, Improving, Implementing Assessment:
The methods, styles, strategies and aims for undergraduate student assessment can and should encompass a full spectrum of options: standardized national tests, special assignments embedded within senior capstone courses, portfolios and videotapes, specially designed competency exams, evaluation by faculty outside the unit or the Boulder campus, etc. For units who need to re-vitalize their assessment processes, we strongly encourage careful examination of the wide range of successful examples posted on the web. Members of the campus Assessment Oversight Committee can also serve as valuable sources of information and guidance.
Key elements for a successful assessment effort include (1) unit faculty buy-in plus a competent, conscientious unit assessment leader, (2) regularity of the process, (3) responsiveness to the results, and (4) adequate resources to accomplish the tasks at hand. Assessment goes to the heart of the academic enterprise: what have students accomplished academically and how well does that match the unit's stated goals for its students in the catalog? Clearly, faculty must control nearly all aspects of assessment for it is they who are charged with accomplishing academic achievement. If the faculty in a particular unit treat assessment as a make-work, unimportant enterprise, the practice will be a waste of time and other resources. On the other hand, those units with strong faculty commitment, demonstrate decided advantages and improvements in their programs. The effort needs to have stable, competent faculty leadership within each unit. When assessment results point up weaknesses, the unit faculty must identify actions to remedy those weaknesses, else the effort on assessment goes to waste. Assessment cannot be a sometime or irregularly implemented process and be effective. When assessment produces changes, the results of those changes also need to be assessed in a spirit of continual monitoring and follow up. Lastly, the campus administration must provide adequate resources to units to allow them to implement assessment plans and consequent actions, all aimed toward improved undergraduate education.
PBA: L:\ir\outcomes\aoc\h010322_02.doc - 5/14/01.