Knowledge and skill goals for this undergraduate degree program are recorded in the most recent CU-Boulder catalog. In some summaries of assessment activity, goals are referred to by number (e.g., K-2 is knowledge goal 2).

The department currently uses four measures to assess its undergraduate program: (1) A 50-item multiple choice test that measures the department knowledge goals. (2) A six-item survey assessing satisfaction with the major administered to graduating seniors, (3) An internship survey administered to field supervisors which assesses displayed communication skills, and (4) number of graduating seniors who are a member of the national Communication honors society (Lambda Pi Eta) or complete university honors.

In 1989-90 and 1990-91 the department assessed knowledge goals by examining students' answers to items from classroom exams in several upper-level courses. In addition, three outside experts evaluated the skills-related goals by rating students' performance on written and oral assignments in COMM 4800 (Current Issues in Communication.) These assessments led to changes in the structure of some courses, and course coordinators were appointed and encouraged to supervise, evaluate, and assist graduate-student teachers. The department initiated a three-day orientation for incoming graduate-student teachers and a corresponding semester-long workshop on teaching. A system of peer teaching evaluation was also established.

In 1991-92 the department suspended the use of outside reviewers for assessing skills. It was proving too costly and too complex -- outside reviewers were increasingly unwilling to participate. That year's focus was the development of a 50-item multiple-choice exam to test all of the program's knowledge goals. The first version of the test was administered to 50 graduating seniors in Spring semester, 1992. For 1992-93, a faculty review committee rewrote items that statistical analysis of the pilot version showed to be poor discriminators. In 1993-94 there was some further revision of a few items and the addition of 12 new items, and the test was administered in a required senior level communication course. In 1994-95, the practice of comparing scores of majors at entry and exit, as well as to non-majors was started. A survey for supervisors of interns was piloted in the 94-95 year and implemented as a formal part of the program in 95-96.

In 1997-98 because of changes in course offerings the knowledge assessment test was revised. In addition, six items were added to the test, to be filled out only by seniors. The senior survey was to measure satisfaction with the major, both overall and in regard to the major's focal skills. Finally, we decided to track number of majors graduating with professional or university honors. Number of honors students seemed a good indicator of how the program was doing with regard to its strongest students.

In 1997-98 the multiple choice knowledge test was updated. Questions were revised to better capture current course content and to represent a reasonable distribution of the four main content areas: (1) communication theory and method, (2) organizational communication, (3) argument and rhetorical theory, and (4) interpersonal interaction. Each area had 12-13 questions for a total of 50 items. In the spring semester of the two years this test was administered in all sections of the introductory public speaking class (Comm 1300) that enrolls a significant number of non-majors as well as students beginning the major. The test was also administered to communication students in senior seminars (4000-level classes). Mean student scores are noted in Table 1 for four groups: lower division and upper division majors, and lower and upper division non-majors.

Table 1: Knowledge Scores

Inspection of the table shows senior majors to perform better than either lower-class majors or non-majors of either rank. T-tests comparing seniors with the other groups were significant at the p=.01 level or higher. Given there is considerable choice in the classes a communication major may take, and the exam measures all possible course content, the pattern shows exiting seniors to know more that either of the three groups.

Although this index provides a general sense that students are learning discipline-relevant information, the department sees the index as a relatively general measure. We would like to see a bigger difference in the performance of seniors relative to the other three groups. This profile led the department to add a course requirement for all communication majors. As of summer 2000, all communication majors will be required to take Comm 2210 (Perspectives on Communication) rather than selecting from a menu of introductory course possibilities. This curriculum change should increase the particular theoretical knowledge that could be expected of all majors yet still retain students' ability to tailor course selection to their intellectual interests. In addition, course prerequisites are being added to senior seminars to insure that students have completed lower level classes before taking the senior seminars. These two changes are being introduced: (1) to increase the number of core ideas with which majors can be assumed to be familiar, and (2) to enable instructors teaching senior seminars to assume a larger common knowledge base for students in the class. Both of these changes we see as creating conditions that will facilitate instructors being more demanding of students in senior seminars.

Six items were administered to a subset of graduating seniors in each year to measure seniors' satisfaction with the communication major. Five items asked seniors to agree or disagree with statements about the value of the major compared to other majors. The focus of these items was on key competencies emphasized by the department. Statements were worded as follows: "Studying communication as a major (rather than some other major) increased my ability to present ideas in public settings in an interesting and organized fashion" (public speaking). Other items were worded similarly, asking seniors to report whether they strongly agreed (SA), agreed (A), neither agreed nor disagreed (N), disagreed (D), or strongly disagreed (SD) that studying communication compared to other majors:

• increased their ability to work with people, talk through difficulties and compete a group project (work w/people),
• increased their ability to analyze critical oral and written arguments (analyze arguments),
• increased their ability to write in a clear, organized and argumentatively compelling fashion (writing),
• increased their ability to manage differences of opinion with friends, supervisors, coworkers and family members (manage conflict).

A summary is presented in Table 2. Responses were aggregated across the two-year time period to create a more stable indicator.

A final item asked students to "assess the variety and quality of their learning experiences as communication major." Table 3 summarizes the responses.

The survey results show reasonably high level of satisfaction among majors. Inspection of differences among items suggests the department needs to give thought to how to improve students' writing skills. Over the next two-year cycle the department will be developing ways to target writing for improvement in all of its courses.

A significant number of majors do an internship in a professional setting related to one they may have a job interest in following graduation. In an internship, students spend a certain number of hours in the field, keep a journal, and write up a paper linking a theory of communication with their observation and experience in the internship site. Toward the end of each semester, interns' field supervisors are sent a 7-item questionnaire and are asked to evaluate their intern.

Five of the items ask the supervisor to rate the student on 7-point scales where 1 =poor and 7 = excellent. Items included, for instance, "I would describe the intern's ability to express himself/herself orally as" followed by a seven-point scale. Besides assessing oral expression, other items ask about the intern's quality of writing, ability to adapt messages to particular audiences, listening skills, and critical thinking and problem-solving ability. In addition, supervisors were asked for open-ended comments and what internship grade they would assign. A summary of the quantitative part of the field supervisory assessment is below. For 97-98 there was a 68% return rate, for 98-99 a 76% return rate. Individual comments were largely positive, often exceptionally so. Not infrequently, supervisors commented that they would hire the student if a position came open. In the 97-98 there were occasional concerns expressed about students' writing skills; this occurred less frequently in 98-99.

In sum, feedback from field supervisors suggests the department is doing a good job preparing its students for the wide range of professional positions that they might take upon graduation. The department regards the internship program as important to the quality of its program and intends to continue encouraging students to get involved in this distinctive type of learning.

In 97-98, nine students graduated with honors, 2 with highest honors (summa). This was the largest number of students completing honors since the department began tracking. In 98-99, the department became a member of Lamda Pi Eta, the national honors society for undergraduates in communication, and established a local chapter (Eta Phi). To be eligible for Eta Phi, an undergraduate must meet several requirements, including a grade-point minimum and involvement with professional and scholarly activities is in the department (e.g., attending research colloquia). In this first year, 25 students graduated with communication honors. In addition, one student completed university honors (summa level). The department regarded the introduction of Eta Phi and the increase in students graduating with honors as a desirable accomplishment. We plan to continue encouraging eligible students to join Eta Phi.

In 1998-99 the number of students completing university honors decreased from the previous year's high. To determine the most likely reasons for this decline, a focus group was run with members of Eta Phi honor society. Two factors appeared to affect students' decision not to pursue university honors in the 98-99 year: (1) The introduction of the professional honors society made it possible to graduate with honors without writing a thesis, a task students saw (and rightfully so) as highly demanding. (2) Students reported that they had inadequate information about doing honors and insufficient encouragement to do so. In light of student feedback, the department's honors recruiting letter was revised to be more encouraging. In addition, the department began to do a number of other practices to encourage students to think about pursuing university honors and writing a thesis (e.g., sending seniors working on honors into lower-level classes to talk about the satisfactions in doing honors). Our goal for the next two year cycle is to increase the number of students who graduate with honors in the discipline and have 5-10 students graduating each year with university honors.

Return to the index of unit summaries

Return to the Outcomes Assessment Table of Contents

L:\www\outcomes\units\Comm.htm
Last updated August 27, 1999