PBA Home > Institutional Research & Analysis > Surveys > Senior Survey 1998 > Background

Senior Survey, 1998

Background

INTRODUCTION

We regularly ask seniors about their satisfaction with their educational experiences at CU-Boulder (UCB) and about their after-graduation plans and activities. Our goal is to provide systematic information:

for academic and service units to use in planning and improvements,
to help the Boulder campus meet state requirements for institutional accountability, and
for use by the state and by prospective and current students and their families.

Questionnaires were completed by 820 students who were seniors in spring 1998, or 51% of those surveyed. This report summarizes the results across all respondents.

METHODS

Population and sample: The population was all students classified as seniors in Spring 1998. A stratified random sample of 1,610 (34% of the population) was contacted. The population included both seniors expecting to graduate in spring or summer 1998 (55% of respondents) and those expecting to graduate in fall 1998 or later. Twenty-three large majors (e.g., economics, psychology) and the schools/colleges of Architecture and Planning, Journalism, and Music were oversampled to allow us to characterize these majors and schools/colleges separately. For all analyses, we have weighted to compensate for oversampling and for differential response rates within the majors and schools/colleges.

Contact method: A postcard asking seniors to complete the 1998 Senior Survey on the web was sent to sample members four weeks into the spring 1998 semester. A $500 lottery incentive was offered to those who completed the survey on the web within two weeks of the postcard mailing. An email reminder about the Senior Survey on the web was also sent out to all sample members four days after the initial postcard. Three weeks later a paper version of the questionnaire was mailed to all web non-respondents. Students who returned the paper version of the survey within three weeks of the mailing were eligible for a $200 cash-prize lottery. A follow-up postcard was sent four days after the paper version of the survey was mailed. Four weeks after mailing the paper version, the questionnaire was remailed to those who had not yet responded. Six weeks after the initial mailing, nonrespondents were telephoned and encouraged to complete and return the questionnaire.

Response rate: Thirty-five percent of sample members completed and returned the written questionnaire, and 16% completed the questionnaire on the Web for an overall response rate of 51% (n=820). The 51% response rate is somewhat lower than that obtained in prior years (1993, 1994, 1995, and 1996 response rates of 60%, 63%, 65%, and 61% respectively). In prior years, questionnaires were mailed with a $2 incentive. In 1998, we used a lottery incentive to entice sample members to respond via the web. Students who responded via paper also were entered into a lottery drawing, but for a lesser dollar amount than the web respondents. It is possible that the lottery did not provide as great an incentive to reply as did the $2 bills included in prior years’ surveys. However, national surveys such as the College Alcohol Study conducted by Henry Wechsler at the Harvard School of Public Health have noted a national trend toward lower response rates. Our response rate may reflect a more general trend toward decreasing willingness of students to respond to mail surveys. At any rate, the lower response rate for the 1998 data means that we have somewhat less confidence in this year’s estimates as compared to prior years.

Changes over time: In comparing data across time, we report senior survey data for the years 1993, 1994, 1995, 1996, and the current year (the survey was not administered in fall 1997). Prior to 1993, the population for the senior survey was all degree recipients, rather than all seniors (as it was from 1993 on). Also, in the 1990-1991 survey cycle, questionnaires were distributed via graduation check-out rather than using a mail survey. Using data from 1993 on provides the most direct comparison of results for similar populations using similar methods of data collection.

QUESTIONNAIRE DESIGN

The 1998 senior questionnaire consists of two general sections:

Reflections on the College Experience (questions in this section focus on respondents' attitudes about and experiences with UCB programs), and
Post-Graduation Activities (questions focus on respondents' employment and educational plans after graduation)

The 1998 questionnaire consists of 121 "forced-choice" items in which respondents check off one or more choices from a list, five short-answer items (to record future job or school plans), and five open-ended items of which the student completes three.
The 1998 senior questionnaire is nearly identical to the 1996 questionnaire. The few changes made to the 1998 questionnaire included:

adding an item measuring satisfaction with opportunity for hands-on, applied experiences in courses
rewording the item assessing use of and satisfaction with the University Learning Center to use its new name, the Student Academic Services Center
replacing the open-ended item assessing "If you were starting over, what would you do differently" with a forced-choice item
revising instructions for the open-ended items to request that students answer only three of the five opens versus all of the opens in 1996.

Questionnaires used in 1993 and 1994 are very similar to the current version.

To see a copy of the questionnaire used for web data collection:
Web Version of the Senior Survey

RESPONDENTS VERSUS NONRESPONDENTS

Response rates by various respondent characteristics were examined to determine if certain groups of seniors were more or less likely to return a questionnaire than other groups (see Table below). The table below shows demographic and academic characteristics for the total sample and for respondents and nonrespondents. The last column shows the response rate for each demographic/academic category (e.g., the response rate was 45% for males and 57% for females).

Spring 1998 Senior Survey -- University of Colorado at Boulder
Sample and Survey Respondents versus Nonrespondents


                                        RESPOND       Response Rate
                           Sample     No       Yes    by demographic

Total Number               1,610      793       817^       51%

Male                        55%       61%       49%*       45%
Females                     45%       39%       51%*       57%

Student of Color            16%       18%       14%*       45%
White                       84%       82%       86%*       52%

Graduated Spring 1998       37%       31%       43%*       59%
Didn’t Graduate in Spring   63%       69%       57%*       46%

Arts&Sciences               57%       59%       56%        50%
Business                    15%       14%       17%        54%
Engineering                 18%       17%       18%        53%
Journalism                   3%        3%        3%        54%
Music                        3%        4%        2%        36%
Arch & Planning              4%        3%        4%        56%

Low GPA (below median 3.0)  50%       56%       44%*       45%
High GPA (above median 3.0) 50%       44%       56%        57%

^ 3 additional students responded to the survey for a total of 820, but 
  removed their identifying label. Therefore, data for these 3 students are 
  not included in this table.
* indicates respondents and nonrespondents differ significantly (e.g., 
  respondents less likely to be male).

Respondents are less likely to be male, less likely to be students of color, and more likely to have graduated in spring 1998. Respondents do not differ by college with the exception that seniors in the School of Music were slightly less likely to respond.

Respondents also have higher Cumulative Grade Point Averages (GPAs). Because all of the above characteristics are related to GPA, we examined whether questionnaire responses varied according to students' GPA. GPA is related to responses on two of six general satisfaction items--the higher the students' GPAs, the higher their reported satisfaction with their overall academic experience at UCB and in their major program. However, less than 5% of the variance in responses can be attributed to GPA. Further, GPA is only weakly related to other items on the questionnaire. Because the relationship between GPA and questionnaire items is weak, we have not weighted the data to correct for over-representation of high-GPA students, nor have we weighted the data to correct for different response rates by gender, ethnicity, or graduation date.

WEB VERSUS PAPER RESPONDENTS

Students were given the opportunity to complete the Senior Survey online via the web. Of the 820 students responding to the survey, 251 (31%) chose to answer via the web. This percentage was substantially greater than in 1996 when only 13% of survey respondents answered via the web. The increase in web responding is, no doubt, the result of the large lottery incentive offered in 1998 to web responders ($500) and, more generally, to increased use of and familiarity with the World Wide Web among CU students.
The table below shows demographic and academic characteristics for all respondents and for paper versus web respondents. The last two columns show the percent in each demographic/academic category who responded via the web and via paper.
Seniors responding via the web are more likely to be male and more likely to be Engineering students; Arts & Sciences and Journalism seniors are more likely to respond via paper.
Spring 1998 Senior Survey -- University of Colorado at Boulder
Web versus Paper Respondents

Demographics and academic characteristics

                                  Responded via Percent       in Category

                                  All     Paper     Web      Web      Paper

Total Number                      820      569      251      31%      69%

Male                               49%      46%      54%*    34%      66%*
Female                             51%      53%      46%*    28%      72%*

Student of Color                   14%      13%      16%     30%      70%
White                              86%      87%      84%     35%      65%

Graduated Spring 1998              43%      42%      45%     32%      68%
Didn’t graduate in spring          57%      58%      55%     30%      70%

Arts&Sciences                      56%      60%      47%*    26%      74%*
Business                           17%      15%      20%     37%      63%
Engineering                        18%      15%      26%*    44%      56%*
Journalism                          3%       4%       1%*    38%      62%*
Music                               2%       2%       2%      7%      93%
Arch & Planning                     4%       4%       5%     26%      74%

Low GPA (below the median 3.1)     50%      51%      47%     29%      71%
High GPA (above the median 3.1)    50%      49%      53%     32%      68%

* indicates web and paper respondents differ significantly.

We also examined select questionnaire items to see if there were systematic difference between web and paper responders. We found that paper responders expressed higher levels of overall satisfaction with UCB and satisfaction with their overall academic experience. Satisfaction with academic experience in the major program did not differ between web and paper responders.

Both groups rated the importance of computer skills to after graduation success as quite high; however, web responders were more likely to say they have attained a high level of achievement in computer skills (67% for web versus 60% for paper responders). Web responders also have higher satisfaction with CU listings on the web and with email. There was no difference between the two groups in level of satisfaction with Mac and PC labs on campus. Not unexpectedly, web responders reported more often using a computer to access the web; paper responders more often used Chinook and other computer indices than did web responders.

While students who responded via the web differed in gender composition and college from paper responders, there is no indication that responding via web altered students’ responses in ways that would make them incomparable to the paper responses. Further, we believe that web responders would have responded to the paper version of the survey had it been the only option. Therefore, web and paper responses were combined for all results reported in the senior survey report.

Spring 1998 Senior Survey -- University of Colorado at Boulder
Web versus Paper Respondents

Key Questionnaire Items

                                                    Responded via
                                              All      Paper      Web
    
High Satisfaction w/ overall exp             72%       75%       68%*
High Satisfaction w/ academic exp            65%       68%       59%*
High Satisfaction w/ major exp               70%       71%       69%

High Importance general computing            86%       86%       86%
High Achievement general computing           62%       60%       67%*
High Satif. with CU listings on web ^        37%       33%       44%*
High Satif. with e-mail ^                    64%       59%       73%*
High Satif. with PC/Mac labs ^               17%       18%       17%

Average # times access web last full term^^  9.0       8.7       9.7*
Average # times use Chinook last full term^^ 5.8       6.0       5.2*

^ % based on respondents who actually used the service
^^ Highest category is "10 or more;" true average may actually be higher
* indicates significant difference between web and paper respondents.

Last revision 07/29/03

PBA Home | Strategic Planning | Institutional Research & Analysis |
Budget & Finances | Questions? Comments?
15 UCB, University of Colorado at Boulder, Boulder, CO 80309-0015, (303)492-8631
© 2001, The Regents of the University of Colorado