PBA Home > Institutional Research & Analysis > Surveys > Alumni Survey 2007 > Method

Alumni Survey, Summer 2007

Method

Population and Sampling Frame

The population for the 2007 Alumni Survey consists of alumni who had earned a bachelor's, master's, or doctoral degree between August 2002 and May 2004. Alumni who had earned JD or MBA degrees were not included in the population. Only alumni with U.S. mailing addresses, received from the Alumni Foundation, were included in the sampling frame. About 94% of all alumni who earned degrees during this period had U.S. addresses available.

Alumni who had earned more than one degree between August 2002 and May 2004 were included only once in the sampling frame, based on the program with a smaller enrollment. For example, if an alumnus earned bachelor's degrees in anthropology and sociology, then he or she was included in the population for anthropology (but not for sociology), because anthropology typically has a smaller enrollment than does sociology. The rationale for this method is that allocating multiple degree earners to small programs increases the likelihood of having sufficient numbers of alumni within those programs for sampling and reporting purposes.

Sample

The target sample size for this survey was 4,000. For bachelor's and master's degree recipients, a simple random sample was drawn from each program. All doctoral recipients with U.S. mailing addresses were included, due to their relatively small numbers. For statistical reasons, a minimum of six respondents were required in order to provide results for a program. The sampling plan reflected this, in that just enough alumni were sampled from medium and large programs to ensure that we could produce a report for them, assuming a response rate of 20%.

For bachelor's recipients, we sampled 10% from programs with 30 or fewer degree recipients. For those programs with more than 30 bachelor's recipients, we sampled the smaller of either 60 or 60% of the total. For master's recipients, we considered Arts and Sciences division when sampling, because there are many small programs in Natural Sciences and Arts and Humanities. For master's recipients from these divisions, we sampled 10% from programs with 18 or fewer degree recipients. For larger Natural Science and Arts and Humanities programs, we sampled the smaller of either 60 or 66% of the total number of master's recipients. In other colleges and schools and in the Social Sciences division, we sampled 10% if the number of master's recipients was 29 or fewer, otherwise we sampled the smaller of 60 or 66% of the total. The table below illustrates these rules.

Sampling Plan 

Degree level

If college/ division is:

And if size of program is:

Then sample:

Total sample

Overall fraction

Bachelor's

Any

1-29

10%

2,697

31%

30 or more

The smaller of 60 or 60%

Master's

A&S Arts & Humanities

1-18

10%

827

48%

19 or more

The smaller of 60 or 66%

Other

1-29

10%

30 or more

The smaller of 60 or 66%

Doctorate

Any

Any

100%

484

100%

Total group

 

 

 

4,008

37%

Although small programs could have been excluded for purposes of providing results for individual programs, it was essential to sample them so that the responses of alumni could be generalized to colleges and to the campus as a whole.

Additional information concerning the sample, sampling frame, population, and respondents is shown for each degree level in the table below. This table illustrates that the percentage of doctoral degree recipients included in the sampling frame (84%) is smaller than the percentages of bachelor's (96%) and master's recipients (90%). These percentages are considerably larger than those of the previous Alumni Survey (67%, 58%, and 66%, respectively). This occurred because the Alumni Foundation had more mailing addresses for alumni in 2007 than they did four years ago when the survey was last conducted. The table also illustrates that doctoral recipients responded at a higher rate than did bachelor's and master's recipients (23% versus 20% each).

Respondents, Sample, Sampling Frame, and Population, by Degree Level

Degree
level

Respon-
dents

Sample

Sampling
frame
(popula-
tion with
addresses)

Popula-
tion

Sampling
frame as
a % of
population

Sample as
a % of
sampling
frame

Sample as
a % of
population

Responses
as a % of
sample
(response
rate)

Bachelor's

550

2,697

8,662

9,037

96%

31%

30%

20%

Master's

167

827

1,740

1,932

90%

48%

43%

20%

Doctorate

111

484

484

579

84%

100%

84%

23%

Total group

828

4,008

10,886

11,548

94%

37%

35%

21%

 Questionnaire

The 2007 Alumni Survey questionnaire has several multiple-choice and short-answer questions that ask alumni about employment, further education, and their opinions of CU-Boulder. It is similar, but not identical to the 2003 questionnaire. For example, the 2007 questionnaire has several additional multiple-choice items (16 versus 10 for 2003). Moreover, the 2007 questionnaire was administered online, whereas the former was entirely a paper questionnaire. There are five questions that are identical in the 2003 and 2007 questionnaires. Information on comparisons over time is available for these questions.

Data collection

Data collection began in May 2007. Postcard invitations were mailed on May 21 to all alumni in the sample, two days before email invitations were also sent. In general, we attempted to mail the reminder postcards out a couple days prior to the emails so that alumni would receive both the postcards and the emails on approximately the same day. The email invitations were sent to the 61% of sampled alumni for whom an email address was available (n = 2,429). Both the postcards and email invitations directed alumni to the online questionnaire.  

Postcard and email reminders were sent according to the schedule below, and data collection ended on July 23, 2007. The end date announced in the final email reminder was July 16; however, we extended this date by one week. 

Date

Task

Type of reminder

6/5/2007

Reminder 1

Postcard and email

6/21/2007

Reminder 2

Postcard and email

7/9/2007

Reminder 3

Email only

Of those alumni who we attempted to contact via email, approximately 500 could not be contacted at all because of incorrect and/or outdated email addresses. This means that about half of the sample did not receive any notification via email. 

We sent large, glossy postcard announcements via U.S. Mail to all 4008 alumni in the sample; only a couple of the postcards were returned as undeliverable. We had approximately 50 of the small postcard reminders returned as undeliverable by the postal service. Thus, approximately 3950 alumni sample members were reached via U.S. Mail. We did not check to see whether the 50 undeliverable postcards overlapped with any of the undeliverable emails, since the number of undeliverable postcards was small and unlikely to have any noticeable effect on an overall adjusted response rate for the survey.  

As an incentive to complete the survey, we offered respondents the chance to win one of two $1000 cash awards or one of four $500 cash awards. Additional information on the incentives is available. 

Response rates vary considerably across programs, ranging from 0% (e.g., applied mathematics, classics, etc.; bachelor's level) to 100% for relatively small programs in which only one alumnus was selected for the sample (e.g., finance and physics; master's level). The overall response rate was 21%. 

Reporting  

Responses to open-ended questionnaire items will be delivered to department chairs. Tables and graphs reporting survey results for multiple-choice items may be viewed by clicking on the links to the left. These results are reported for the entire campus, by degree level, by college and degree level, and by program or major. Results are reported here for colleges and majors with at least six respondents at the bachelor's, master's, or doctoral level. Results are excluded for degree levels with fewer than six respondents. Numbers of respondents, overall and for each item, are reported in the tables but not in the graphs.

The statistics reported in the tables and graphs for individual programs are unweighted. In contrast, those reported for the entire campus, for degree levels over the entire campus, and for colleges are weighted to adjust statistically for different sampling fractions and response rates over programs.

Last revision 07/03/08



PBA Home  |  Strategic Planning |  Institutional Research & Analysis |  Budget & Finances | 
Questions? Comments? |  Legal & Trademarks |  Privacy
15 UCB, University of Colorado Boulder, Boulder, CO 80309-0015, (303)492-8631
  © Regents of the University of Colorado