PBA Home >
Institutional Research & Analysis >
Information by department >
Comparing to peers
Comparing CU-Boulder departments to peers -- Notes on measures and available data
No one measure works for all departments. Few are related to undergraduate education. Many are indicative only for science and engineering departments.
Note: Most comparative data available to PBA are public. Some are non-public exchanges through the AAU data
- Agreed-upon measure is dollar amount of expenditures, not awards
- Not useful for units in arts and humanities
- Comparisons across institutions
- NSF does by discipline, for many science-related major disciplines. See
the NSF web-site.
- Many NSF listings are for CU system, not UCB alone
- Same data are in the AAUDE warehouse (AAU data exchange; PBA can access), probably for UCB alone instead of CU system. Have not used.
- See http://www.colorado.edu/pba/facstaff/research/aau.htm for UCB share of national research expenditures by discipline among AAU institutions.
- Comparisons among UCB departments. See
http://www.colorado.edu/pba/facstaff/research/index.htm for expenditures, award amounts, and number of awards, over time, by UCB department, with institute activity alone and mapped to instructional departments.
- Thomson-ISI covers 21 science and social science fields.
- http://www.isinet.com/demos/esidemo/default.htm ISI indicators for citation analyses. Not free.
- http://www.isihighlycited.com/ ISI Highly cited individuals; can list by institution. UCB has 25 listed.
- PBA can access ISI citation counts for AAU institutions for 1998-2002
by broad discipline (arts and humanities; life sciences;
physical science, math, engineering; social/behavioral science) by institution. For
each institution - broad discipline combination, we can get
number of papers published on journals tracked by ISI, and
number of citations to them tracked by ISI. The ratio
of citations to papers is often labelled "citation impact."
More detailed data are available from ISI for a fee.
Numbers of students
NRC (National Research Council assessment of research doctorate programs) is primarily a reputation survey, of doctoral programs only.
Scheduled every 10 years, last issued in 1995. Next collection in 2006-07 if funds are raised, with publication at least two years after.
Considered the gold standard because it's non-profit, doctoral focus, and carefully executed.
http://www7.nationalacademies.org/resdoc/index.html Methods and scope under debate.
- US News also uses reputation surveys, mostly for PhD and professional programs (some master's, some broad undergrad areas)
- E.g., ratings of graduate history, psych, soc depend entirely on reputation surveys
http://www.usnews.com/usnews/edu/grad/rankings/about/faq_meth_brief.php for methods and links to results.
PBA has a password to detailed results.
- Ratings of larger programs such as MBA's are based on reputation plus other factors such as graduate employment rates and incoming-student credentials.
From the US News site: Researchers Evan Rogers and Sharon J. Rogers have compared rankings produced by the 1995 NRC study with those U.S. News has developed, looking specifically at the U.S. News rankings based on peer assessment data only. They concluded that there was "a very high positive association between U.S. News peer assessment scores and rankings and those reported by the NRC." Their article in the May 1997 issue of the American Association of Higher Education Bulletin gives further discussion of this topic.
- If reputation is enough, the US News rankings are more specific and far more current than NRC
- Any reputation survey measures exactly that. Reputations are slow to change, affected by sheer numbers, have halo effects, reflect graduate programs and faculty publishing rather than undergraduate instruction, and are sometimes based on only a few members of a faculty.
Faculty and faculty salaries
Degrees granted available by degree level (bachelor's, master's, PhD), degree discipline, institution, fiscal year, by gender by ethnicity, from IPEDS - public data. In AAUDE warehouse; PBA can gather.
- Enrollment not available by discipline except in some graduate level college guides (e.g., Peterson's), which are not available electronically. AAUDE working on this.
Student activities, satisfaction
Only collection is the Delaware study, with recent UCB participation for all departments
- See http://www.colorado.edu/pba/course/delaware/ovv.htm for overview
- Everything is by department. Data include
Degrees by level
- Student credit hours (SCH) and sections taught by faculty type (TTT, TA, other) and course level (lower, upper, grad, for group vs. individual)
- Instructional expenditures
- Sponsored research expenditures
- Many derived measures. E.g., percentage of upper-division SCH taught by tenured and tenure track
- Excel tool for comparing one UCB department to a selection of other UCB departments is posted
- PBA can construct similar for a department vs. AAU publics, but cannot post with other institutions identified. A sample is posted.
- In January 2004 PBA compared UCB "superdepartments" (e.g., "life sciences") to 9 AAU peers. Results are not posted but are available to departments.
- The Delaware study is piloting a collection on out of class faculty activities - committees, service, etc. PBA submitted data from FIS (from the annual FRPA collection)
to this collection; these data are available by department. But no other AAU's have submitted.
Faculty honors and awards
- NSSE, National Survey of Student Engagement. National instrument with heavy AAU public participation. Colorado coordinates AAU data sharing. Freshmen and seniors only.
- Graduate survey: In fall 2003 we surveyed all Graduate School students except those entering fall 2003.
Almost 3000 students responded, enabling comparison by program and degree level, for those programs with 6 or more
students responding within each degree level (29 master's level programs and 36 doctoral level programs). A listing of students' open-ended
responses was distributed to departments. See
We hope, but as yet have been unable, to obtain comparative data from other AAU's.
- Alumni survey, 2003: surveyed bachelor's, master's, and doctoral recipients about 4 years after graduation. Results posted campus-wide,
by college, by A&S discipline cluster, by department, and by degree level -- for those degree-level programs in which we received six or
more responses: 32 bachelor-level major programs, 12 master's level programs, and 4 doctoral level programs. See
http://www.colorado.edu/pba/surveys/alumni/03/index.htm; completed forms were sent to departments.
- Senior survey: In spring 2004 we surveyed current seniors. Medium to large majors were over-sampled, enabling comparison by many
departments, as well as by college and by A&S discipline cluster for the 38 departments/colleges with six or more responses per major program.
See: http://www.colorado.edu/pba/surveys/senior/04/index.htm; seniors' comments to open-ended items were distributed
to departments and are posted.
NRC 2006(+) may collect data by discipline on doctoral programs
- Especially important in humanities
- Various AAU's track 85 or more different honors and awards
- We have counts (not lists) of awards/honors by award name, year, and institution for all AAU's -- e.g., American Academy of Education, Nobel prize.
- Student demographics, graduation rates, time to graduation, support, satisfaction
- Faculty publishing
- Other program characteristics - mission, research emphases, space, etc.