Map A to Z Index Search CU Home University of Colorado
 
  Strategic Planning Institutional Research and Analysis Campus Budget and Finances About PBA

PBA Home > Institutional Research & Analysis > Information by department  > Comparing to peers

Comparing CU-Boulder departments to peers -- Notes on measures and available data

No one measure works for all departments. Few are related to undergraduate education. Many are indicative only for science and engineering departments.

Note: Most comparative data available to PBA are public. Some are non-public exchanges through the AAU data exchange (AAUDE).


Research money

  • Agreed-upon measure is dollar amount of expenditures, not awards
  • Not useful for units in arts and humanities
  • Comparisons across institutions
    • NSF does by discipline, for many science-related major disciplines. See the NSF web-site.
    • Many NSF listings are for CU system, not UCB alone
    • Same data are in the AAUDE warehouse (AAU data exchange; PBA can access), probably for UCB alone instead of CU system. Have not used.
    • See http://www.colorado.edu/pba/facstaff/research/aau.htm for UCB share of national research expenditures by discipline among AAU institutions.
  • Comparisons among UCB departments. See http://www.colorado.edu/pba/facstaff/research/index.htm  for expenditures, award amounts, and number of awards, over time, by UCB department, with institute activity alone and mapped to instructional departments.
Research publishing
  • Thomson-ISI covers 21 science and social science fields.
  • http://www.isinet.com/demos/esidemo/default.htm ISI indicators for citation analyses. Not free.
  • http://www.isihighlycited.com/  ISI Highly cited individuals; can list by institution. UCB has 25 listed.
  • PBA can access ISI citation counts for AAU institutions for 1998-2002 by broad discipline (arts and humanities; life sciences; physical science, math, engineering; social/behavioral science) by institution. For each institution - broad discipline combination, we can get number of papers published on journals tracked by ISI, and number of citations to them tracked by ISI. The ratio of citations to papers is often labelled "citation impact." More detailed data are available from ISI for a fee.
Reputation
  • NRC (National Research Council assessment of research doctorate programs) is primarily a reputation survey, of doctoral programs only. Scheduled every 10 years, last issued in 1995. Next collection in 2006-07 if funds are raised, with publication at least two years after. Considered the gold standard because it's non-profit, doctoral focus, and carefully executed. http://www7.nationalacademies.org/resdoc/index.html Methods and scope under debate.
  • US News also uses reputation surveys, mostly for PhD and professional programs (some master's, some broad undergrad areas)
    • E.g., ratings of graduate history, psych, soc depend entirely on reputation surveys
    • See http://www.usnews.com/usnews/edu/grad/rankings/about/faq_meth_brief.php  for methods and links to results. PBA has a password to detailed results.
    • Ratings of larger programs such as MBA's are based on reputation plus other factors such as graduate employment rates and incoming-student credentials.
  • From the US News site: Researchers Evan Rogers and Sharon J. Rogers have compared rankings produced by the 1995 NRC study with those U.S. News has developed, looking specifically at the U.S. News rankings based on peer assessment data only. They concluded that there was "a very high positive association between U.S. News peer assessment scores and rankings and those reported by the NRC." Their article in the May 1997 issue of the American Association of Higher Education Bulletin gives further discussion of this topic.
  • If reputation is enough, the US News rankings are more specific and far more current than NRC
  • Any reputation survey measures exactly that. Reputations are slow to change, affected by sheer numbers, have halo effects, reflect graduate programs and faculty publishing rather than undergraduate instruction, and are sometimes based on only a few members of a faculty.
Numbers of students
  • Degrees granted available by degree level (bachelor's, master's, PhD), degree discipline, institution, fiscal year, by gender by ethnicity, from IPEDS - public data. In AAUDE warehouse; PBA can gather.
  • Enrollment not available by discipline except in some graduate level college guides (e.g., Peterson's), which are not available electronically. AAUDE working on this.
Faculty and faculty salaries Instruction
  • Only collection is the Delaware study, with recent UCB participation for all departments
  • See http://www.colorado.edu/pba/course/delaware/ovv.htm  for overview
  • Everything is by department. Data include
    • Degrees by level
    • Student credit hours (SCH) and sections taught by faculty type (TTT, TA, other) and course level (lower, upper, grad, for group vs. individual)
    • Instructional expenditures
    • Sponsored research expenditures
    • Many derived measures. E.g., percentage of upper-division SCH taught by tenured and tenure track
  • Excel tool for comparing one UCB department to a selection of other UCB departments is posted
  • PBA can construct similar for a department vs. AAU publics, but cannot post with other institutions identified. A sample is posted.
  • In January 2004 PBA compared UCB "superdepartments" (e.g., "life sciences") to 9 AAU peers. Results are not posted but are available to departments.
  • The Delaware study is piloting a collection on out of class faculty activities - committees, service, etc. PBA submitted data from FIS (from the annual FRPA collection) to this collection; these data are available by department. But no other AAU's have submitted.
Student activities, satisfaction
  • NSSE, National Survey of Student Engagement. National instrument with heavy AAU public participation. Colorado coordinates AAU data sharing. Freshmen and seniors only.
  • Graduate survey: In fall 2003 we surveyed all Graduate School students except those entering fall 2003. Almost 3000 students responded, enabling comparison by program and degree level, for those programs with 6 or more students responding within each degree level (29 master's level programs and 36 doctoral level programs). A listing of students' open-ended responses was distributed to departments. See http://www.colorado.edu/pba/surveys/grad/03/index.htm. We hope, but as yet have been unable, to obtain comparative data from other AAU's.
  • Alumni survey, 2003: surveyed bachelor's, master's, and doctoral recipients about 4 years after graduation. Results posted campus-wide, by college, by A&S discipline cluster, by department, and by degree level -- for those degree-level programs in which we received six or more responses: 32 bachelor-level major programs, 12 master's level programs, and 4 doctoral level programs. See http://www.colorado.edu/pba/surveys/alumni/03/index.htm; completed forms were sent to departments.
  • Senior survey: In spring 2004 we surveyed current seniors. Medium to large majors were over-sampled, enabling comparison by many departments, as well as by college and by A&S discipline cluster for the 38 departments/colleges with six or more responses per major program. See: http://www.colorado.edu/pba/surveys/senior/04/index.htm; seniors' comments to open-ended items were distributed to departments and are posted.
Faculty honors and awards
  • Especially important in humanities
  • Various AAU's track 85 or more different honors and awards
  • We have counts (not lists) of awards/honors by award name, year, and institution for all AAU's -- e.g., American Academy of Education, Nobel prize.
NRC 2006(+) may collect data by discipline on doctoral programs
  • Student demographics, graduation rates, time to graduation, support, satisfaction
  • Faculty publishing
  • Reputation
  • Other program characteristics - mission, research emphases, space, etc.
 
Revised 4/26/05

Last revision 10/07/08


PBA Home  |   Strategic Planning  |   Institutional Research & Analysis |  Budget & Finances  | 
Questions? Comments? |  Legal & Trademarks |  Privacy
15 UCB, University of Colorado Boulder, Boulder, CO 80309-0015, (303)492-8631
  © Regents of the University of Colorado