|
PBA Home >
Institutional Research & Analysis >
NRC Assessment of Research-Doctorate Programs >
Measures and methods+
Measures and ranking methods
October 2010 (updated 10-7-10; new highlighted)
NRC released rankings (or "ranges of illustrative rankings") on
9-28-10 within disciplines on
- Overall quality with two versions, known as
- R: regression, implicit, indirect --
Programs are ranked highly if they have features similar to programs
viewed by faculty as top-notch. Weights are based on relationships
between program characteristics and reputation ratings gathered from
a sample of faculty on a sample of programs
- S: survey, stated, explicit, direct
--
Programs are ranked highly if they are strong in the criteria
that scholars say are most important
- Three dimensions to "spotlight program characteristics not prominent in the overall rankings"
- Research activity (RA)
- Student treatment and support (SS)
- Diversity (D)
The number of programs ranked varies widely by field,
from over 200 in psychology (including two programs at CU-Boulder,
psychology and cognitive science), over 150 in chemistry and in physics, to
under 40 in aerospace engineering and in theatre. This means that a
program ranked 10th in the U.S., say, may be in the top 5% of all
programs (if 200 were ranked), or in the top 25% (if 40 were ranked).
Therefore we consistently present
both
- The ranking itself, of a CU-Boulder program -- e.g., 16th
to 30th among all programs (NRC always presents ranges of
rankings)
- A short-hand categorization of the ranking -- e.g.,
top 10, top 25, or top 50 among all programs
- How the ranking translates to "top X percent
among all programs -- e.g., 16th
is at the median (the 50th percentile) in a field with 32
programs, but is in the top 10% of all programs in a field with 160.
All rankings are based on characteristics of students, of the
program itself, and of faculty.
| The Key
Measures Used in the Rankings - all (but one) are used in the overall rankings, whereas only the
ones specified below are used in the three dimensional rankings (research activity, student
support, and diversity) |
| Measures of Research Activity |
|
Measures of Diversity |
- Publications, 2000-20061 per allocated faculty
- Average citations per publication (not used for
Humanities)
- Percent faculty2 with grants
- Honors and awards per allocated faculty
|
|
- Percent core & new faculty non-Asian minority
- Percent core & new faculty female
- Percent students non-Asian minority
- Percent students female
- Percent students international
|
Measures of Student Support and
Outcomes
|
|
Overall Measures (not
used in the three
dimensional rankings, but included in the overall) |
- Percent of 1st year students with
full financial support
- Completion rate: Percent of entering students completing in 6 yrs. or less (8 yrs. for
humanities)
- Median time to degree
- Percent PhDs with definite plans for an academic
position
- Program collects data about post-graduation employment
(not used in overall)
|
|
- Average number of PhDs awarded, 2002-2006
- Percent interdisciplinary faculty (associated faculty as percent
of total)
- Average GRE score for students entering 2004-20063
- Percent of students with externally-funded fellowships, 2005
- Student workspace provided(100% vs. less than 100%)
- Student health insurance provided (yes/no)
- Student activities offered - number offered
from a list of 18
|
1From ISI for non-humanities; books
from faculty curriculum vitae in humanities
2Of core and new faculty responding to the NRC
faculty survey
3Verbal
for humanities, quantitative for all other disciplines |
Details on the measures used in the rankings - including information
on source and contribution to rankings, issues with measures and issues
with use in rankings (added 10-7-10)
The CU-Boulder process
- how data were submitted to NRC
About the
Program Questionnaire, including the
CU-Boulder program questionnaire mark-up
About the
Faculty Questionnaire
|| Section G, with importance ratings of the 21 measures
(added 10-7-10)
About the
Ratings
Questionnaire
Institutional questionnaire
(PDF) - CU-Boulder institutional questionnaire mark-up
|
