PBA Home >
Institutional Research & Analysis >
NRC Assessment of Research-Doctorate Programs
> CU-Boulder process
The CU-Boulder process for submitting data
NRC 2006 data submission
At CU-Boulder, the Graduate School was in charge of coordinating the
CU-Boulder participation in the NRC assessment, with technical support from Planning, Budget, and Analysis (PBA).The CU-Boulder NRC steering committee was:
- Susan Avery, institutional coordinator (IC)
- Lou McClelland, Director of Institutional Analysis (IA), data collection and exchange coordinator
- Carol Lynch, former VC for Research and Dean of the Graduate School, now faculty associate in Academic Affairs, and member of the national NRC steering committee
- John Stevenson, Associate VC for Graduate Education and Associate Dean of the Graduate School
As the project went forward, Lou McClelland assumed the role of IC.
NRC data collection used five separate questionnaires
(link goes to blank versions of all five)
- The Institutional Questionnaire collected data about institution-wide policies and practices
and graduation or completion rates by broad field and race/ethnicity. PBA collected this information from UCB central records and submitted it to NRC.
- The Program Questionnaire collected information about each of the 33 doctoral programs assessed at CU-Boulder.
- The steering committee answered some questions consistently for all programs (e.g., continuous enrollment policy).
- Using a short web questionnaire
(PDF), PBA collected policy/practice information from each program and submitted these data for all programs.
- PBA also collected from departments counts of post-doctoral fellows.
- For each program, PBA generated completion rates, enrollment and degree counts, financial support data, and other information from central records. PBA then
sent all information to programs for review before submitting the information to NRC.
- CU-Boulder markup of program questionnaire
showing data generation process by item.
- The Faculty Questionnaire went to all new and core
faculty, with questions on names used in publishing, publications, educational
and work background, research and teaching, and importance of the 21 measures
in program quality
- PBA "prepopulated" part of the Faculty Questionnaire with information (e.g., highest degree earned) generated from faculty lists
and other UCB records and sent the available information to NRC.
questionnaire and mark-up ||
checking info pre-populated for an individual
- NRC generated for each person categorized as “core” and “new” faculty a
web questionnaire including the information sent by PBA.
- Faculty respondents could override what was already recorded (i.e., sent
by PBA) before resubmitting their faculty questionnaire.
- The Student Questionnaire was administered to doctoral candidates in five fields only:
economics, English, chemical engineering, neuroscience, and physics. Students were asked to provide
information related to their studies and career development.
- PBA generated lists of students eligible for the survey and submitted the lists to programs for review.
- NRC contacted the students and handled the data collection.
- The Program Quality Questionnaire was
used to collect reputational ratings of programs. NRC selected a sample of respondents to the faculty questionnaire.
NRC mailed (via US mail), to each person in the sample, information on about 15 programs (across the country) in one
field (e.g., in History). Information about each program included:
Respondents provided quality ratings for each program. No program was rated by all raters,
and some programs were not rated by anyone. Ratings were not published. Instead, they were used
to derive statistical relationships between program “objective factors” (publications
per person, graduation rate, etc.) and quality or reputation.
- Name of institution and program
- List of faculty by name
- Graduation rate and time to degree
- Faculty gender and minority percentages
Faculty productivity factors
As part of its characterization of doctoral programs, NRC matched faculty names (from lists and
faculty questionnaires) with the Institute for Scientific Information (ISI) citation database and other
sources to collect data on publications per faculty member, citations per publication, and dissertation keywords.
Timelines for NRC at UCB