Map A to Z Index Search CU Home University of Colorado
  Strategic Planning Institutional Research and Analysis Campus Budget and Finances About PBA

PBA Home > Institutional Research & Analysis > NRC Assessment of Research-Doctorate Programs - Archive of instructions

NRC at UCB: Overview and Timelines

As of Jan 16 2006.
Upcoming deadlines and targets in red.
Deadlines for UCB faculty and programs are further highlighted in yellow.

Update 1-23-07: Some time points in this document have changed slightly. For current targets and deadlines see

List of programs with primary and secondary fields

  • Submitted 10/12/06. 33 programs listed.

Faculty lists

  • Submitted 12/15/06. 980 individuals, each listed in one, two, or three programs, for 1,141 total person-by-program listings
  • Info submitted on each: Name, program, NRC classification in program (Core, New, Associate), chair or advisor count, total committee count, campus address and phone, e-mail address, faculty rank and tenure status, highest degree
  • The 907 individuals listed as Core or New will receive NRC faculty questionnaires (described below)
  • We have posted an Excel compilation of the faculty lists, with committee counts and those getting the faculty questionnaire marked.

Faculty questionnaire

  • Asks about committee work, research specialties, scholarly activity, placement of PhD students, prior education and employment, demographics, and importance of program characteristics associated with program quality.
  • All faculty listed as Core or New in any program on the faculty lists will receive a questionnaire – one per person, not one for each program the individual is listed in.
  • NRC has set two response “deadlines”
    • Responses received by 4/1/07 will be used to characterize programs. This may be extended to 4/30/07.
    • Responses received by 2/18/07 will also be eligible to be sampled for the ratings questionnaire (described below).
  • UCB will submit info to NRC to “pre-populate” many sections of the faculty questionnaires.
    Target: By 2/1/07.
    • This will allow NRC one week to populate and activate the web questionnaires for UCB faculty, and allow UCB faculty who wish to be eligible for the ratings questionnaire 1+ week to respond before 2/18.
  • PROGRAMS: PBA will solicit data from programs for two sections of the faculty questionnaire, student placements and subfields the program covers. Solicitation timeline: Alert; files and instructions available by 1/22, deadline to return to PBA: 1/29/07.
  • NRC will send invitations by US and/or e-mail faculty with access to the web questionnaire. Promised: By one week after we submit pre-population info.
  • NRC will not use the pre-populated data unless the individual faculty member responds to the questionnaire.
  • Faculty who respond (on web) by 2/18/07 will be eligible to be sampled to receive the ratings questionnaire (described below).
  • All faculty responses received by 4/1/07 (or maybe 4/30/07) will be used to characterize programs.
  • Note: We realize that the schedule above gives UCB faculty only about 9 days (Feb 9-18) to respond and be eligible for the ratings questionnaire. We adopted this approach and schedule because
    • Responding will not take long given the pre-population
    • Putting many UCB faculty into the ratings-questionnaire sample is not a high priority, but we know some will want the opportunity
    • Pre-populating will produce more complete, accurate, and consistent data, and will save UCB faculty time and hassle – but is requiring extensive work by PBA.

Program questionnaire

  • Asks about program enrollments, degrees, faculty and student demographics, graduation rates, time to degree, student financial support, student programs and policies, and associated post-docs.
  • UCS will submit all info on all programs directly to NRC in a large batch upload.
  • NRC has set two submission “deadlines”
    • Submissions received by 4/1/07 will be used to characterize programs. This may be extended to 4/30/07.
    • Submissions received and processed by 2/18/07 will also be eligible to be sampled for the ratings questionnaire (described below). NRC requires 1-2 weeks for processing, so the effective submission deadline for inclusion in the ratings questionnaire is around 2/1.
  • UCB upload target: 3/9/07.
    • We will not make the 2/18/07 cutoff for inclusion on the ratings questionnaire. This is OK, and approved by Steering. It’s OK because only a sample of programs will be rated, with the ratings never published, but used to derive equations between ratings and “quantitative factors” such as graduation rates.
  • PROGRAMS: Programs will provide some info for the questionnaire in a web questionnaire, some in Excels, and will review inputs to and tables of student data reported to NRC. Dates for alerts to, file availability for, and deadlines for UCB programs will be between 2/5 and 2/28.
  • Program data submitted and processed by 4/1/07 will be used to characterize programs.
  • Note: We adopted this approach and schedule because
    • Putting UCB programs onto the ratings questionnaires is not a high priority – only a sample would be listed anyway, and the ratings will be used only to derive equations, never published for programs in individual institutions
    • Getting the faculty-questionnaire pre-population upload in by 2/1 is a higher priority
    • Generating most of the data centrally (rather than having each program respond directly to NRC) will produce more complete, accurate, and consistent data, and will save UCB programs time and hassle
    • It will take 2-3 weeks for programs (UCB departments) to review and respond to various components.
    • NRC requires that we upload all or none of the program-questionnaire data.

Allocation overrides

  • NRC will calculate “faculty productivity” averages for programs as a weighted average of the per-person “productivity” of individuals on the program’s faculty list.
    • Per-person “productivity” is measured in several ways including publications 2001-2006 for a single person.
    • The weight is called the person’s “allocation.” For most faculty the weight is one.
    • Faculty listed in multiple programs, and some others, will have weights less than one.
  • NRC will calculate initial weights or allocations from the faculty lists, using the classification (Core, New, or Associate), chair counts, total committee counts, and NRC rules.
  • NRC will return the initial allocations to UCB. We can then override them. The Steering committee will do this. NO individual’s allocation can exceed one.
  • Target for return to UCB, submission formats, and submission deadline: Unknown, currently stated by NRC as “late March and early April.”
Ratings questionnaire, aka “anchoring study”
  • Asks individual faculty to rate the quality of 15-20 programs nationwide in a single field (e.g., history, psychology) in which the individual was listed. Raters will receive each program’s name, institution, URL, graduation rate, time to degree, and faculty demographics, plus the names on the program’s faculty list, including Core, New, and Associated.
  • Raters will be sampled from Core/New faculty responding to the faculty questionnaire by 2/18/07.
  • Programs listed will be sampled from programs with program-questionnaire data processed by 2/18/07.
  • NRC will send US mail paper questionnaires to each rater. Target: April 2007.
  • Deadline for response: Unknown.
  • Results will be used to derive equations relating the “quantitative factors” such as graduation rate to the quality ratings.
Institutional questionnaire
  • Asks about policies such as health insurance. Also collects graduation rates for the entire campus by broad field (e.g., natural science) and student race/ethnicity.
  • PBA will submit for the campus. Target: 3/16/07. Deadline, 4/1/07.
Student questionnaire
  • Asks about students’ experiences in the program.
  • To doctoral candidates in 5 fields only: English, physics, neuroscience, chemical engineering, economics
  • Lists of students with US and email addresses were submitted 11/7/06. Not posted.
  • NRC will contact students directly. Date unknown.
  • NRC Steering will e-mail the students and contacts in the 5 programs when we know students have been contacted by NRC.
  • Deadline for response: April 1 with possible extension to April 30.

NRC work: Collection of scholarly activity (publications, etc.) and use of all data including allocations to characterize programs: May 2007 and following

NRC releases data to the public: End of 2007. Many are skeptical that NRC will meet this target.

NRC publication: 2008

UCB NRC Steering

  • PBA with John Stevenson: Weekly Tues 3-4
  • Carol and Susan and John with PBA, Wed /14, 12:30-2:30, PBA

Last revision 01/23/07

PBA Home  |   Strategic Planning  |   Institutional Research & Analysis |  Budget & Finances  | 
Questions? Comments? |  Legal & Trademarks |  Privacy
15 UCB, University of Colorado Boulder, Boulder, CO 80309-0015, (303)492-8631
  © Regents of the University of Colorado