Report of the
Boulder Faculty Assembly Administrator Appraisal Committee[1]
Concerning
Paul Voakes, Dean, School of Journalism and Mass Communication,
University of Colorado at Boulder
The Administrator Appraisal
Program (AAP) seeks to provide in-depth feedback with a high rate of faculty
response, in order to assure a representative survey. Faculty have the opportunity to provide AAP
feedback to the review/ reappointment process when the president, chancellor,
provost, or the dean of their school or college is undergoing the third- or
fifth-year review. Faculty are also requested
to complete and return a “BFA Satisfaction Survey” addressing campus-wide
concerns such as salary and benefit programs. In regard to the appraisal of administrators,
the surveys contain questions addressing the effectiveness of each
administrator's performance in key areas. Various questions--addressing general
administrative style, salary process, and diversity--are common for all
administrators. Responses are solicited using an effectiveness scale of
1 to 5, in which 1 = very ineffective, 3 = effective, and 5 = very effective (the committee has interpreted the ratings in between as 2 = less than effective, 4 = more than effective), plus a "Don't Know" option.
The evaluation process for Dean Voakes was designed to take into account the broad scope of faculty across the School. While chairs, directors, associate and assistant deans have regular opportunities to interact with the dean, most individual faculty members do not, and therefore may be less well informed or simply uninformed about many aspects of the dean’s performance.
In an attempt to address the problem of different levels of familiarity, the AAP Committee sent the questionnaire to all faculty designated as eligible by the BFA, and then analyzed the responses from three groups among the recipients. The first group of recipients was all faculty within the School of Journalism and Mass Communication. [2] There were 25 responses of a possible 26 in this group, a response rate of 96%. The “Satisfaction Survey” in which faculty are asked to rate overall conditions in the University had a response rate of 88%, 23 of 26 faculty responding. The third group comprised 7 faculty who were in a position to have a close working knowledge of the dean’s work; 100% of the “knowledgeable” faculty responded to the survey. It is important to note that all knowledgeables, selected at the recommendation of Dean Voakes, were also tabulated in the overall response, thus are included among the 25 of the 26 faculty responding.
The AAP Committee and Boulder Faculty Assembly have agreed, on the advice of faculty who specialize in survey methods, that a 60% return rate is needed for a representative statistical study. The knowledgeable group met this requirement, with a 100% response rate. The response rate for the entire faculty was 96%, also well above the 60% threshold.
The AAP Committee hopes to provide a developmental picture of strengths and areas for improvement for administrators standing for reappointment in AY 2005-2006. Broadly viewed, we define four general categories in evaluating responses regarding Dean Voakes:
Strengths to Build On represent areas of the administrator’s performance which were rated as highly effective by a substantial majority of the faculty (60% or greater rating Dean Voakes a 4 or 5).
Assets to Protect were areas where at least half of the responding faculty found the administrator’s performance to be better than effective (in which 50% - 59% of respondents gave ratings of 4 or 5). Thus, any reallocation of effort to correct weaknesses or changes in priorities should try to protect these strengths.
Issues to be Mindful Of were judged to be effective or better by the majority of the respondents, but less than effective by a significant minority of respondents (25% - 39% gave ratings of a 1 or 2).
Areas that Need Improvement were areas judged by a significant fraction of the faculty as unsatisfactory (40% or more gave ratings of 1 or 2).
The survey of all respondents revealed a rather bi-modal picture of the faculty’s opinion of Dean Voakes’ performance on a significant number of the questions. For example, questions 1 through 3, 8-10, 13, 14, 18, 20, 21 and 24 showed a higher than 50% rate of return rating him “effective to very effective”(ratings of 3,4 and 5), while 40% of the faculty considered him to be “ineffective” or very ineffective” (ratings of 1 and 2) in those very same response categories. All but one of the questions had a response rate of greater than 30% in the 1 and 2 rating; all but one of the questions had a response rate at or greater than 30% rating Dean Voakes a 4 or 5. Knowledgeables’ responses were far more positive: Only two of the questions yielded responses of lower than 50% in the 4 rating levels, and 17 of the 24 were above the 70% response rate for 4 and 5 rating levels.
There were two categories which yielded more than a 60% response rating of 4 and 5, from among all faculty respondents: They were:
Being receptive to the concerns of students
Representing the School to external audiences.
To demonstrate the difficulty of analyzing these data, the chart, below, was created to illustrate the responses above 60% among just the knowledgeable faculty, compared with the percentage of responses from all participants in the 4&5 ratings (Column II), and the percentage of responses from the larger group responding in the 1&2 ratings (Column III):
I II III
Knowledgeable 4& 5 Ratings: 4&5’s ALL: 1&2’s ALL:
· Providing vision, strategic plan (86%) (44%) (40%)
· Acting with integrity (86%) (56%) (40%)
· Supporting high-quality teaching (86%) (48%) (30%)
· Supporting high-quality research (83%) (41%) (36%)
· Supporting high quality service (86%) (42%) (38%)
· Recruiting new faculty (86%) (52%) (36%)
· Making decisions in a timely fashion (71%) (38%) (46%)
· Creating an atmosphere of trust (67%) (38%) (54%)
· Supporting/Mentoring assistant profs (71%) (38%) (43%)
· Making progress towards diversity goals (71%) (48%) (35%)
· Being receptive to concerns of faculty (86%) (52%) (40%)
· Being receptive to concerns of staff (80%) (57%) (43%)
· Being receptive to concerns of students (86% (67%) (22%)
· Managing conflicts among faculty (71%) (30%) (57%)
· Treating faculty fairly and inclusively (86%) (54%) (42%)
· Communicating in an effective manner (71%) (42%) (46%)
· Representing School on campus (83%) (50%) (44%)
· Representing School to ext. audiences (83%) (63%) (32%)
· Making sound budget decisions (86%) (56%) (31%)
:
Assets to Protect (50% - 59% ratings of 4 or 5)
The following categories of assets in both the entire
faculty and
knowledgeable faculty samples were rated 4 or 5:
·
Acting with integrity
·
Recruiting new faculty
·
Being receptive to concerns of faculty
·
Being receptive to concerns of staff
·
Treating faculty fairly and inclusively
·
Representing School on campus
Making sound budget decisions
Issues to be Mindful Of (25% - 50% ratings of 1 or 2)
The following were judged to be effective or better by the majority of the respondents, but less than effective by a significant minority of respondents (25% - 50% ratings of 1 or 2):
Providing vision, strategic
plan
Acting with integrity
Supporting, improving quality of doctoral programs
·
Supporting high quality teaching
·
Supporting high quality research
·
Supporting high quality service
·
Supporting/mentoring assistant professors
·
Making progress towards diversity
Being receptive to concerns of faculty
Being receptive to concerns of staff
Treating faculty fairly and inclusively
Communicating in an effective manner
Representing School on campus
Representing School to external audiences
Making sound budget decision
The following categories
received a rating of “Effective” or better (3, 4 and 5) from less
than a
majority of the respondents , and a rating of 1and 2 from more than 40%
of the respondents:
·
Creating an atmosphere of trusSupporting the continued development of
senior faculty
·
Managing conflicts among faculty
·
Managing conflicts among staff
·
Appropriately involving faculty in decisions
·
COMPARISON TO BFA SATISFACTION SURVEY RESULTS
The BFA also asked the respondents in the SJMC to complete a satisfaction
survey which asked 20 questions about their satisfaction with the level of
university support. The number of
respondents was 23 out of 26 surveyed, a
response rate of 88%. Respondents rated their satisfaction with conditions in
the University generally somewhat higher than they rated Dean Voakes in the
categories listed below. The
Satisfaction Survey reported areas of strength, defined as 60% of both knowledgeables
and all faculty respondents in the 4 or
5 ratings as follows:
·
Teaching responsibilities
·
Relationships with colleagues
·
Library access to outside resource
Among the knowledgeables only, the faculty also considered the following
as a strength:
·
Teaching responsibilities
·
Relationships with colleagues
·
Involve faculty in Library holdings decisions
·
Library access to outside resources
·
Education and training support offered by the Library
Three areas were listed as Assets (ratings in the 4 or 5 category by
50-59% of all respondents):
· University support for research and creative
work
· Education
and Training support offered by the library
Among the knowledgeables
only, respondents listed the following categories as assets:
·
Classroom facilities
·
University support for research and creative work
·
Faculty governance/progress towards shared governance
The faculty listed 5 Issues to be Mindful of (25%-39% of all respondents
rate the following category 1 or 2):
Technological support in teaching
Departmental support services
· Space and facilities
· Other Benefits, including retirement
· Evaluation of teaching
.
Knowledgeables listed the
following 4 Issues to be mindful of:
Classroom facilities
Benefits other than health plan
Faculty governance/progress of shared governance
University efforts to recruit/retain diverse faculty
The faculty also listed 7 areas In Need of Improvement (40% or more
respondents rate the following categories 1or 2):
· Support for soliciting outside money
· Salaries compared with peer institutions
· Equitable distribution of salary
· Current health plan
Faculty
governance/progress towards shared governance
· University efforts to retain diverse faculty
· University efforts to retain diverse
undergraduates
The knowledgeables listed the following areas In Need of Improvement:
Technological support in teaching
Departmental support services
Support for soliciting outside money
Space and facilities
Salary compared with peer institutions
Current health plan
University efforts to retain diverse undergrads
Evaluation of teaching
Overall, the Satisfaction Survey indicates substantial dissatisfaction in those areas involving salary. The highest percentages of dissatisfaction were for Salaries in comparison with peers (61% ratings of 1 and 2), and Equitable distribution of salaries (53%). Among these, only one of them, Equitable distribution of salaries, is under Dean Voakes’ control.
CONCLUSIONS
Dean Voakes scored very
high nearly accross the board among knowledgeable faculty members. Among this cohort there are no areas that meet
the BFA threshold as “needing improvement,” and not even areas that meet the
threshold “to be mindful of.” However, among the sample of all respondents, there
are many response categories that meet both of these thresholds. Categories
appearing to be of greatest need for improvement are in the areas of trust,
managing conflicts among faculty and staff, and supporting the continuing
development of senior faculty. These were themes that were often emphasized in
the written feedback provided by respondents, who also referred to low morale, and
a sense that some faculty were perceived to be included, or even part of an
“inner circle,” while others were perceived to excluded, not consulted. The bi-modal
tendencies of the data can be seen to reflect such comments. With respect to
the Satisfaction survey, there appears to be great satisfaction with categories
associated with the Library, and general dissatisfaction with issues of
material support, including salaries and benefits.
Issues of morale,
and feedback that indicates improvement is needed in managing faculty conflict
represent extraordinarily complex and difficult isssues to address. It may
prove helpful in this situation to appoint a mentor for Dean Voakes, perhaps an
ombuds person as well, to assist him with making progress in these areas.
[1] The
members of the committee are: Sedat BiringenJane Elizabeth Dougherty,
Paul
ErhardWayne Gazur, Lawrence FreyJudith Glyde,
Ken
IwamasaAnne Heinz, Richard LaverBruce
Henderson, Clayton Lewis, Uriel Nauenberg (chair), Lynn
Ross-Bryant, Jeff Schiel, Robert Schulzinger, Daniel Sher.
[2] Faculty isare
defined as all members of the Boulder Faculty Senate, including all rostered
contract instructors .