Report of the

Administrator Appraisal Committee

Concerning

Dean Harold Bruff, School of Law

December 8, 2000

The Administrator Appraisal Program (AAP) has been developed and directed by the Boulder Faculty Assembly (BFA), with technical support by the Office of Planning, Budget and Analysis (PBA) since the 1992 pilot year. In the past the AAP consisted of a short questionnaire mailed to all faculty on an annual basis, seeking input on the faculty member’s dean plus all campus-wide administrators. While the AAP questionnaire presented an opportunity for written comments from faculty, the principal evaluation tool was grading an administrator’s performance on the familiar scale of A-F with reference to a limited number of factors.

The revised AAP used as the basis for this report is different in several respects. First, the number of administrators under review is limited. In the revised AAP, faculty have the opportunity to provide AAP feedback only when the president, chancellor, vice chancellor or the dean of their School or College is undergoing a third- or fifth-year review. Second, faculty are requested to complete and return a "BFA Satisfaction Survey" speaking to campus-wide concerns such as salary and benefit programs. Third, the survey questionnaire dispenses with the A-F grading, substituting scaled opinions of administrator effectiveness in more areas. In that regard, the survey questionnaire for Dean Bruff contained 20 questions. Some questions were common for all administrators, but other questions were tailored to the specific circumstances of the administrator, School, or College.

On October 3, 2000, PBA sent the survey packages to the campus mailboxes of all fifty-two (52) eligible faculty raters in the School of Law. A notice reminding faculty to respond was published in the October 12, 2000 Silver & Gold Record. An e-mail reminder was sent to all eligible faculty raters on October 18, 2000. Finally, the Chair of the AAP Committee, Cathy Comstock, broadcast Voice Mail reminders on October 9, 16, and 19.

At the request of the AAP Committee, each administrator furnished a two-page description of his "Activities and Accomplishments" for the period under review. Accordingly, the survey package for Dean Bruff contained this information, focusing on the School of Law building project, capital campaign, faculty hiring and retention, academic programs and support, students, library, budget and administration, and Dean Bruff’s teaching and service. Reflecting a fifth-year review, Dean Bruff’s information sheet reported activities and accomplishments for 1996-2000 (a copy of the information sheet is appended to this report).

Questions 1-10 and 16 of the survey questionnaire for Dean Bruff consisted of questions asked for all administrators under review. Questions 11-15 and 17-20 were constructed in light of specific circumstances and concerns at the School of Law. Each question requested a numerically scaled response of 1-5, ranging from "Very Ineffective" (ascribed a value of 1) to "Very Effective" (ascribed a value of 5). Each question was followed by a blank space inviting "COMMENTS", and the general instructions noted that "[i]f you need additional space for comments, include them on a separate piece of paper with reference to the question number and the administrator." A copy of the survey questionnaire is appended to this report.

As of the October 25, 2000 cut-off date for processing, thirty-three (33) of the fifty-two (52) eligible faculty raters in the School of Law had completed and returned the survey package as it pertained to Dean Bruff. This response rate of 63% is much higher than the 47% response rate produced by the 1999 survey conducted under the prior AAP methodology.

As noted above, Dean Bruff prepared an "Activities and Accomplishments" information sheet that describes the highlights of his deanship for the period 1996-2000 in eight functional areas. The questions on the appraisal form largely reflect issues pertaining to those areas. The organization of this report will first follow the functional areas established in the Activities and Accomplishments Information Sheet and then review the findings from a developmental perspective.

The faculty raters gave Dean Bruff his highest marks in the two areas that were also listed as items #1 and #2 in his Activities and Accomplishments Information Sheet, the "[School of Law] Building Project" and "The Capital Campaign", respectively. The faculty raters evaluated the effectiveness of his "moving the building project for the law school forward to completion" with a mean score of 4.3 (again, with 5.0 ("Very Effective") as the highest possible score) and effectiveness in "raising funds for the building and other law school programs" with a mean score of 4.0.

In general, the responding faculty strongly acknowledged Dean Bruff’s effectiveness in this functional area. As an absolute count, 27 of the 33 faculty raters, or 81.82%, gave Dean Bruff’s efforts with respect to the School of Law Building Project a score of 4 or 5, while 24 of the 33 (72.73%) faculty raters gave Dean Bruff’s efforts with respect to The Capital Campaign a score of 4 or 5.

"Faculty Hiring and Retention" was the third area discussed in Dean Bruff’s Activities and Accomplishments Information Sheet. In this broad category he included faculty hiring and retention, the diversity of new hires, faculty salaries, and summer research grants. The faculty raters gave Dean Bruff good marks in terms of "foster[ing] an atmosphere that supports diversity in the recruitment of faculty, staff, and students" and "effective leadership in hiring decisions" with mean scores of 3.7 and 3.2, respectively. Apparently reflecting the loss of 4-5 faculty to lateral moves described in the Activities and Accomplishments Information Sheet, the faculty raters gave a mean score of 2.9 to "effective[ness] in the retention of faculty."

While the faculty raters gave Dean Bruff good marks for "encouraging and rewarding faculty scholarship" with a mean score of 3.6, "effective[ness] in leading the law school to an equitable, merit-based increment salary system" was accorded a mean score of 3.0.

Two questions on the BFA Satisfaction Survey addressed compensation. On Question #12 "salary relative to your peers at comparable research institutions" the law school raters gave a mean response of 2.3. On Question #13, "equitable distribution of salary, given the current salary structure and resources at CU-Boulder" the law school raters gave a mean response of 2.4, significantly lower than the 3.0 mean response to the question in the preceding paragraph that was couched in terms of Dean Bruff’s leadership in salary matters.

Several of the appraisal form questions raise broader issues that can best be organized as a separate category focusing on Dean Bruff’s leadership or management style.

The faculty raters gave Dean Bruff high marks for "acting with integrity" producing a mean score of 3.8, but gave him a mean score of only 2.8 in terms of "creating an atmosphere of trust." This seeming contradiction is not explained in the very limited transcribed comments expressed by the participating faculty.

The faculty raters gave him more modest marks for "listening to faculty, staff, and students (whether or not you agreed with his decisions)" with a mean score of 3.3, and "treating faculty of all levels, including instructors, in a fair and inclusive way" with a mean score of 3.3. Listening and inclusiveness are apparently different matters from decision processes, and the faculty raters gave Dean Bruff a mean score of 2.9 with respect to "appropriately involving faculty in decisions."

In terms of whether Dean Bruff "effectively articulated an appropriate vision for the School of Law" the faculty raters gave a mean response of 2.8. The responding faculty were sharply divided. Of the 33 responding faculty, 14 of the faculty raters gave him a score of 1 or 2 on this question, while 13 of the faculty raters gave him a score of 4 or 5 on this question, producing a standard deviation of 1.50.

The Faculty Hiring and Retention category of Dean Bruff’s Activities and Accomplishments Information Sheet speaks to efforts "to improve relationships within the faculty." Dean Bruff’s lowest marks in the survey were on questions generally relating to faculty or staff conflicts, as well as responding to broader "problems." In "constructively managing conflicts among faculty or staff" the mean score was 2.5, with 17 faculty rating his effectiveness as 1 or 2, and only 9 rating it as 4 or 5. In spite of this appearance of conflict among the faculty, the response to BFA Satisfaction Survey Question #5 that speaks to an admittedly narrower "collaborative relationships with colleagues with whom you work on a routine basis" gained a mean score of 3.4.

In "providing decisive action and appropriate proactive responses to potential problems arising within the School of Law community" the mean score was again 2.5 (and the split of marks was again 17 giving a 1 or 2 grade and 9 giving a 4 or 5). In the potentially related question of "making decisions in a timely fashion" the mean score was 2.7.

The faculty raters gave Dean Bruff good marks for "supporting innovation in teaching" with a mean score of 3.6, and "supporting innovation in research" with a mean score of 3.7. Similarly, the faculty raters gave him a mean score of 3.4 "in distributing resources for infrastructure and support programs."

On the BFA Satisfaction Survey the law school raters’ mean response to Question #1 dealing with "teaching responsibilities and opportunities" was 4.1. With respect to Question #2 dealing with "teaching resources" the mean response was 3.4. Question #9 dealing with "departmental support services" won a mean response of 3.7.

In evaluating Dean Bruff’s "effective[ness] in optimizing the quality of the student body" the faculty raters gave him a score of 3.3.

A concentration of opinion is evident among a subset of the respondents. For example, out of 102 grades of "1" given to Dean Bruff on all 20 questions, 93 (91.41%) can be attributed to 8 of the 33 (24.24%) responding faculty members.

The revised AAP utilizes scaled responses that cannot be converted to a letter grade since there is no weighting of the individual questions. The goal of this AAP was to conduct an evaluation that would provide feedback that would facilitate professional development. In this spirit, this section of the report highlights the strengths and weaknesses of the dean along four categories.

The survey data disclosed three areas of pronounced strength for Dean Bruff, as demonstrated by two-thirds or more of the faculty raters ranking his effectiveness in this area as a 4 or 5: (1) moving the building project for the law school forward to completion (mean score of 4.3); (2) raising funds for the building and other law school programs (mean score of 4.0); and (3) acting with integrity (mean score of 3.8).

Assets To Protect•

The survey data disclosed four areas of solid effectiveness, as demonstrated by approximately one-half or more of the faculty raters ranking his effectiveness in this area as a 4 or 5, and producing a mean score far above simply "Effective": (1) supporting innovation in teaching (mean score of 3.6); (2) encouraging and rewarding faculty scholarship (mean score of 3.6); (3) supporting innovation in research (mean score of 3.7); and (4) fostering an atmosphere that supports diversity in the recruitment of faculty, staff, and students (mean score of 3.7).

Issues To Be Mindful Of•

The survey data disclosed eight areas that produced a mean score at approximately, or somewhat above, "Effective": (1) retention of faculty (mean score of 2.9); (2) involving faculty in decisions (mean score of 2.9); (3) leading the law school to an equitable, merit-based increment salary system (mean score of 3.0); (4) treating faculty of all levels, including instructors, in a fair and inclusive way (mean score of 3.1); (5) effective leadership in hiring decisions (mean score of 3.2); (6) listening to faculty, staff, and students (mean score of 3.3); (7) optimizing the quality of the student body (mean score of 3.3); and (8) distributing resources for infrastructure and support programs (mean score of 3.4).

Areas For Consideration•

Finally, the survey data disclosed five areas producing a mean score below "Effective" in varying degrees: (1) providing decisive action and appropriate proactive responses to potential problems arising within the School of Law community (mean score of 2.5); (2) constructively managing conflicts among faculty or staff (mean score of 2.5); (3) making decisions in a timely fashion (mean score of 2.7); (4) articulating an appropriate vision for the School of Law (mean score of 2.8); and (5) creating an atmosphere of trust (mean score of 2.8).

•Most of the responding faculty considered Dean Bruff to be highly effective in his time-intensive, externally directed pursuits involving the law school building project and fundraising.

•The picture of Dean Bruff’s performance in some internal matters is clearly mixed. On the one hand, Dean Bruff’s greatest challenges appear to lie in the areas of articulating a vision for the School of Law and acting more decisively and opportunely in the face of potential problems (whether of a personnel nature or otherwise). On the other hand, the survey data suggests more than effective performance in a number of core academic functions, such as supporting innovation in teaching and research, supporting faculty scholarship, and supporting diversity in the recruitment of faculty, staff, and students. Furthermore, in spite of the apparent divisions of opinion on some matters of internal functions, the faculty raters gave him high marks in terms of acting with integrity.

•The remaining areas span a number of different functions (such as retention of faculty, salary decisions, inclusiveness, leadership in hiring, optimizing the quality of the student body, and distributing resources for infrastructure and support programs). The faculty raters generally found Dean Bruff’s performance in these areas to be at, or somewhat above, effective.