Report of the

Administrator Appraisal Committee

Concerning

Dean James Williams, University Libraries

February 2001

The Administrator Appraisal Program (AAP) has been developed and directed by the Boulder Faculty Assembly (BFA), with technical support by the Office of Planning, Budget and Analysis (PBA) since the 1992 pilot year. In the past the AAP consisted of a short questionnaire mailed to all faculty on an annual basis, seeking input on the faculty member’s dean plus all campus-wide administrators. While the AAP questionnaire presented an opportunity for written comments from faculty, the principal evaluation tool was grading an administrator’s performance on the familiar scale of A-F with reference to a limited number of factors.

The revised AAP used as the basis for this report is different in several respects. First, in order to provide more in-depth feedback, the number of administrators under review is limited. Faculty have the opportunity to provide AAP feedback to the review/ reappointment process when the president, chancellor, vice chancellor or the dean of their school or college is undergoing a third- or fifth-year review. Second, faculty are requested to complete and return a "BFA Satisfaction Survey" speaking to campus-wide concerns such as salary and benefit programs. Third, the survey questionnaire dispenses with the A-F grading, substituting scaled opinions of administrator effectiveness in more areas. In that regard, the survey questionnaire for Dean Williams contained eighteen (18) questions. Some questions were common to all administrators, but other questions were tailored to the specific circumstances of the administrator, School or College.

On October 3, 2000, PBA sent the survey packages to the campus mailboxes of all forty-three (43) eligible faculty raters in the University Libraries. A notice reminding faculty to respond was published in the October 12, 2000 Silver & Gold Record. An e-mail reminder was sent to all eligible faculty raters on October 18, 2000. Finally, the Chair of the AAP Committee, Cathy Comstock, broadcast voice mail reminders on October 9, 16 and 19.

At the request of the AAP Committee, each administrator furnished a two-page description of his "Activities and Accomplishments" for the period under review. Accordingly, the survey package for Dean Williams contained this information, wherein he focused on a variety of accomplishments under general headings of administration, faculty governance, and planning & development. Reflecting a third-year review, Dean Williams’ information sheet reported activities and accomplishments for 1998-2000.

Questions 1-10 and 16 of the survey questionnaire for Dean Williams consisted of questions asked for all administrators under review. Questions 11-15 and 17-18 were constructed in light of specific circumstances and concerns in the University Libraries. Each question requested a numerically scaled response of 1-5, ranging from "Very Ineffective" (ascribed a value of 1) to "Very Effective" (ascribed a value of 5). The questionnaire also permitted a response of DK (Don’t Know). Each question was followed by a blank space inviting "COMMENTS". The instructions noted that "…comments will be transcribed unless illegible," to assure the anonymity of the commenting faculty member.

As of the October 25, 2000 cut-off date for processing, twenty-eight (28) of the forty-three (43) eligible faculty raters in the University Libraries completed and returned the survey package as it pertained to Dean Williams. This response rate of 65% was substantially higher than the 37% response rate produced by the 1999 survey conducted under the prior AAP methodology. The AAP Committee had agreed, on the advice of several faculty who specialize in survey methods, that a 60% return rate would be needed. The 65% return rate for Dean Williams fell comfortably within the range where the results are truly representative of the opinions of the Libraries Faculty group.

The AAP Committee hopes to provide a developmental picture of strengths and weaknesses for administrators standing for review or reappointment in AY 2000-2001. Broadly viewed, we considered four general categories in evaluating responses:

Strengths to Build on represent areas of the Dean’s performance which were rated as highly effective by a substantial majority of the faculty respondents (over 60% of respondents gave ratings of 4 or 5). This category identifies areas of perceived strength that may be used to build consensus or achieve common goals.

Assets to Protect were areas where at least half of the responding faculty found the Dean’s performance to be better than effective (50% or more of respondents gave ratings of 4 or 5). Thus, any reallocation of effort to correct weaknesses or changes in priorities should try to protect these strengths.

Issues to be Mindful of were judged to be effective or better by the majority of the respondents, but less than effective by a minority of respondents (25%-35% gave ratings of 1 or 2). Thus, any significant change in attention to these areas could move performance to less than effective. Also, actions in these areas by the Dean must be accompanied with sensitivity to the legitimate concerns of faculty who found performance to be less than effective.

Areas in Need of Improvement were areas judged by a number of respondents as unsatisfactory (over 40% gave ratings of 1 or 2). Performance in this category should trigger a plan for improvement that is mindful of how changes in actions, priorities, and performance will maintain areas of strength while addressing concerns of faculty who rate performance as less than effective.

Dean Williams received a high proportion of ratings wherein his performance was judged to be effective or above. Approximately three-fourths (67% - 97%) of library faculty respondents judged the Dean to be effective to very effective (rating of 3, 4, or 5) on all questions. In fact, for sixteen of the eighteen questions, nearly four-fifths (79%-97%) of the respondents rated the Dean as effective to very effective.

The Committee defined these to be areas in which more than 60% of the respondents gave ratings of 4 or 5. Dean Williams received ratings in this category on fourteen of the eighteen questions.

The highest ratings (75% - 86% of respondents gave ratings of 4 or 5) were received by the Dean for his effectiveness in activities associated with future, long-term planning:

• Collaborative/consortial activities to create inter-institutional relationships with other libraries. (Q12)
• Alleviation of space problems through off-site storage planning. (Q14)
• Articulation of plans to build an endowment for collections and services, to construct a new Science Library, and to retrofit the Norlin Library. (Q13)
• Handling the dual issues of fiscal constraints and inflationary increases to journal costs through an aggressive journal cancellation project. (Q15)
• Appropriation of resources for state-of-the-art automated systems for internal management and provision of information services to the campus. (Q11)

High ratings (73% - 82% assigned ratings of 4 or 5) also were received by Dean Williams for his effectiveness in the following personnel- and process-oriented areas:

• Acting with integrity. (Q9)
• Articulating to campus administration the need for ongoing supplemental compensation plans for library faculty. (Q10)
• Listening to faculty, staff, and students. (Q1)
• Treating faculty of all levels, including Instructors, in a fair and inclusive way. (Q7)
• Fostering an atmosphere that supports diversity in the recruitment of faculty, staff, and students. (Q16)
• Establishing awards or other mechanisms to recognize and encourage library faculty excellence. (Q18)

Dean Williams’ support for innovation was recognized as strengths by 65% to 84% of respondents, who gave ratings of 4 or 5 for these areas:

• Supporting innovation in teaching. (Q3)
• Supporting innovation in research. (Q4)
• Supporting innovation in the overall operation of the Libraries. (Q17)

Dean Williams’ commitment to work collegially with faculty was recognized by 50% to 57% of respondents, who gave ratings of 4 or 5 for these areas. Also, average ratings strongly supported the Dean’s effectiveness (79%-82% gave ratings of 3, 4, or 5):

• Appropriately involving faculty in decisions. (Q8)
• Creating an atmosphere of trust. (Q6)

For the remaining two areas, 30%-34% of respondents assigned ratings of 1 or 2; however, the average ratings were strong enough for these areas to be considered effective (67% - 70% gave ratings of 3, 4, or 5):

• Making decisions in a timely fashion. (Q5)
• Constructively managing conflicts among faculty or staff. (Q2)

A review of the faculty ratings for Dean Williams reveals that his performance was never placed in the "Needs Improvement" category by 40% or more of respondents.

Two questions on the BFA Satisfaction Survey sought opinions from all university faculty raters regarding Norlin Library services and Norlin Library facilities and holdings. Of the 419 faculty who responded to the Satisfaction Survey, 361 (86%) rated services and 372 (89%) rated facilities and holdings. Norlin Library, however, is only one of seven libraries on the University of Colorado campus. Given the high percentages of response to these two questions, it was unclear whether some respondents considered just Norlin Library or whether they rated the branch library specific to their discipline and with which they likely were more familiar.

Norlin Library services were rated high (4 or 5 on the 5-point scale) by 46% of all respondents and rated low (1 or 2 on the 5-point scale) by 22% of all respondents. However, further analysis of the survey results shows that faculty in the College of Arts and Sciences and the School of Law rated services consistently lower overall than did faculty in other colleges. High ratings for library services ranged from 40% (Law) and 42% (Arts and Sciences), to 50%-63% from faculty of other colleges. Low ratings for library services ranged from 25% (Law) and 29% (Arts and Sciences), to 4%-17% from faculty of other colleges.

Norlin Library facilities and holdings scored lower than library services overall: high ratings from only 33% and low ratings from 37% of respondents. Respondents from the College of Arts and Sciences and the Libraries rated Norlin Library facilities and holdings substantially lower than faculty in other colleges. Only 25% of Arts and Sciences’ and 36% of Libraries’ respondents gave high ratings of 4 or 5. There were more respondents (47% and 48% respectively) from these two groups who gave low ratings of 1 or 2. Ratings of Norlin Library facilities and holdings by respondents from the other colleges varied from high ratings given by 39%-63% to low ratings from only 17%-23%.

A majority of the Libraries Faculty rated Dean Williams to be effective to highly effective in sixteen of the eighteen areas being appraised. He received particularly high ratings on several questions that spoke to long-range planning activities. The Dean’s collaborative efforts to forge inter-institutional relationships received highest ratings, followed closely by strong support for his handling of the journal cancellation project and long-term storage plans for the library.

Dean Williams’ support for innovation in teaching, research and in the overall operation of the Libraries all were recognized by the majority of raters as strengths in which he has been effective to highly effective.

Several activities within the personnel- and process-oriented category drew mixed responses. Dean Williams received high ratings in eight areas, most notably acting with integrity and articulating to campus administration the need for ongoing supplemental compensation plans for library faculty. However, opinion was divided about the Dean’s effectiveness in managing conflicts constructively among faculty or staff and his effectiveness in making decisions in a timely fashion.

Two questions on the BFA Satisfaction Survey solicited ratings from university faculty about Norlin Library services and Norlin Library facilities and holdings. Nearly half of all faculty who responded rated services high (4 or 5), although faculty in the College of Arts and Sciences and School of Law consistently rated services lower overall than did faculty in other colleges. On the question about library facilities and holdings, only one-third of faculty respondents assigned high ratings. Faculty in the College of Arts and Sciences and the Libraries consistently assigned lower ratings to facilities and holdings than did faculty in other colleges.

In summary, Dean Williams received ratings of effective to very effective from nearly four-fifths of library faculty respondents for sixteen of the eighteen appraisal questions.

Addendum: Activities and Accomplishments