PBA Home > Institutional Research & Analysis > Outcomes Assessment > AOC meeting notes 3/22/01

Assessment Oversight Committee (AOC) meeting notes 3/22/01

Handouts:

1. Vice Chancellor DiStefano's Charge to the Assessment Oversight Committee.
2. Draft of assessment guide for units.

Grant, introduction, distribution of handouts

• Invited comments on guide draft. May need to add section on "now what do I do?" for unit assessment coordinators.
• Noted and distributed copies of charge.
• Called for suggestions for replacement committee members for Copley and Main, who have withdrawn. Dave Chiszar (psychology) and Lorrie Shepard (education) mentioned. Grant asked for suggestions to be passed to him in near future. Someone from Boulder Faculty Assembly?
• Announcement of AAHE Assessment Conference in Denver, June 23-26. Looked for more info on web site; not much there on conference. Money available for those who want to go. Could send 1-4 teams.

Member comments/reviews of individual units reports currently posted on PBA Outcomes Assessment website. Each committee member was assigned 3-4 to review after December 00 meeting.

Themes from the individual unit reports

Units cited for making good use of assessment :

• Geography
• Classics
• History
• UWRP
• Women's Studies
• American Studies
• East Asian Languages and Civilizations
• Theater/Dance
• Film Studies
• Political Science
• International Affairs

What committee members value in the assessments

• External review sometimes.
• Not just multiple choice tests.
• Not just satisfaction.
• Use and discussion of results.
• Use in changing curriculum.
• Involving all the students, not just best or volunteers.

Writeups posted to web need clear record of date submitted and date of last assessment activity. And less coverage of long-ago activities.

Opportunities and challenges are clearly different for:

• Large and small units ("small" meaning around 30 or fewer seniors a year). Small units can assess the entire experience of every student. Larger units can sample student work (e.g., 1 of every 10 essays in 3 senior courses) or students themselves.
• Units emphasizing skills vs. knowledge (how are these defined/divided?)
• Units into which students declare early (frosh, soph) vs. later.
• Units with vs. without a common required senior-year course or project.

Discussion of next steps raised several issues

• What are, should unit obligations be for assessing effectiveness of services to non-majors? UWRP does much in writing, but there's little activity outside that.
• What are, should be our obligations and interests in "general education" or campus-wide goals vs. discipline-specific goals?
  • CCHE's QIS item explicitly cites "skill and knowledge in the major discipline." No mention of general education
  • NCA is interested in general education and campus-wide goals (and in the campus' stating such goals)
  • NCA also explicitly asked for graduate-level assessment
  • Perhaps AOC could/should be coordinating across campus - gathering and distributing information on what is going on in
    • Departments re discipline specific skills/knowledge (as now)
    • A&S core committee re general education. Oliver Gerland, chair. At least two AOC members are former A&S core committee members.
    • Other colleges re their own accreditation standards, efforts. E.g., ABET for Engineering. Also Music, Journalism, Business.
• Do the web assessment reports well reflect what has actually gone on? E.g., we suspect Business has engaged in significant assessment activity, but not reported it. Each Engineering department does something different on assessment, but there's just one overall report.
• Even in departments where assessment has led to change, has the effort been worth the benefit?
• What do we really want departments to do, and what means can and should be used to ensure compliance?

Next steps

• AOC members
  • Comments on draft guide for units, to Mike
    • In particular, does it tell a coordinator what to do? Should it?
• Mike contact A&S core committee and other colleges re accreditation standards, efforts. E.g., ABET for Engineering. Also Music, Journalism, Business. Goals: Find out what's going on that's not reflected in the outcomes site writeups, and consider assessment of both discipline-specific and campus-wide goals
• Mike and PBA, for the AOC (when guide for units is ready)
  • Report/feedback to units that were reviewed by committee members March '01.
    • Should communicate with departments privately
    • Should include some general material as well as feedback on the unit's submission/activities.
    • Reporting back to departments simply yes or no doing well not sufficient.
    • From committee, not just PBA. Letter from VC.
  • Prompt departments that have not reported in some time
  • Mike, with AOC advice: Feedback to units that submit reports
• AOC: Consider NSSE and senior survey results when available

Other next steps considered/suggested.

• Summary document about general patterns we've seen.
  • Something comprehensive. "Norms and Best Practices" and "Concerns."
  • Cite good departments by category (see categories under "themes")
  • Guidelines, concerns by category.
  • Take test case, try to find "what are problems in trying to do a successful outcomes assessment? Document (or part of the document) then is - "How you can solve certain problems in assessment."
• Don't want to alienate departments but want some consistency, so ask
  • Methods and goals tied? How?
  • When most recent assessment? What is plan for next?
  • How is assessment used for curriculum change?
  • How to assess resultant improvement?
  • Or just ask "how have you used?"
  • But some said, do not ask departments anything until the committee has produced something
• Mike Grant will write initial drafts dealing with above step . He notes that Committee is/has been thinking mostly at "whole campus" level, although most of the action is at the unit level.

PBA: PS-- L:\IR\OUTCOMES\aoc\notes010322.doc - 5/14/01.

Last revision 03/27/06