APS DEI Action Plan Assessment Guide

Amy Moreno-Sherwood, Director of Inclusive Culture

College of Engineering and Applied Science

The purpose of the APS DEI Action Plan Assessment Guide is to provide students, staff, and faculty with structure and guidance to effectively design, implement, and assess localized DEI initiatives and efforts.  Developing an assessment plan will enable the department to demonstrate the progress and impacts of our various initiatives, continuously build upon and enhance our various DEI efforts, apply for funding, and overall answer the important questions: what are you doing, why are you doing it, and what difference are you making? Use the following guide to create an assessment strategy for our DEI initiatives.

Definitions

Goal– Broad, long-term statement of outcomes you want to achieve. Gives overarching direction and defines where you want to be to help the unit realize its vision. Example: Eliminate equity gaps in retention.

Outcome – What you hope to achieve, the change expected as a result of the tactics/actions taken. A programmatic outcome will measure the result of an action against its intended or projected results. Example: Applications from individuals from underrepresented groups increase after implementing a recruitment event. A learning outcome would be what a student will know or be able to do after an event or course. Example: 100% of students indicate their correct name and pronouns are consistently used in the classroom, and 100% of students indicate at least one theorist or highlighted expert within the course that shares an aspect of their cultural background.

Tactics – Specific programs, initiatives, and actions that are necessary to accomplish the goals and objectives.* How you plan to achieve or make progress toward the larger goal.

Target - Result or value that will represent success at achieving a given outcome/objective.

Measurement/Metrics – The type of measure or tool used for assessing the objectives and targets. Use indicators (qualitative or quantitative) which are specific measures or themes used to track progress:

  • Progress Indicator* - Formative data that appear during the planning and operation of an activity and help drive the desired outcome, normally measuring some aspect of progress in terms of intermediate processes and activities. Example: # of students participating in DEI training, first-year retention rates, and annual review of faculty, administrators, and staff
  • Outcome Indicator* - Measure that captures summative information and historic performance. Example: 6-year graduation rates, perceptual measures of the campus climate (captured through survey data), and number of DEI-themed publications produced annually.

* From "Strategic Diversity Leadership" by Dr. Damon Williams

APS DEI Action Assessment Plan

Action Title

Goal

Outcome(s)

Tactics

(How to Achieve the Outcome)

Baseline

(Current Status)

Target

(Desired Future Status)

Metrics

What measures or tools will be used to assess the objectives and progress toward targets?

●Progress Indicator(s)

●Outcome Indicator(s)

Assessment Timeline

(Outline frequency of data collection and sharing results)

Responsible Parties

Action Assessment Plan - EXAMPLE

Action Title: APS Friday seminar series

Goals:

  • Provide opportunities for scientists from diverse backgrounds  to speak about their science and optionally DEI issues.
  • Improve retention and success in our APS community by providing a sense of belonging, by demonstrating that we value scientists from diverse backgrounds, and normalizing a more diverse view of scientists
  • Network and build working relationships with a pool of diverse junior scientists whom we can potentially recruit for future faculty positions in APS.
  • Provide an opportunity for our students to network with a diverse group of junior scientists who excel in their field.

Outcome(s):

  • Having a well-attended weekly seminar.
  • The seminar should be well known and sought out in the broader astronomy community
  • Members of our community have ongoing communication/collaboration with the speakers.
  • Speakers are recruited to apply for APS postdoc and faculty positions.
  • Increased awareness of scientists from diverse backgrounds to students and faculty.

Tactics:

  • Change name from “CASA/JILA Friday seminar” to “ seminar” or something that is more understandable and elevates the status, and include planetary/solar scientists as well.
  • Introduce and roll out new seminar, with new name, organized email distribution list, blurb about goals and why people should attend, and link to self-nomination.
  • Invite junior scientists from diverse backgrounds to give weekly Friday seminars
  • Advertise well within our community, public forums (astrobites, etc)
  • Create a self-nomination form for speakers.
  • Emphasize that they are invited to talk about scientific research and/or topics of their choice
  • Pay honorarium/travel expenses for speakers
  • Standardize the invitation email list and have a staff person administer the lists, and make sure talks are broadly advertised in APS/CASA/JILA/LASP (and maybe externally -- SWRI, NSO, etc).
  • In addition to title/abstract, include a short biography of the speaker that humanizes them and gives an overview of professional and personal interests. This is to make the seminar and speaker more approachable. Emphasizes the human side of scientists -- you can be a scientist and a person with diverse interests.
  • Give seminar credit for attending – or assign as seminar topic? (Not consensus on this tactic.)
  • Create a seminar committee for continuity and to distribute work. (Colloquium committee? R3DAT?) Want to be careful not to put this all on postdocs. Big burden, and postdocs turn over quickly.
  • Make high-quality recordings of the talk (but not Q&A), and host those.
    • Benefits the speaker -- they can point others to the talk
    • Benefits APS -- community members can watch after the event.
  • Define niche to distinguish this series from APS colloquia: Junior scientists, and also more casual in format. More room for discussion.

Baseline:

  • During AY20/21, CASA/JILA seminars were held remotely on a regular basis. Excellent speakers, typically but not exclusively junior scientists from underrepresented groups.
  • Science talks, but optionally on DEI issues at their university, or that they have encountered. Made it clear that we invited them because of their science, not because of their identity. 
  • Attendance ranged from 8-20 people.
  • One announcement was sent out at the beginning of the semester to all. Angela managed email list, Troy got on the list, and then forwarded this to APS lists. Advertisement to APS folks was somewhat spotty.
  • Was hosted as a JILA event, and treated like a JILA seminar. Somewhat opaque process.
  • Over the summer 2021, the ball was dropped because it was unclear who was going to run this during AY21/22.

Target:

  • Attendance regularly exceeds 25 people.
  • Two or more department members have ongoing contact with any of the speakers, including collaborating on scientific projects and/or mentoring
  • Five external speakers per year contact us asking to speak, or fill in self-nomination form
  • At least one speaker applies for a job (postdoc or faculty position)  in our department.

Metrics:

What measures or tools will be used to assess the objectives and progress toward targets?

Progress metrics

  • Track attendance and attendee affiliation/role
  • Survey attendees with questions related to our goals, how many people have had ongoing contact with the speaker, etc.
  • Track how many external speakers have contacted us asking to speak.
  • Track if any speakers applied for jobs in our department.
  • Track number of self-nominations for speaking slot (increased visibility of series in broader astro community)

Outcome Indicator(s):

  • Co-authored publications with speaker and CU affiliates
  • Speakers serve on APS dissertation and/or comps committees
  • Speakers offered postdoc or faculty jobs at CU
  • Improvement in sense of belonging to APS in future climate surveys

Assessment Timeline:

Mid-October: roll out renamed series. Assessment with metrics in Spring 2022, and every AY thereafter.

Responsible Parties:

  • Ann-Marie Madigan -- faculty advisor. Responsible for continuity.
  • Joe Simon -- organization, speaker invitations, advertising talks to APS
  • Nils Halverson -- funding and faculty whip
  • Aimee Schechter -- speaker hunter and graduate student whip
  • Graduate Program Coordinator -- web page and administrative support