Interview with Julie Volckens, Director of Assessment in the Office of Institutional Equity and Compliance & Stephanie Foster, Assessment Lead in the Center for Teaching & Learning - March 2021
Stephanie Foster
"Hello! We're here today with Julie Volckens. Julie is going to talk with us a bit about the Culture Survey that she has been running here at CU. Julie, why don't you tell us a little bit about yourself and what you do here at CU."
Julie Volckens
"Well, thanks for having me. I'm currently Director of Assessment in the Office of Institutional Equity and Compliance, OIEC. But I've been on campus doing assessment since about 2009 when I started out in Community Health. I then worked in assessment in Housing and Dining before coming to OIEC. And before that I was a social psychologist teaching out in Vermont. I guess I was looking for an opportunity to impact more people and think about things in a larger public health framework rather than an individual-level approach."
Stephanie Foster
"Wonderful! We're so glad to have you! I've asked you to come talk to us today about the Culture Survey. I understand this is a survey that you have created here at CU with others, and you've rolled out a pilot, and now you're getting ready to roll it out to the campus at large. So we'd love to hear more about that: what's it about, what did you set out to learn and how's it going."
Julie Volckens
"So in 2014 the campus, as it had done--for I think a couple of decades--the campus assessed undergraduate and graduate students with the Social Climate Survey. And from that survey, across administrations, we were able to see that there was a disparity in the experiences that students were having. Students of Color, LGBTQ+ (Lesbian, Gay, Bisexual, Transgender, and Queer or Questioning and others) students, and students with disabilities were having a less positive experience. That got us thinking about how we could do something more comprehensive with the campus including looking at the impact of other aspects of identity and broadening the scope to staff and faculty.
Through analysis, we identified those parts of the survey that were giving us the richest information. For instance, we found that 2014 first-year students’ sense of belonging, which includes feeling respected, supported, and valued, was a significant predictor of the likelihood that they would be retained at CU until the spring of their junior year. We also found that between 2014 and 2017, we were losing LGBTQ+ graduate students at a higher rate than non LGBTQ+ students; they were leaving before completing their degree or program at twice the rate and a diminished sense of belonging was an important factor in not retaining them.
We wondered how we could make that survey even better, and then more importantly, how we could extend that to everyone by using the 2014 surveys as the basis for assessing faculty and staff as well. So we spent the next three years working on that, identifying the best questions from the surveys that were administered in 2014, adapting them to make them appropriate for the different contexts and experiences of staff and faculty roles, and piloting those questions in collaboration with different units on campus.
We drew from our years of work with students and employees, and from feedback provided by the students, staff, and faculty who worked closely with us to develop the survey, especially the LASP Climate Committee. Especially important to us was making sure that every question would be actionable. We also drew inspiration from surveys administered at other universities and research institutions, finally landing on a set of questions that we first piloted at LASP (Laboratory for Atmospheric and Space Physics). LASP was willing to be our inaugural site and with that administration, we were able to provide LASP with really usable data. With that LASP data, we used factor analyses and statistical modeling to pare the questions down by about half. We were then able to pilot the revised version at different departments on campus. In fall 2019, the survey was adopted as part of the Academic Review and Planning (ARPAC) process to inform the self study.
Beginning the week of October 18, all students, staff, and faculty will receive an invitation to participate in the Campus Culture Survey (CCS). We had planned to administer the survey last year, but we decided to put it on pause during the pandemic. In addition to supporting the ARPAC process, the CCS will serve as an important source for benchmark data for the IDEA plan and will be one way to measure our progress going forward as we reassess every 4 years."
Stephanie Foster
"So in the pilot process you piloted this new survey with several different departments. What were a couple of the most important things that you learned from that process?"
Julie Volckens
"Well, one thing that we didn't expect when we launched was how important people's experiences of hostile or negative treatment were to understanding how students, staff, and faculty are experiencing the campus culture. When we were determining how to measure hostile behavior we looked to the large body of research on workplace incivility. We know that “incivility” is not an unproblematic term and we don’t use it in the survey--what we’re trying to understand are negative behaviors that undermine the work culture. These behaviors might also be called bullying or harassing. When people attribute this type of behavior to an aspect of their identity, that could meet the definition for a microaggression, and could possibly represent a violation of the discrimination and harassment policy. So we were really surprised at how many people reported experiencing this type of hostile treatment and then how much those negative experiences were shaping people's overall experience of the workplace culture.
Having examples of the specific problematic behaviors that are occuring in a workplace provides a roadmap for what departments need to address to improve their culture. For instance, one of the most common examples of workplace hostility is being constantly interrupted or talked over. That's very actionable behavior. Units can make agreements about how to reduce and eliminate this problem, and how they will respond and repair when these types of situations happen going forward.
The other thing that we found very surprising was how many people perceive that department leaders, whether it's faculty, chairs, supervisors, or directors, need more skills to address problematic behaviors that undermine the work environment. The survey has uncovered an area of great need that can be comprehensively addressed; people can get more skills to manage problematic behaviors, and also people can get better at anticipating these kinds of problems and putting measures in place to reduce the likelihood of these harms happening in the first place."
Stephanie Foster
"So in this process -- you've been working with for quite some time now -- what's your favorite part?"
Julie Volckens
"I guess my favorite part, and I think anyone who does assessment would say this, is the data analysis, where you get to understand the dynamics of a unit; what are the strengths that make them shine, and where they are struggling? Honestly, no two departments are alike. Strengths vary, concerns vary. So I find it super satisfying. I work with this incredible team and together, we put together a report that helps units understand where they are, where they could go, and options for getting there."
Stephanie Foster
"So what will be the next steps as this survey rolls out this fall?"
Julie Volckens
"Since the late summer, we’ve been communicating to the campus that the survey is coming, including what the survey will be able to tell us about our strengths (at the campus, division, and department levels) as well as where we have work to do. Since 2019, we've been extraordinarily successful at getting a high response rate, more than 50% of units responding to the survey. That's been really exciting. The way we've been able to do this is through our outreach efforts--really helping people understand what the survey measures and how it can be used to support change and accountability. There's a lot of enthusiasm on campus for improving the campus culture, and for understanding what we’re doing well and where we need to grow. This includes understanding how different groups may be having less positive experiences. We have been working closely with our key campus stakeholders and meeting with groups across campus to share information about the survey, including past findings and recent efforts that have been undertaken in response to the findings that have led to meaningful change. The launch of the official campaign and announcement from the Chancellor, Provost, and Chief Operating Officer will start in early October. There will also be a coordinated print, bus ad, and social media campaign. The survey will launch the week of October 18 and members of the CU community will receive an invitation in their CU email to participate. Students will receive a $7 incentive for participating. We have more information about the survey and a list of FAQs on our webpage, including how we are protecting the privacy and anonymity of our participants."
Stephanie Foster
"What would be a really good result from all of this work?"
Julie Volckens
"A really good result would be having a robust rate of participation that would yield an irresistible call to action and provide a roadmap for change and accountability. We need to hear every voice. We already know that the survey findings can be used to implement practices that can improve workplace culture. We worked with one unit whose results showed that they had issues with supervisors and department leaders needing more skills for addressing problematic behaviors that undermine the workplace; this included about 40% of the department indicating they had experienced hostile workplace behaviors. They invested in a strategic plan that included capacitating supervisors and department leaders with more skills for supervision, in conducting more inclusive hiring searches, and in building skills for having difficult conversations and being effective bystanders. After 18 months, they had cut their rate of incivility by half, and saw statistically significant improvements in sense of belonging, perception of supervisor skills, and feeling respected by colleagues. So units can make big changes as long as they understand where the the points of entry are."
References:
2021 Campus Culture Survey -- Information, Timeline, Security & Privacy
Microaggressions - OIEC
Workplace Incivility
Workplace incivility is defined as, “low-intensity behavior with ambiguous intent to harm the target, in violation of workplace norms for mutual respect; uncivil behaviors are characteristically rude and discourteous, displaying a lack of regard for others” (Andersson and Pearson, 1999, p. 457).
These behaviors frequently lead to an unfriendly and unproductive work/academic environment that has consequences for employees’ loyalty, retention, productivity, health, and work quality. When targets attribute incivility directly or ambiguously to identity, these acts characterize microaggressions, and possibly represent conduct prohibited by the discrimination and harassment policy. A review of the research on the consequences of workplace incivility is available in a 2019 OIEC literature review.