URSSA is the Undergraduate Research Student Self-Assessment, an online survey instrument for programs and departments to use in assessing the student outcomes of undergraduate research (UR). URSSA focuses on what students learn from their UR experience, rather than whether they liked it.
The self-assessment includes both multiple-choice and open-ended items that focus on students’ gains from undergraduate research. These gains include:
Other items probe students’ participation in important research-related activities that have been shown to lead to these gains (e.g. giving presentations, having responsibility for a project). These activities, and the gains themselves, are based in research and thus constitute a core set of items. Using these items as a group helps to align a particular program assessment with research-demonstrated outcomes.
In addition, optional items can be included to probe particular features that are included along with research in a UR program (e.g. field trips, career seminars, housing arrangements).
URSSA is a tool for measuring students’ self-reported gains from their research experience. Our research shows that students are very capable of noticing their own growth—and also where they have grown little or not at all. Student self-report is not the only measure of the success of a UR experience, but it is an important component. URSSA measures some outcomes, such as growth in confidence, or the decision to become a scientist, that only students can tell us about. We encourage faculty and departments to use URSSA as one part of a more comprehensive evaluation plan that addresses all their program goals and outcomes.
The survey questions (“items”) in URSSA are based on our group’s extensive, interview-based research and evaluation work on undergraduate research. This work includes:
This grounding in research means that URSSA measures things we know to be important—it “asks the right questions.”
Once initial survey items were developed based on this body of research, they were tested with students in “think-aloud” interviews to see if students interpreted the wording as we intended; the items were then refined and tested again.
With this refined version of URSSA, we solicited help from faculty and UR program directors to gather a large student data set. This pilot study included over 500 students in 24 colleges and universities, which enabled us to conduct statistical tests of the items’ validity and reliability. Using Confirmatory Factor Analysis, we compared how student responses fit the hypothesized structure of the survey and found that the data met accepted standards for model fit. We also tested survey items to learn if they functioned as we intended. Based on these results, some survey items were removed from the survey or changed when they did not meet our criteria for acceptable item functioning.
Optional items about UR program elements were developed in consultation with UR program developers and department UR leaders.
After URSSA had been in the field for some time, data from over 3400 students was used to carry out a statistical validation study. See
Weston, T. J., & Laursen, S. L. (2015). The Undergraduate Research Student Self-Assessment (URSSA): Validation for use in program evaluation. CBE-Life Sciences Education, 14(3), ar33. DOI 10.1187/cbe.14-11-0206 http://www.lifescied.org/content/14/3/ar33.full
The URSSA team includes people with a broad range of applicable expertise. Anne-Barrie Hunter, Sandra Laursen, and Heather Thiry are education researchers who specialize in qualitative research, especially interviews. With other colleagues, they have studied several undergraduate research programs as researchers and program evaluators since 1999. Laursen is also a chemist who has conducted laboratory research both as an undergraduate and as a research advisor to undergraduates. Timothy Weston is an expert in student assessment, survey development, and quantitative research in education.
The value of undergraduate research (UR) as part of a student’s science education has indeed been long known to the faculty who work with UR students—but only recently have the outcomes of UR been documented in well-designed research and evaluation studies. Research also suggests that there are many different paths by which to develop effective UR programs, and that good assessment data can help programs refine what they offer to optimize the UR experience for students.
Science departments and UR programs at universities and labs need well-designed, inexpensive evaluation tools so that they can assess student outcomes, improve their programs, and inform their stakeholders. Funding agencies also need these tools to measure the impact of their efforts, for example in examining innovations.
URSSA is
For groups of 10 or more students, URSSA is delivered through the web platform developed for the Student Assessment of their Learning Gains (SALG), an online instrument for assessing students’ learning gains from college science classes. Using salgsite's online tools, UR programs or departments can set up a customized copy of URSSA and generate a link to send to students to complete the survey.
At present, the SALG site offers a transitional version of URSSA, so that URSSA is available for summer 2009 and later. We are working with the SALG developers to develop a more robust and stand-alone version of URSSA for the future.
To examine URSSA or to create a version of URSSA specific to your department or program, download these directions and follow them to open an account on salgsite.org and prepare URSSA for your own use.
To protect the confidentiality of students’ responses and the trust between students and their research advisor, URSSA should not be used with groups fewer than 10 students. URSSA is designed for evaluation use by departments and programs, not by individual research advisors.
We encourage research advisors to gather feedback from their own students about their research experience. We suggest that you meet with your research students as a group for informal conversation over lunch or a snack. Ask your students what they have gained from doing research—wait to see what they discuss spontaneously, then perhaps explore the broad gains areas from URSSA that interest you most. Other good questions to ask students are what was the “best” thing about their research experience, what could be improved, and what surprised them about doing research.
If you report findings from URSSA in a presentation or publication, please cite the instrument:
URSSA, Undergraduate Research Student Self-Assessment (2009). Ethnography & Evaluation Research, University of Colorado at Boulder, Boulder, CO. www.salgsite.org
If you modify URSSA items for your own purposes, please cite the instrument but explain the changes you have made, as these may alter the validity of the instrument or meaning of the items.
Contact Us
If you have questions or comments about URSSA and its use, please contact Anne-Barrie Hunter at abhunterATcolorado.edu. If you have questions or comments about the SALG site, please send an e-mail to help@salgsite.org.
Further Reading
This article describes the validation of the URSSA instrument:
Our book and the articles listed here describe the research that underpins the URSSA instrument. Several also include extensive reference to others’ published work on undergraduate research.
Acknowledgements
The URSSA team comes from the University of Colorado at Boulder and includes Anne-Barrie Hunter, Tim Weston, Heather Thiry and Sandra Laursen. We thank the SALG team and all the students, faculty, and program directors who helped us test and refine URSSA.
Development and testing of URSSA has been supported by the National Science Foundation through its Divisions of Chemistry and Undergraduate Education, the Biological Sciences Directorate, and the Office of Multidisciplinary Affairs, under grant #CHE-0548488.
Additional support has been provided by the Biological Sciences Initiative and the NIH Scholars program, both at CU Boulder, through their grants from the Howard Hughes Medical Institute and the National Institutes of Health.