Benjamin Shear

This report examines whether a statistical method called Empirical Best Linear Prediction (EBLP) can reduce the year-to-year volatility of Colorado school test score metrics used for school accountability. Using three years of Colorado state test score data, the results demonstrate that EBLP produces more accurate and more stable results than standard single-year measures, with the largest improvements for the smallest schools. However, the use of EBLP measures in place of single-year measures did not increase stability of School Performance Framework Growth Ratings. The report discusses potential explanations for these findings and considerations relevant to the use of statistical stabilization methods such as EBLP when using school test score metrics for accountability.

To address disruptions caused by the COVID-19 pandemic, the Colorado Department of Education (CDE) launched the Expanded Learning Opportunities (ELO) Grant Program using Elementary and Secondary School Emergency Relief (ESSER III) state reserve funds. Between 2022 to 2024, CDE awarded 66 ELO grants to support out-of-school programs aimed at accelerating student learning and increasing student engagement. This report uses quantitative methods to describe ELO program focus areas, participating student populations, and subsequent student achievement and attendance outcomes.

This report examines how changing Colorado’s minimum student count thresholds ("minimum n-size") could affect the availability and reliability of aggregate school and district achievement and growth measures. Using statewide test score data from 2017-2019 and a hierarchical linear modeling framework, the report documents that lowering the minimum student count thresholds modestly increases reporting for overall school results but can substantially increase reporting for disaggregated student groups, often with lower reliability. The analyses demonstrate how reliability of these aggregate test score measures increases with sample size, is higher for achievement than growth, and for schools compared to districts. The findings underscore tradeoffs between transparency and statistical stability, providing guidance for policymakers setting minimum n-size requirements.

In November 2024 the Colorado 1241 Task Force released their final report detailing recommendations and areas for further study to guide improvement of Colorado’s Education Accountability System. The Task Force Report describes 30 recommendations across five domains and four areas for further study. Implementation of any Task Force recommendations should be directly tied to a clearly stated theory of action articulating the goals for Colorado’s school accountability system and explaining how implementation of the proposed recommendations would make the accountability system more effective at achieving these goals. In this commentary we highlight points for consideration relating to nine of the 30 task force recommendations to inform decisions about implementing recommendations from the Task Force Report.

This study analyzes the redesignation patterns of a cohort of Multilingual Learner (ML) students in Colorado from 3rd to 8th grade (from 2018 to 2023), focusing on how redesignation varies across demographic groups. Redesignation as fluent English proficient (FEP) is a two-step process in Colorado: students must first earn a minimum score on the ACCESS exam and then produce a standardized body of evidence (BOE) demonstrating readiness to transition to mainstream English classrooms, with BOE criteria defined locally by districts and schools.

In this report, we review a widely used method for calculating effective weights and apply it to evaluate the alignment of nominal and effective weights for composite accountability scores produced in the Colorado School Performance Framework (SPF).

This paper investigates differences between male and female student performance on multiple-choice (MC) versus constructed-response (CR) items used on the large-scale math and reading PISA tests. The results, based on data for high school-aged students in 35 countries including the US, provide consistent evidence that on average male students tend to earn relatively higher scores on MC test items whereas female students tend to earn relatively higher scores on CR test items. The pattern was consistent across countries, although the magnitude of differences varied by country and were larger on average in reading than in math. Policymakers, researchers, and other audiences using test scores to compare student achievement across gender groups should consider the types of item formats used on tests when interpreting results.

This paper describes threats to making valid causal inferences about pandemic impacts on student learning based on cross-year comparisons of average test scores. The paper uses Spring 2021 test score data in Colorado to compare three different statistical adjustments that have been used to make inferences about pandemic impacts. Results illustrates how unadjusted cross-year comparisons may be misleading and highlight instances when the statistical adjustments can lead to different conclusions about the relative impacts for different groups of students.

IntroductionThe COVID-19 pandemic led to historic changes in public school enrollment in the US. There has been considerable interest in quantifying the magnitude of these changes, and in understanding their causes and consequences (e.g., Dee &

This is an addendum to a report prepared for the Colorado Department of Education (CDE) by the Center for Assessment, Design, Research & Evaluation (CADRE) at CU Boulder (Shear et al., 2021). CDE requested an addendum to the 2021 CADRE report presenting similar descriptive analyses for schools that participated in the Connect for Success (CFS) program and providing initial descriptive analyses of academic outcomes for schools participating in two other Empowering Action for School Improvement (EASI) programs: The Multi-Tiered Systems of Supports (MTSS) and Accountability Pathways Planning and Implementation (Pathways).