Evaluation of the Youthbuild Program

Maxine V. Mitchell

Davis Jenkins

Dao Nguyen

Alona Lerman

Marian DeBerry

Applied Real Estate Analysis, Inc.

Citation: Mitchell, Maxine V., et al. “Evaluation of the Youthbuild Program Final Report.” Washington, D.C.: U.S. Department of Housing and Urban Development. Republished in Children, Youth and Environments 13(2), 2003. Retrieved [date] from http://colorado.edu/journals/cye.

Executive Summary

Youthbuild, a youth and community development program, was authorized in 1992 under the Housing and Community Development Act (HCDA) of 1992. It was based on the example of a privately funded project originating in New York in which youth who had dropped out of school participated in a program of personal and community development that taught leadership and public-speaking skills, helped the youth to earn their General Educational Development (GED) certificate, and provided both formal and informal counseling. While in this program, participants also developed job skills through projects to renovate abandoned buildings in their neighborhoods. A demonstration program in the early 1990s encouraged the implementation of a national program. Since then, a national network of Youthbuild programs has evolved, incorporating the same elements as the New York model. The U. S. Department of Housing and Urban Development (HUD) administers this program and provides funding, through an annual competitive grant application process, to local organizations who implement the Youthbuild program.

Federal regulations establish basic guidelines for the HUD Youthbuild program. Briefly, these guidelines specify that the program serves very-low-income youth, within the 16-to-24 age limits, who demonstrate an educational need either by virtue of being high school dropouts or by justifying their need for inclusion in the program. Programs must provide these youth with academic training and on-site construction training, in equal proportion, and offer leadership training. Federal regulations also require that grantees assist with job placement, offer counseling and other social services to address the many problems participants encounter, and follow up for up to 12 months after participants leave the program to ascertain their living conditions and employment status, and to offer further counseling and job placement assistance. In addition to these youth training goals, HUD Youthbuild programs must expand the supply of permanent, affordable housing for low-and very-low-income families.

Applied Real Estate Analysis (AREA), Inc. was retained by HUD to evaluate the federally funded Youthbuild program. The purpose of this study is to describe the program's funding, activities, and accomplishments; assess its cost-effectiveness; and compare its accomplishments with those of other work-force development programs. The focus of this analysis is on HUD Youthbuild programs awarded grants between 1996 and 1999. The research included: (1) a detailed examination of reports filed at HUD Headquarters by Youthbuild grantees from 1996 to 1999 to determine the overall characteristics of Youthbuild grantees, program components, and youth participants; (2) site visits with 20 local Youthbuild programs to obtain more detailed information regarding variations in program design, accomplishments, and cost of operation; and (3) a comparison with other youth employment training programs using data from a literature review of past evaluations of comparable programs.

Between 1996 and 2002, over 18,000 at-risk youth have participated in Youthbuild, receiving academic instruction and construction training and other pre-employment services. The vast majority of these participants were high school drop-outs and came from very-low-income households; many also abused drugs and/ or alcohol and had run-ins with the law. Youthbuild provided these youth with an opportunity to improve their academic achievement, to gain hands-on experience in a living-wage industry, and to obtain employment.

Key Findings

This evaluation of the HUD Youthbuild program compares the program with four other federally funded programs designed to serve disadvantaged youth: Job Corps, JOBSTART, and the youth components of Supported Work Demonstration (SWD), and Job Training Partnership Act (JTPA). Each of these programs serves roughly the same age group from a minimum age of 16 to 17 to a maximum of 21 to 24. They assist economically disadvantaged individuals, offering them various mixes of academic and occupational skills training and providing training-related support services and job placement assistance.

Only one of the comparison programs, Job Corps, is still operating. Job Corps is the largest and most expensive of the federally funded job training programs for youth, in part because it requires participants to reside on site. JOBSTART was modeled after Job Corps, but did not offer a residential setting. Both the Supported Work Demonstration and the Job Training Partnership Act assisted economically disadvantaged individuals in all age groups, including out-of-school youth.

All four of these comparison programs have been rigorously evaluated through (1) randomly assigned experiment and control groups, (2) extensive data on participant demographics and program performance, and (3) tracking of outcomes over time (in the case of Job Corps for four years following program intake) using administrative data (wage record data collected by states) to track employment outcomes. Obviously, such evaluations are complex, long-term, and quite costly.

The principal conclusion from these evaluations is that helping disadvantaged youth advance economically and educationally is very difficult and expensive. Of the four rigorously evaluated comparison programs, only Job Corps a very intensive and expensive residential program was found to have significant impacts on participants' educational achievement and earnings, although it did not show an impact on employment rates.

Studies of the four comparison programs of federally funded youth employment programs compare participants in the programs, referred to as the experimental groups, with control groups made up of non-participants youth who are excluded from participation in the programs for set periods of time.

Any comparison of these four programs with the HUD Youthbuild program in terms of outcomes must be rough at best. Youthbuild has not been subjected to a rigorous evaluation similar to those of the comparison studies. The evaluation conducted for the HUD Youthbuild program is largely qualitative, relying on data supplied by program providers, which varied in quality and completeness and tracked immediate outcomes (GED completion and job placement) for program participants and completers. (The other program evaluations tracked outcomes up to three years after program exit and had access to administrative data.) Also, our sample was not a statistical sample, and it is possible that our 20 sites may not properly represent the universe of Youthbuild programs though we have no reason to believe that they are not representative.

While we cannot definitively determine the impact of Youthbuild or its effectiveness, we can bring evidence to bear on the program s outcomes and costs and how they roughly compare with other youth employment training programs.

Given these caveats, the information gathered through our research allows us to make the following comparisons across programs:

Relatively High Costs. Youthbuild is expensive relative to other federally funded programs for youth. The average cost per Youthbuild participant, excluding construction costs, is estimated at nearly $15,000. This is comparable to the cost of SWD, which is close to $13,700 per participant in present-value dollars. Costs for Youthbuild and SWD programs are probably high because they both offer subsidized work for participants. In present-value dollars, JOBSTART and JTPA have significantly lower costs per participant about $6,000 and $3,900 (maximum) respectively. Job Corps costs ($18,480 per participant) are significantly higher than those of all the other programs, primarily because the program provides participant housing and an array of intensive social support services. However, if the costs of construction are included, Youthbuild may even exceed Job Corps in cost per participant.

Similar Academic Achievements. Approximately 29 percent of all participants who entered Youthbuild without a high school credential succeeded in obtaining that credential. By comparison, high school credentials were obtained by 35 percent and 29 percent of the Job Corps and JOBSTART control groups, respectively. Youthbuild's figure is lower than the percentage of Job Corps (47 percent) and JOBSTART (42 percent) participants who attained the credential.

Limited Employment Achievements. The Youthbuild findings show that as they left the program, 36 percent of participants in the 20 Youthbuild sites were employed. Because of the nature of the research design, the worst-performing grantees were not included among the potential 20 site visit locations. This is explained in more detail under Site Selection Criteria in Chapter 1, and in Appendix A, which describes the research design.

Although the other employment training programs do not have a housing construction component, it is a key part of the Youthbuild program; therefore, it is appropriate to include the cost of housing construction as part of the total program costs. However, while the new and rehabilitated affordable housing does provide additional benefit and value, those benefits were not considered as part of this evaluation.

This rate is actually lower than the rate for the control groups in the comparison studies. The most comparable data we have are on participants in the SWD, where a short-term (typically three months) follow-up of the treatment group after they had left the program found 42-51 percent to be employed. Job Corps and JOBSTART findings do not show any significant impact on the employment levels of participants. However, Job Corps participants did have higher earnings than control group members.

Further Pursuit of Academic Training. The HUD Youthbuild program data show that 12 percent of participants pursued higher education upon program exit. Data are not available for how many attained a two-or four-year degree. Comparable data are not available for the other programs, but we do know that only 2 percent of Job Corps participants completed a two-or four-year degree during the extended follow-up period.

The HUD Youthbuild program also includes key elements leadership skills training and an increased supply of affordable housing for low-and very-low-income families that are not part of the comparative youth employment training programs. Analysis of the leadership component of the program is difficult, since Youthbuild grantees do not maintain or report quantitative information on leadership training activities undertaken by participants. However, our examination of the 20 local programs indicated that grantees offer many opportunities for such training. Several programs have established advisory boards, which are participant-run councils that address participant issues and serve as vehicles for developing and fostering leadership. Youthbuild's success in achieving its housing-related objectives are somewhat easier to assess. Following are key findings from our analyses of the program's housing component:

Limited Contribution to Affordable Housing Production. With a few exceptions, local HUD Youthbuild programs produce only a small number of affordable housing units. The vast majority of grantees proposed producing between one and five housing units per grant period, and less than that each year. During the 18-to 30-month grant period, site-visit programs worked on an average of 12.7 housing units and a median of eight units. The reason for this relatively high average is that in a few of the programs the participants contribute a small portion of the labor required to construct or rehabilitate a relatively large number of dwellings. Most of the programs that produce higher numbers of housing units involve the rehabilitation of a multi-unit building on which participants perform some work on numerous units.

There is no doubt that housing construction and/ or rehabilitation activity provides good opportunities for youth to learn important job skills, such as promptness and thoroughness, as well as specific trade skills. But in most instances, the training process is slow, and a relatively small number of units can be completed during each grant period.

Provision of Housing for Low-and Very-Low-Income Households. Federal regulations for the HUD Youthbuild program mandate that 90 percent of the units produced must be occupied by households earning no more than 60 percent of the area median household income a requirement that the program clearly meets. Given the federally funded programs used to finance Youthbuild housing (such as HOPE VI and Low-Income Housing Tax Credits) and the income restrictions these programs have for some units, it is assumed that the households who receive units built by Youthbuild meet the required definition of low-and very-low-income. Although documentation of residents income levels was limited at least among the 20 site-visit programs that we examined Youthbuild program staff members also provided anecdotal evidence that mostly low-to very-low-income households rent or purchase housing produced using Youthbuild participants' labor.

Major Conclusions

Based on a review of the outcomes and costs of the four federally-funded employment training programs with which the HUD Youthbuild program was compared, it is a very difficult task to exert positive impacts on the employment, earnings, and educational achievement of disadvantaged youth. Only Job Corps, a high-cost program that was subjected to a rigorous, experimental analysis, could demonstrate that the benefits of intensive intervention exceed the costs of mounting the program.

For a nonresidential program, Youthbuild is relatively expensive (in present-value dollar terms) when compared with the other four programs. And based on a very rough comparison, the GED results and employment rates of Youthbuild participants seem to be, on average, closer to those of the control group (those not served) in the Job Corps study than those who went through the Job Corps program. Unlike the comparison programs, Youthbuild has not been rigorously evaluated, so we cannot make definitive claims about its impact and effectiveness. Even if HUD were to invest in a more rigorous assessment of Youthbuild's cost and effectiveness, such an evaluation would be complicated by the fact that, relative to the comparison programs, there is far less agreement among Youthbuild providers and HUD about what the specific objectives of the program are and how to measure and document the program impacts. The HUD Youthbuild program achieves its goal of producing affordable housing that benefits the low-and very-low-income households for which it is intended. However, the number of units produced is small usually one to two units each year per program grantee. While some Youthbuild programs report a larger number of completed housing units, their participants have usually contributed a small portion of the labor required to produce the dwellings.