Inconsistent analysis in recent CEA report highlights need to invest in – and collect data on success of – workforce and education programs vital to workers and businesses

Earlier this week, the White House Council on Economic Advisers (CEA) released a report on the effectiveness of job training programs, calling for further investments in a new Industry Recognized Apprenticeship system at the expense of investments in a broad spectrum of other workforce programs. The report offers inconsistent analysis of workforce programs, dismisses shortcomings of studies on which it relies and conflates goals and outcomes of a diverse set of workforce programs when drawing conclusions about effectiveness.

Despite these shortcomings, it does make two important points.

• Workforce programs in the U.S. are drastically underfunded when compared to investments made by other industrialized countries. A yearly $80 billion investment is needed to reach even the median level of other international peers; and
• Despite the overwhelming availability of data on skills training programs and their effectiveness, a national strategy is needed to make this data digestible for policy makers and understandable for participants and businesses.

The report was released a week after the administration’s Advisory Board on Workforce Policy held its most recent meeting. That advisory board – comprised of business and labor leaders, educators, and governors – is tasked with identifying policy recommendations to help develop a skilled workforce to fill open jobs. The CEA report is unlikely to be the early stages of a new workforce platform for the administration, as it reiterates previous White House recommendations instead of offering new proposals. It may, however, influence the conclusions of the advisory board in their recommendations that could inform future agency priorities.

Understanding the shortcomings in the report

The report’s incorrect conclusions about the impact of job training are simply inaccurate. There are millions of US workers who have earned credentials and degrees through community colleges, community-based organizations, labor-management partnerships, and other providers who have leveraged these credentials into family-supporting jobs.

Several mistakes in the report lead to this incorrect conclusion.

• Inconsistent Analysis: The CEA Report is inconsistent in its analysis of literature on the effectiveness of job training programs. It praises the strong positive outcomes that non-random evaluations have found for registered apprenticeship programs but dismisses the same type of methodology when it comes to positive results for other training programs.

For other training programs, the CEA Report finds only randomized controlled trial evaluations to be reliable, citing a 33-year-old study of evaluation methodology, while acknowledging in a footnote that statistical techniques have improved substantially since that time.

Non-random net impact evaluations of workforce training programs using statistical techniques to control for differences between participants and a similar group have frequently found substantial positive impacts on employment and earnings.

• Ignoring inherent weakness in most randomized controlled trial evaluations: Moreover, as the CEA Report notes but largely ignores, there is a weakness common in most randomized controlled trial evaluations. Individuals not randomly selected to take part may well receive the same type of service funded through another program.

The “gold standard” evaluation of the Workforce Investment Act, the predecessor to WIOA, conducted by Mathematica, did not preclude individuals who were not offered WIA-funded training from receiving training financed by a different source. In fact, 22 percent of the control group completed a vocational credential funded through other means, while just a slightly higher 27 percent of the study group completed a vocational credential. As a result, in contrast to the claim of the CEA Report, the study did not reach a conclusion one way or the other about the effectiveness of training. “Because differences across groups in rates of enrollment in training were small, our study produced inconclusive evidence on the impact of training.” (p. xxxiii)

• Conflation of diverse workforce services: Employment and training programs often combine a range of services from career counseling, to job search assistance, to basic education, to occupational training. Many critiques of training have not differentiated the impact of these services or their intended outcomes and instead referred to them as workforce training. In its critique of workforce training the CEA Report cites MDRC’s Hamilton and Scrivener (2012) report on welfare-to-work programs as evidence. But of the 12 local programs covered in the Hamilton and Scrivener report, none provided additional skills training to the participants. They instead provided other types of workforce services.

Workforce education and training is effective at meeting worker and business need

Skills training is overwhelming popular with both likely 2020 voters and leaders of small and mid-size companies. It’s also an effective strategy to help workers increase wages and find employment. Numerous studies have found training to positively impact employment and earnings. Just two examples –

• Studies of sectoral training, including one conducted by MDRC, found that programs that brought together employers and education providers in a local area to develop training and connect workers with open jobs lead to significant skills gains and strong employment outcomes for workers.
• There is also strong data on the impact of training provided at community and technical colleges, frequently the provider of training funded by the workforce training programs referred to in the report.

Congress also took important steps to address necessary modernization in workforce programs in the overwhelmingly bipartisan passage of WIOA in 2014. The reauthorization strengthened alignment across education, training, and human services programs and provided modest increases in funding for core programs to help rebuild our nation’s skills investments.

Data on WIOA continues the trend found under its predecessor – that individuals in training have higher employment rates and median earnings than individuals receiving fewer services. And continued technical assistance from the Department of Labor and Education, along with meaningful investments in a program starved for resources, will empower states and local areas across the country to continue to make the important progress enabled by bipartisan passage of WIOA.

The CEA Report is correct when it states, “[t]he large number of [training] programs and the heterogeneity in the types of programs make it difficult to establish a single general conclusion, but rather suggests that some programs are effective whereas others are failing to live up to their hoped-for potential.” These programs are not monolithic; they serve different populations, with different services, to achieve different purposes, and their degree of success can vary from one local provider to another. It is impossible to draw a single conclusion about their effectiveness.

NSC will continue to work with the administration and Congress to make necessary investments in those effective workforce education and training programs to empower local areas to serve workers, meet business demand, and adequately evaluate the impact of federal, state, and local investment in skills programs.