About the Economic Mobility Catalog

In our work with cities and counties across the country, Results for America found that many local leaders wanted a wide-ranging, easy-to-use resource that helped them identify specific evidence-based approaches to improve economic mobility in their communities. 

While there are many resources currently available, few of them are designed with local policymakers in mind, and few provide content focused on implementation. The Economic Mobility Catalog aims to fill this gap, with material written specifically for local leaders and a significant focus on implementation.

The Catalog allows users to search for evidence-based solutions by policy area and by outcome.

Within each policy area and outcome page, users will encounter a curated selection of relevant evidence-based strategies. 

Within each strategy page, users will find detailed content on the main components of that strategy, synthesis of the evidence that supports the strategy's effectiveness, implementation best practices, links to relevant case studies, and a curated selection of examples of evidence-based programs representative of the strategy. 

On program pages, users will find key details on the specific evidence-based model, synthesis of the evidence that supports the program's effectiveness, and implementation best practices. 

The Catalog's case studies are linked to specific evidence-based strategies and programs. These resources examine how a particular community was able to successfully implement the approach.

The Catalog ranks the strength of evidence in support of broader strategies and specific programs in different ways.

For strategies, the evidence ranking assigned reflects the number of studies conducted on programs and practices within that category, weighted by the rigor of the research designs used and the magnitude of the effects demonstrated. This ranking system is drawn directly from the methodology used in the County Health Rankings and Roadmaps "What Works for Health" clearinghouse.

For a strategy to be designated as "proven," positive results must be demonstrated across:

• 1 or more systematic review(s); or at least 3 experimental studies; or 3 quasi-experimental studies with matched concurrent comparisons
• These studies must have strong research designs and statistically significant, positive findings

For a strategy to be designated as "strong," positive results must be demonstrated across:

• 1 or more systematic review(s); or at least 2 experimental studies; or 2 quasi-experimental studies with matched concurrent comparisons; or 3 studies with unmatched comparisons or pre-post measures
• These studies must have statistically significant, positive findings but have less rigorous research designs and more limited effects

For a strategy to be designated as "promising":

• There is no more than 1 experimental study or quasi-experimental study with matched comparisons; or 2 or fewer studies with unmatched comparisons or pre-post measures
• These strategies are often recommended by experts but their effectiveness has not been fully demonstrated in rigorous evaluations

The Catalog ranks the strength of evidence in support of broader strategies and specific programs in different ways.

For specific programs, the assigned evidence ranking reflects how these approaches are scored in one of the major government- or philanthropy-led clearinghouse resources. Nearly all of the programs in the Catalog are drawn from these clearinghouses. 

The specific clearinghouses the Catalog draws from are as follows: 

• The U.S. Department of Education, Institute for Education Sciences’ What Works Clearinghouse
• The U.S. Department of Labor’s Clearinghouse for Labor and Evaluation Research (CLEAR)
• The Department of Justice’s Crimesolutions.gov
• The Pew-MacArthur Results First Initiative
• The County Health Rankings and Roadmaps’ What Works for Health Clearinghouse
• Blueprints for Healthy Youth Development
• Arnold Ventures’ Social Programs That Work

While rankings criteria vary slightly between the clearinghouses, in general:

• The highest rankings are assigned to programs which have demonstrated positive results in multiple independent experimental or quasi-experimental studies with strong research designs.
• The second highest rankings are assigned to programs which have demonstrated positive results in a single experimental or quasi-experimental study, or in multiple studies with less rigorous research designs.
• The third-highest rankings are assigned to programs which have demonstrated positive preliminary results in an ongoing experimental study with a strong research design.

The Catalog uses these resources to rank the strength of evidence for specific programs as follows:

• When one or more clearinghouse(s) have reviewed the same program and assigned it the same ranking, the Catalog defers to that ranking.
• When two clearinghouses have reviewed a program and have assigned different rankings, the Catalog defers to the lower ranking.
• When more than two clearinghouses have reviewed a program and have assigned different rankings, the Catalog averages the "score" and rounds it to the nearest ranking.

A small number of programs in the Catalog are not yet in the major clearinghouses. The studies demonstrating the effectiveness of these approaches have been conducted by Results for America's evaluation partners.

For the "best practices in implementation" sections throughout the Catalog, Results for America synthesized existing publicly available research and spoke to experts, program staff, and/or researchers involved in the evaluation of that strategy. 

No. While the strategies in the Catalog have been proven effective in the locations in which they were evaluated, the quality of an evidence-based strategy’s implementation effort will always determine its effectiveness. 

The "best practices in implementation" sections throughout the resource and the implementation-focused case studies are the Catalog's attempt to guide local leaders as they invest in these evidence-based strategies in their communities.

Results for America consulted with the U.S. Department of Treasury's Interim Final Rule on Coronavirus State and Local Fiscal Recovery Funds to identify which Catalog strategies would be eligible for Fiscal Recovery Funds, then drew on the expertise of Holland and Knight LLP to provide additional analysis and specificity on strategy eligibility.

No. The Catalog is a dynamic product and will continue to grow in breadth and depth, with new strategy and program content, new implementation-focused case studies, and a range of additional features to be launched through 2022-2023.

The Economic Mobility Catalog is intended to be a dynamic research product. We welcome your corrections, suggestions, and feedback. Please reach out to [email protected]