High-dosage tutoring: Saga Education in Chicago Public Schools May 06, 2022

Saga Education in Chicago Public Schools: High-dosage tutoring to improve student outcomes in math

Results

2 x

In a randomized controlled trial, researchers at the University of Chicago’s Urban Labs found that students who participated in Saga Education tutoring doubled (and sometimes tripled) the amount of math students learn in a single year.

50 %

Saga Education has reduced disparities in grades and test scores between students from higher and lower income families in Chicago by as much as 50 percent.

63 %

Participants in Saga's tutoring program in Chicago were up to 63 percent less likely to fail math courses, and up to 26 percent less likely to fail other subjects.

$ 6

The University of Chicago's Urban Labs projects a return of $6 for every $1 invested in Saga or similar high-dosage tutoring programs.

The Challenge

  • Struggling schools create urgency for new investments: For years, many Chicago Public Schools experienced poor academic performance, low graduation rates, high levels of violence, and below-average employment outcomes. By 2011, there was growing recognition that new interventions were needed. In an effort to improve school performance, then-Mayor Rahm Emanuel closed some public schools and announced “turnaround” policies for others. The city also invested $18 million in strategies to improve schooling environments and reduce violence.
  • Momentum generated by new programs, additional academic supports still needed: In 2011, the city piloted a program called Becoming A Man (BAM), which aimed to reduce youth violence, among other outcomes. In an evaluation conducted by the Urban Labs at the University of Chicago, BAM’s pilot showed improvements to social-emotional outcomes and graduation rates, and was subsequently implemented across the CPS system. But the program had limited impact on academic outcomes. Local leaders began to look for an academic intervention that could complement BAM.
  • Bringing an evidence-based math tutoring program to Chicago: As local leaders sought new interventions, there was strong interest in math-focused tutoring programs, as math achievement was widely recognized as the main obstacle to many students' graduation. Researchers at the Urban Labs identified Match Education, a charter school in Boston, as one of the few tutoring approaches backed by strong evidence (Saga Education grew out of Match in 2014). Researchers visited the program in Boston and Lawrence, MA, ultimately deciding to move forward with a pilot in Chicago in 2012.

The Solution

  • High-dosage, in-school-day math tutoring as part of a student’s education: Saga Education provides daily math tutoring sessions led by trained, full-time staff to public high school students. It is built on the belief that students will only invest effort into academic material if they feel that the tutor cares about their success.
  • Thoughtfully designed tutoring sessions that encourage active learning: Tutorial periods are structured to follow Saga Education’s evidence-based model. Tutoring sessions occur during the school day, rather than before or after. Saga tutors work with a few students at a time, providing significantly more personalized instruction than a larger classroom setting. The student spends the first four minutes on a silent “do now” exercise, then the bulk of the session is spent working collaboratively with the tutor on material recently covered in math classes. The final period is a brief assessment of what the student learned, which informs the next day’s lesson.. Currently, tutors in Chicago work with 4-5 students at a time, using a blended learning model that includes the use of a virtual learning platform.
  • A successful pilot followed by expansion: Saga Education first arrived in Chicago through a partnership with the Urban Labs at the University of Chicago, which piloted the intervention at Harper High School in 2012. The pilot demonstrated strong positive results, leading the Urban Labs and Saga to pursue an expansion across Chicago Public Schools.
  • District-wide implementation: Saga, the Urban Labs, and Chicago Public Schools agreed to implement the program in an initial four year contract in 12 schools (serving 600 students) in 2013-2014, and then in 16 schools (serving 1,000 students) in 2014-2015. After demonstrating positive results in continued evaluation by the Urban Labs, Chicago Public Schools renewed and expanded its contract with Saga for an additional six years. The program currently serves 2,819 students in 31 high schools across the district.

Major Accomplishments

  • Significant and rapid improvement to math achievement: According to two rigorous, randomized controlled trials performed by researchers at the University of Chicago Urban Labs, students who participated in Saga’s tutoring program gained between one to two and a half years of math in a single year. These results were groundbreaking in the education field and disproved the notion that by high school, academic interventions would be less effective.
  • Continued evidence of effectiveness leads to expansion throughout the school district: The results of the Urban Labs’ evaluations persuaded Chicago Public Schools to expand the program from a pilot in a single school in 2012 to 12 high schools in Chicago in 2013, reaching over 2,000 students. As principals and school leaders witnessed Saga’s success during the 2013-2014 school year, three more schools adopted the program in the 2014-2015 school year. As data continued to demonstrate effectiveness, buy-in among school and district leaders continued to grow. Ultimately, the school district grew the Saga program to 31 schools in 2021-22, and added a city-wide tutor corps in other sites.
  • Success in Chicago facilitates expansion to other areas: The evidence that Saga has generated in Chicago Public Schools has enabled the organization to expand work into public schools in New York, the District of Columbia, Providence RI, Broward County Florida, and Charleston SC, significantly advancing the adoption of high-dosage tutoring in schools across the country.

Keys to Success

  • Consistent connection with caring adults: Saga’s program is built on the belief that students will only invest effort into academic material if they feel that the tutor cares about their success. The close relationship built between student and tutor through daily tutoring sessions motivates students to make their best effort, which often leads them to be more deeply engaged in the material and eventually experience success.
  • Staffing structure ensures high quality program delivery: Saga manages their own tutors with full-time site directors who monitor the tutoring sessions, oversee tutors and provide continuous feedback on instruction. Saga’s site directors liaise with the school’s math department and grade-level teams to ensure the alignment of curricula and to share information about student progress and challenges. Saga’s leaders strongly believe that these site directors are a critical piece of the model because they ensure that the program is a “turnkey” solution that can guarantee high-quality tutoring that fits well into the school and does not require extra work for existing faculty and staff.
  • An independent organization that served as an incubator, evaluator, and champion: The Urban Labs at the University of Chicago enabled Saga’s expansion throughout Chicago possible by identifying the program, providing the resources to run the first pilot, conducting the studies that demonstrated its effectiveness, and championing the program to civic leaders across the city.
  • Strong mayoral support: The administration of then-Mayor Rahm Emanuel championed the Saga program from the outset. Saga leaders believe that his support was critical because he and his administration advocated for the program with school district leaders, who then helped bring individual school leaders on board.
  • Close partnership with the district enables strategic school selection: In close collaboration with Rukiya Johnson, Chicago Public Schools’ then-Executive Director of STEM and Strategic Initiatives, Saga developed criteria for initial school selection, then the district chose schools that were willing and able to implement the program effectively. Evaluation of potential schools included examining the culture of the union at the school, the staff dynamic in the math department, and the ability of schools to provide Saga with dedicated classroom space within the building.

Biggest Challenges

  • Addressing reservations from school leaders on operations and oversight: To implement Saga, schools had to rearrange how staff and students spent their day, reallocate staff hours, adjust school schedules, and shift resources. In addition to these changes, Saga required schools to allow an external party into the building and classroom, which created some trepidation for some principals. In some cases, this created a need for Saga to persuade principals that all of the changes would be worth the effort. Some teachers initially were initially skeptical as well, but soon saw Saga as a teacher-friendly intervention that put little burden on teachers and supported their ability to teach to a higher level in their own classes.
  • Skepticism about non-certified teachers: In many early discussions, it was common for district leaders to question whether non-certified teachers should provide a full class period worth of instruction. Given the significant time investment of daily tutoring sessions, principals and district leaders wanted to be reassured that the instruction would be effective. The substantial training provided to incoming tutors and the evaluation of the pilot helped assuage these concerns. Even more school leaders were persuaded once they saw the program’s effects on academic performance in their schools. Reflecting school and district confidence in Saga, in 2013, the district decided to provide class credit to students enrolled in Saga tutoring.
  • Persuading individual school leaders to participate in Saga: According to Saga leaders, each school made decisions autonomously and did not necessarily consult with others, and the district could not direct individual principals to bring Saga into their schools. Saga staff therefore put significant time into convincing principals one at a time and helping them work through how the program could fit into the school’s unique context. In many cases, Saga then went through the same process convincing the schools’ math departments to implement the program.

Timeline

  • Match is founded in Boston

    2000

    The first Match school was founded in Boston. In 2004, under the leadership of Alan Safran, the school embeds high dosage tutoring into its school day for all of its 200 students, making it the first school that centered on the model of “high dosage” math tutoring. Safran later co-founds Saga Education in 2014 with A.J. Gutierrez, a 2006 graduate of the Match School.

  • In Chicago, a social emotional learning program shows significant impact

    2009-2010

    Responding to deep disparities in graduation rates and rates of youth violence, researchers from the Urban Labs at University of Chicago carried out two randomized controlled trials (one in 2009-2010, another in 2013-2015) on a program called “Becoming A Man.” Among participants, the program showed reductions in total arrests of between 28-35%, violence-crime arrests 45-50%, and other arrests by 37-43%. These substantial results helped convince school leaders that third party programs could make big impacts in public schools.

  • Researchers take notice of Match Boston and build evidence in Texas

    2010-2011

    Roland Fryer, an economist at Harvard University, reached out to Match for help launching a tutoring initiative in Houston (then the 7th largest US public school district) called the Apollo 20 Project. A comparative study on the program led by Fryer showed impressive results of three to four years of math growth in one year for thousands of 6th and 9th graders

  • Chicago researchers search for an academic program to complement success in behavioral learning

    2011-2012

    While BAM reduced criminal activity and improved graduation rates, it had little impact on academic performance. After uncovering these findings, the Urban Labs begins focusing its attention on finding a complementary, school-based academic program with good evidence, ultimately discovering Match

  • Urban Labs researchers visit Lawrence, MA to learn about the Match approach

    2012

    To learn more about the Match program, researchers from the Urban Labs initiate a discussion with Match in Boston and travel to Lawrence, MA to see the Match program in person. The city of Lawrence had recently gone into state receivership and contracted with Match to implement its program in two of Lawrence’s 6 high schools. Students receiving math tutoring from Match showed a 52 percentage point increase in student growth percentile (SGP), representing the greatest one-year change in SGP in high-school math ever seen in Massachusetts.

  • Match launches a pilot at a Chicago Public High School and sees extraordinary results

    2012-13

    Impressed by the results among students in Lawrence, MA, the Urban Labs and Match collaborate to launch a pilot program at Harper High School in West Englewood, Chicago, serving 50 students with 3 tutors. The Urban Labs designed the pilot to measure the results of the program with a randomized controlled trial. The first year is successful, the pilot expands into a two-year commitment at Harper, with outcomes evaluated by Urban Labs researchers. The results are substantial, with participating students gaining between 1-2.5 years of math in a single year. After the results are published, The New York Times publishes a story about the program.

  • Mayor Emanuel becomes a champion of the program

    Winter 2013

    Following the pilot, Match and Urban Labs brief the mayor, securing his support for the program and leading him to encourage school district officials to support its expansion where possible.

  • The U.S. Department of Education approves use of federal ESSA Title I funds to pay for in-school-day tutoring

    Spring/Summer 2013

    In a call with U.S. Deputy Secretary of Education Jim Shelton, the Urban Labs, CPS and Match receive approval for the district to pay for tutoring services with Title 1 funding.

  • School district leadership identifies potential schools and meets with principals

    Summer 2013

    Drawing from knowledge and experience of past in-school programs, school district officials, led by Rukiya Johnson, develop a list of schools well-suited for Match’s program and convene a group of principals to get them on board. In identifying schools, the district considers each school’s union culture, the math departments, space availability, and other factors. Representatives from Match join the meetings to help make the case.

  • Large scale partnership between Match and Chicago Public Schools begins with a two-year contract

    2013

    With the mayor and school district on board and participating schools identified, Match secures a two-year contract with Chicago Public Schools of $2.5M per year, covering about 80 percent of Saga’s operational costs to expand into 12 Chicago Public Schools and serve 600 students in the fall of 2013.

  • Match recruits tutors and site directors

    Summer 2013

    Match uses a multi-step recruiting process to ensure that tutors demonstrate their ability to create meaningful relationships with students and provide effective instruction. The majority of tutors were recent college graduates and site directors had experience tutoring, teaching, and/or managing classrooms.

  • Saga spins off from Match to form an independent organization

    Fall 2014

    Following the positive results from the pilot in Chicago, Saga Education is founded as an organization independent from Match Education, the Boston charter school network which developed the high-dosage tutoring model.

  • Saga secures funding for expansions and adds new schools

    Fall 2014

    After a successful first year of operating in Chicago Public Schools, in the fall of 2014, Saga expands into three additional Chicago Public Schools. With this addition, Saga operates in 15 schools and serves hundreds more students, approximately 1,000 in total. To support the additional expenses of the expansion, Saga raised philanthropic funding from several local foundations to supplement school district funding.

  • Saga grows in Chicago and beyond

    2015-2021

    Saga Education continues to grow around the country. By the end of 2019, the organization serves 3,000 students in three districts across the country (NY, DC, and Chicago). Saga has partnered with AmeriCorps, who will help recruit tutors and provides resources for their compensation in Chicago, New York, and Washington D.C. Saga’s award from AmeriCorps is the second largest of 75 first-time grantees nationwide. Today, Saga Education serve 7,000 directly in six cities: Chicago, New York, Charleston, Broward County, Providence, and Washington D.C. The program operates in 58 high schools nationwide.

  • The Saga model receives endorsement from U.S. Department of Education

    2021-2022

    In its announcement of the distribution of the American Rescue Plan’s Elementary and Secondary School Emergency Relief (ESSER) funds, the U.S. Department of Education (USDOE) encourages districts to use American Rescue Plan ESSER Funds for high dosage in-school tutoring. The USDOE also included key design elements of the Saga program in its COVID-19 guidelines to schools and districts.

The Process

Confronting the problem

  • A growing sense of urgency to address public school underperformance, led by the mayor: Many Chicago Public Schools in neighborhoods with high rates of poverty had seen years of poor academic performance and high rates of violence. Then-mayor Rahm Emanuel approached the University of Chicago’s Urban Labs to discuss developing a program to address youth violence.
  • Chicago deploys and evaluates a youth counseling program that emphasizes social-emotional learning: “Becoming a Man” is a social-emotional program run by the organization Youth Guidance which aims to reduce youth arrests. In an effort to help the city address high rates of violence and criminal justice involvement among disadvantaged youth, the Urban Labs conducted an evaluation of the program. The evaluation found that the program caused significant reductions in arrests for both violent and nonviolent crime and positively impacted high school graduation rates.
  • Urban Labs seeks an evidence-based tutoring program: While BAM had positive impacts on graduation rates, its primary purpose was to improve social-emotional outcomes and reduce involvement in crime. Urban Labs researchers were interested in finding a program that focused on academic outcomes which could complement BAM, eventually discovering the Match Charter School in Boston. With Match's high-dosage tutoring model showing promising results, Urban Labs researchers visited the school in Boston and in Lawrence, MA. Impressed by their visit, the Urban Labs invited Match to pilot their program in Chicago.
  • Pilot demonstrates strong outcomes, presenting opportunities for scaling: The first implementation of Match’s model at Harper High School in Chicago showed promising results for participants. This led Match and Urban Labs to explore how the program could be expanded in the city. The Urban Labs introduced Match to the Mayor’s Office and found that the Mayor was enthusiastic about the effort. The Mayor’s Office helped set up meetings with Chicago Schools’ Superintendent’s office staff and made clear that he supported the model.
  • Chicago Public Schools move forward with Match: With promising results and a champion in the Mayor, principals and district officials agreed to implement Saga’s model of school-day intensive math tutoring. District and school leaders agreed that students behind on math could forgo some arts or music classroom time if it meant significant improvements in math. School and district leaders, working closely with Saga and the Urban Labs, began evaluating which schools could effectively implement the model.

Designing the strategy

  • Strategic implementation site selection: Saga in Chicago would need to be implemented school-by-school. To achieve this, CPS first came up with a list of about 20 schools who would be good candidates for Match’s services based on a set of criteria determined by Saga. A briefing was held for interested schools. Then, Rukiya Johnson, the Executive Director of STEM & Strategic Initiatives for Chicago Public Schools, invested significant time into speaking with individual schools and working with principals to determine how Saga could fit into the school’s processes.
  • A tutoring model that centers relationships with students over the course of the school year: Saga leaders knew that many students who struggle with math in high school ultimately fail to graduate, and that many students faced barriers related to poverty and neighborhood violence. Therefore, Saga’s model relied on trained tutors who could serve as a positive and emotionally supportive mentor who could help assure students that they are capable of succeeding. Tutors had 100 hours of training during the summer, covering topics including how to manage students, wait time, learning to check for things like the “joy factor,” the ratio of time spent with students talking vs. tutors talking, providing precise praise vs. general praise, checks for understanding, and more.
  • Structured tutoring sessions aligned with class material: Because success in math requires active learning and practice, Saga centered active learning in their tutoring model that complemented in-class instruction. Tutoring follows and reinforces the material covered in recent classes and every session begins with a “do now” problem for students to solve. The remainder of the session continually requires the student to solve problems and engage in discussion with the tutor.
  • Instruction during the school day up to five times per week: Unlike after-school tutoring programs, Saga tutoring requires an entire class period in its own classroom. Saga worked with schools to determine how tutoring could fit into the school day as well as its implications for teaching schedules, union scheduling, grading, and other process needs.

Allocating the funding

  • Urban Labs funding for a pilot at Harper High School: In order to test the effectiveness of Match’s high dosage tutoring model in Chicago, the Urban Labs received grants from local philanthropies for a pilot in Harper High School, situated in a high-poverty neighborhood, West Englewood.
  • School district and philanthropic funds for expansion: Once Saga had proved its efficacy through the pilot at Harper High School, the Chicago Public School district awarded Saga a four-year contract for their services (2015-2019). The district committed about $2.5M per year, covering about 80 percent of Saga’s operational costs. In order to fund operational expenses that the district funding did not cover, Saga and Urban Labs raised additional funding from local foundations. Ultimately, about a half dozen foundations made contributions that met the remaining costs.
  • AmeriCorps funds to expand Saga’s reach: In 2019, Saga received a national AmeriCorps award that provided additional resources. These funds could either be used to reduce the school district’s commitment or expand Saga’s tutoring services into more classrooms. The district chose to devote the funds to further expansion.

Implementing the plan

  • Following pilot and buy in from district, recruiting and selecting schools: Officials from the Chicago Public School District worked closely with Saga to develop a list of criteria with which to determine the schools that were best suited to Saga’s program, starting with a single pilot school. The criteria for school selection included consideration of each school’s student body, union culture, and math departments. Following the success of the pilot, Urban Labs met with several foundations to present the results and raise funds for the program’s expansion. Once funding was in place, Saga expanded to 12 Chicago schools in 2013 and 15 in 2014.
  • Recruiting the first crop of tutors and site directors: Saga worked diligently to find local tutors, all college graduates, and created a new and expanded training program. Tutor candidates were required to demonstrate their ability to establish rapport with students in a mock tutoring session with current students who then voted on whether or not they felt they could see themselves learning from the tutor.
  • Building relationships with schools and math departments: After identifying a list of potential schools, district officials then helped Saga convene the school principals and help persuade them to adopt the program. In addition to principals, Saga spoke with the leadership of the schools’ math departments to secure their buy-in.
  • Training the first group of Chicago tutors: To account for the differences between disadvantaged students in Chicago and the environment of Boston’s Match school, Saga adapted and expanded the tutor training program. Following training, Saga planned to continue to monitor these teaching practices and refine the tutors’ skills continuously throughout the year.
  • Identifying the right students: Working closely with the Urban Lab, Saga’s leadership spoke with teachers and parents to identify students who would most benefit from daily tutoring. School staff then worked with these students to schedule tutoring into their daily class schedule.
  • Launching and expanding Saga in Chicago: With schools selected, tutors and site directors placed in schools, and students enrolled in tutoring sessions, implementation of Saga tutoring began in the new schools. An evaluation was built in to the expansion, with the University of Chicago’s continuing research building a body of rigorous evidence to demonstrate Saga’s impact.

Measuring and refining the approach

  • Experimenting with tutor-student ratios: In recent years, Saga has begun using e-learning resources in tandem with in-person tutoring and are currently assessing the efficacy of new models. In the traditional Saga model, tutors would work with a total of 12-14 students. In one new model under assessment, tutors have 24 students total in their caseload. In this delivery method, tutors provide live instruction to two students at a time, while two others work on a virtual learning platform right next to the tutors in the same room. On the next day, the two sets of students switch places, so that every student is receiving full-class live attention every other day. So far, Saga’s research has shown that this model has the same efficacy as the original model but allows tutors to take on twice the caseload.
  • Pilot testing e-learning models: In addition to trying new tutor-student ratios, Saga is testing a model where tutors remotely work with students while a site director supervises the students in the classroom in person. This has also shown promising early results.
  • Adjusting site director qualifying criteria: Initially, many of Saga’s site directors (who monitor classrooms during tutoring sessions) were hired from the tutoring pool. Saga has since moved to hiring site directors who were former teachers of 6th to 10th grade math, with some exceptions.
  • Building school capacity to implement high dosage tutoring: Over the next several years, Saga plans to develop programs that will allow schools to implement their model on their own and is working with the Chicago Public Schools system to launch its own district-led tutoring program.

High-dosage tutoring: Saga Education in Chicago Public Schools

Confronting the problem

High-dosage tutoring: Saga Education in Chicago Public Schools

Designing the strategy

High-dosage tutoring: Saga Education in Chicago Public Schools

Allocating the funding

High-dosage tutoring: Saga Education in Chicago Public Schools

Implementing the plan

High-dosage tutoring: Saga Education in Chicago Public Schools

Measuring and refining the approach

Acknowledgements

Results for America would like to thank the following individuals for their help in completing this case study: Alan Safran, Antonio Gutierrez, and Barbara Algarin of Saga Education, Thomas Rodgers of White Board Education, Roseanna Ander and John Wolf of the University of Chicago's Urban Labs, Rukiya Johnson of the Rush University System for Health (formerly of Chicago Public Schools), and Anna Pavichevich of Chicago Public Schools.

This case study was written by Jonathan Timm and Ross Tilchin.