Impact Evaluation for the Palm Beach County Quality Improvement System 

Quality Improvement System (QIS) exposure moves afterschool programs to higher quality, increasing access to developmentally powerful settings and building children’s social and emotional learning skills. Higher quality is defined in terms of the quality of instruction (i.e., individuation, basic/advanced SEL, enrichment content), the stability of staff tenure, and evidence of children’s SEL skill growth.

In this study, we used performance data generated by Prime Time Inc. in Palm Beach County and fully pattern-centered methodology to describe the chain of causal effects as a cascade of sequential impacts. We sought to answer two specific questions about implementation and children’s SEL skill growth: What is the impact of QIS exposure on program quality (i.e., best practices, low staff turnover, great content), particularly for programs that have lower program quality at baseline? What is the impact of exposure to high program quality on student SEL skills?

Findings demonstrate that (1) QIS exposure causes program quality improvement to occur and (2) exposure to high quality corresponds to SEL skill growth. Specifically, (1.a) quality increased dramatically over three years of exposure to the Palm Beach County QIS; (1.b) programs with Low Quality at QIS entry improved when exposed to even moderate QIS Fidelity; (2.a.) children exposed to higher-quality programs had greater SEL skill maintenance and gains compared to children exposed to lower-quality programs; and (2.b) children with Low SEL Skill at entry made greater gains at all levels of program quality.

This pattern of findings suggests that the Prime Time QIS design is successfully building the quality of services available in the county in substantively meaningful ways – by increasing the quality of instruction, increasing the tenure of staff, and growing SEL skills for students who need it most.