Tag: youth outcomes

Impact Evaluation for the Parent Child Plus Program

The Newark Trust for Education (NTE) Parent Child Plus (PC+) program is an evidence-based early childhood education program for families in the Newark, NJ. NTE seeks to evaluate performance by conducting analyses of existing data for a cohort of over 80 families, assessed four times over 46 weeks using observational measures of parenting practices and children’s socio-emotional skills. PC+ is intended to result in “improved child behaviors related to social‐emotional development and self‐regulation skills” (Organizational Research Services [ORS], 2010, p. 23).

The Quality-Impact-Equity Design and Methods (QDM) Toolbox (Smith, Peck, Roy, & Smith, 2019; Smith, Peck, & McNeil, 2020) was used to: (a) reconfigure existing measures for Parenting Practice Quality and Child SEL Skill to maximize reliability and validity for measuring socio-emotional skills and learning (SEL); (b) produce holistic profiles of parent and child skill (e.g., “whole child”) at each timepoint; and (c) apply pattern-centered analytics to estimate impact and equity effects of the PC+ program as implemented in Newark. Please note: We define impact in terms of the actual “in-the-world” structure of causes and effects, not in terms of counterfactuals. A brief description of the QDM methodology is provided in Appendix A (see also Smith et al., 2019)..

The PC+ program results reveal an overall impact pattern that suggests both a strong relation between parent and child skills and an effect of home visitors on both parent and child skills. Although, in almost all cases, the children of parents with high or growing parenting skills outperformed children with low or declining parenting skills, many children with parents in the low-skill profile for Parenting Practice Quality still experienced growth in SEL skills. This finding suggests that PC+ is working as it should, with parents and home visitors both having direct effects on child SEL skill growth. To fully demonstrate the impact of the NTE PC+ program given this “triadic” causal flow, we recommend (a) improving measures of PC+ fidelity and (b) including a small no-program sample of parents and children.

Contact QTurn to request Report Appendices A-I.

Impact Evaluation for the Palm Beach County Quality Improvement System 

Quality Improvement System (QIS) exposure moves afterschool programs to higher quality, increasing access to developmentally powerful settings and building children’s social and emotional learning skills. Higher quality is defined in terms of the quality of instruction (i.e., individuation, basic/advanced SEL, enrichment content), the stability of staff tenure, and evidence of children’s SEL skill growth.

In this study, we used performance data generated by Prime Time Inc. in Palm Beach County and fully pattern-centered methodology to describe the chain of causal effects as a cascade of sequential impacts. We sought to answer two specific questions about implementation and children’s SEL skill growth: What is the impact of QIS exposure on program quality (i.e., best practices, low staff turnover, great content), particularly for programs that have lower program quality at baseline? What is the impact of exposure to high program quality on student SEL skills?

Findings demonstrate that (1) QIS exposure causes program quality improvement to occur and (2) exposure to high quality corresponds to SEL skill growth. Specifically, (1.a) quality increased dramatically over three years of exposure to the Palm Beach County QIS; (1.b) programs with Low Quality at QIS entry improved when exposed to even moderate QIS Fidelity; (2.a.) children exposed to higher-quality programs had greater SEL skill maintenance and gains compared to children exposed to lower-quality programs; and (2.b) children with Low SEL Skill at entry made greater gains at all levels of program quality.

This pattern of findings suggests that the Prime Time QIS design is successfully building the quality of services available in the county in substantively meaningful ways – by increasing the quality of instruction, increasing the tenure of staff, and growing SEL skills for students who need it most.