Publication Author: Gina McGovern

Preparing Youth to Thrive: Methodology and Findings from the SEL Challenge

The Social and Emotional Learning (SEL) Challenge was undertaken in pursuit of two ambitious goals: To identify promising practices for building SEL skills with vulnerable adolescents, and to develop technical supports for use of these SEL practices at scale in thousands of out-of-school time (OST) settings. The study design included a qualitative methodology, expert practitioners, and performance studies at each of eight exemplary programs. The products of the Challenge—standards for SEL practice and the suite of SEL performance measures—is designed to help OST programs focus deeply on SEL practice, assess their strengths, and improve the quality and effectiveness of their services using a continuous improvement approach.

By focusing systematically at a granular level of adult and youth behavior, the Challenge content supports use in practice-oriented settings and systems—youth programs, school day classrooms, mentorships, residential treatment, apprenticeships, workplace, families—where the qualities of adult-youth interaction and learning are a primary concern. We hope that local policy makers and funders will use the Challenge as a template for identifying the exemplary SEL services already available in their communities and make sure that they are adequately recognized, resourced, and replicated.

The promising practices [were] featured in a Field Guide, Preparing Youth to Thrive: Promising Practices for Social and Emotional Learning (Smith, McGovern, et al., 2016), a companion website, and a suite of tools and technical assistance. This report, Preparing Youth to Thrive: Methodology and Findings from the SEL Challenge, describes how the partnership carried out the work of the Challenge and what we learned as a result. Findings from the SEL Challenge include:

  1. The Challenge methodology successfully identified exemplary SEL offerings and produced 34 standards, 78 practice indicators, and 327 vignettes for building SEL skills with vulnerable youth.
  2. The suite of performance measures developed for the Challenge is feasible to implement and demonstrates sufficient reliability and validity for both continuous improvement and evaluation uses.
  3. The performance studies indicate that the exemplary offerings were exceptionally high quality compared to other OST programs and that youth skills improved in all six SEL domains. Skill growth also occurred for the higher risk groups. Benchmarks for SEL performance include: (a) Diverse staff and youth, intensive participation, and expert adult guidance; (b) Collaborative organizational cultures; (c) Exceptionally high quality instruction and youth engagement; (d) A consistent pattern of positive SEL skill growth across measures, offerings, and risk status.
  4. The offerings shared an OST-SEL intervention design: project-based learning with intensive co-regulation.

The Discussion section addresses generalizability of findings, cautions about SEL measurement, and study limitations.

Preparing Youth to Thrive: Promising Practices for Social Emotional Learning

Executive Summary

The Social and Emotional Learning (SEL) Challenge was designed to identify promising practices for building skills in six areas: emotion management, empathy, teamwork, initiative, responsibility, and problem-solving. The Challenge was a partnership between expert practitioners (youth workers, social workers, teachers) delivering exemplary programs in eight unique communities, a team of researchers, and a national foundation.

Although each of the exemplary out-of-school-time (OST) programs that were studied uses a different curriculum, their approaches to building social and emotional skills have important similarities, and these are the subject of the guide. This guide presents 32 standards and 58 indicators of SEL practice in six domains as well as four curriculum features that were shown to be foundational for supporting SEL practices.

For teens, social and emotional learning helps build resiliency and a sense of agency—skills critical for navigating toward positive futures of their own design. Social and emotional skills are the skills for action that help youth on that path. These skills go by several names: 21st-century skills, soft skills, and character education, and are experiential learning, positive youth development, etc. We focused on translating the “action” that staff and youth see in exemplary out-of-school-time programs into plain language. The guide sets things to share widely and in plain language how professionals can embed practices that support social and emotional learning with greater intentionality.

This guide is designed to start conversations about the kinds of social and emotional skills readers hope will grow in the adolescents they know and care about and to support the adult practices that help these skills to grow. We hope that readers will use the guide to create and pursue their own action plans for implementing SEL in their OST programs and networks. The guide is designed for readers to use on their own terms, not as a book to be read front-to-back—advice to readers is provided at the end of the introduction.

Framing an Evidence-Based Decision About 21st CCLC

Summary

In this commentary we’ve described a mismatch between the afterschool theory of change and the intent-to-treat evaluation design, suggesting that when these powerful evaluation designs are applied to broad developmentally focused programs such as 21st CCLC, the effect sizes are likely to be small but substantively important. We’ve also suggested that afterschool evaluations need to include description and measurement of critical program qualities and the specific skills these programs are focused on growing. An example of a 21st CCLC evaluation which combines both an intent-to-treat impact design with a substantial effort to understand how all of the pieces fit together is the Texas 21st CCLC Year 2 Interim Evaluation Report. We further suggested that 21st Century has intentionally fostered an important social innovation, the afterschool QIS. Because 21st CCLC represents ethic of accountability for service quality in many states, the program has also sparked a broader social movement that has improved the state of afterschool for all American youth.

The STEM supplement to the Youth Program Quality Assessment

Introduction

curricula from the fields of environmental science, technology, engineering, and mathematics (“STEM”) at 10 sites(one partner delivered the same curriculum at two sites). The offerings were organized at 10 school-based Afterzone sites and each offering included field work in the local Providence region. Across the 10 sites, STEM curricula were delivered to a total of approximately 250 middle school students (about 25 students per Afterzone section).

In order to evaluate the Afterzone Summer Scholars model and collect information for future improvement, PASA (a) hired an external evaluator for the project; (b) committed to providing continuous improvement supports to participating program managers and content providers (quality assessment and coaching); and (c) formed an evaluation advisory board to monitor the development and implementation of the external evaluation. In addition, PASA contracted with the David P. Weikart Center for Youth Program Quality (Weikart Center) at the Forum for Youth Investment to develop an observation-based measure of instructional practices to support continuous improvement during STEM programming. This report describes the process of development of the STEM supplement to the Youth Program Quality Assessment (Youth PQA; HighScope, 2005) and preliminary reliability and validity evidence based on data collected during Afterzone Summer Scholars program.