QTurn’s Afterschool Evaluation Plan 2020

In partnership with the Michigan Department of Education and Michigan After-School Partnership, QTurn has put together a suite of evaluation tools specifically developed for afterschool programs during the 2020-21 school year – The Afterschool Evaluation 2020 (AEP2020). The Management Practices Self-Assessment (MPSA), Guidance for Learning at a Distance (GOLD), and the Adult Rating of Youth Behavior (ARYB) are a suite of tools originally designed for 21st Century Community Learning Center programs, however any youth-serving out-of-school time (OST) organization will find the AEP 2020 applicable to their work.

Guidance for Out-of-School Time Learning at a Distance (GOLD)

Management Practices Self Assessment (MPSA)

Adult Rating of Youth Behavior​ (ARYB)

The multilevel person in context ~ neuroperson (MPCn) model: Guidance for quality improvement systems (QIS) focused on socio-emotional skill growth and transfer outcomes

Reviewing theory and research on socio-emotional learning (SEL) highlights the importance of a wide range of psychological and behavioral skills, ranging from very specific psychological processes that occur on the order of milliseconds (e.g., updating working memory) to broad patterns of behavior that occur over minutes, days, and months (e.g., teamwork and relationship skills). Attempts to organize this vast array of skills into a coherent theoretical or measurement framework has yielded dozens of unique but overlapping frameworks. For example, a recent review of SEL theory, research, and practice by the American Institutes for Research (Berg et al., 2017) found over 100 different SEL frameworks.

The multilevel person-in-context model of youth development programs (Smith, McGovern, Peck, et al., 2016) facilitates thinking about how the SEL skills being developed at the point of service (POS) are both (a) embedded within the wider context of policy decisions, family background, and out-of-school time (OST) program quality and (b) related subsequently to shorter-term youth outcomes (e.g., SEL skill growth) and longer-term youth achievements (e.g., graduation and employment). The multilevel person-in-context model, and the corresponding neuroperson model that focuses on the structure and dynamics of SEL skill growth (described below), together constitute the multilevel person-in-context, neuroperson (MPCn) framework that was developed to improve the precision, validity, and comprehension of performance data used in lower-stakes quality improvement systems (QIS) in the OST sector (Smith, McGovern, Larson, et al., 2016; Smith, McGovern, Peck, et al., 2016; Smith et al., 2019).

Citation: Peck, S. C., Smith, C., & Smith, L. (2019). The multilevel person-in-context ~ neuroperson (MPCn) model: Guidance for quality improvement systems (QIS) focused on socio-emotional skill growth and transfer outcomes. Ypsilanti, MI: QTurn.

Promoting Healthy Development of Young People: Outcomes Framework 2.0

In the summer of 2018, the Local Government Association (LGA) in England commissioned the Centre for Youth Impact to produce an outcomes framework to help partners across the English youth sector to develop and agree on mutual aims to support young people in their local areas. The work was in response to LGA’s consultations that led to its vision statement described in the report, Bright Futures: our vision for youth services, published at the end of 2017. In that report, the authors noted:

“A clear outcomes framework can help to effectively monitor the impact of a service at key milestones to spot where things aren’t working and provide opportunities to make changes where needed. It can also support evidence of collective impact across the system.”

The proposed framework was intended to support partners’ efforts to track and understand the short-, medium-, and longer-term impacts of their work on the lives of young people. The framework needed to be simple and adaptable to provision for different groups of young people and for diverse approaches.

This document is an update on the framework and is the result of two phases of work: an initial phase including desk research and widespread consultation with practitioners, commissioners and elected members, and a second phase to test the proposed framework in action. The work was undertaken by the Centre’s network of regional impact leads and its central team.

Impact Evaluation for the Palm Beach County Quality Improvement System (QIS) using Fully Pattern-Centered Analytics

In this study, we used performance data generated by Prime Time Inc. in Palm Beach County and fully pattern-centered methodology to describe the chain of causal effects as a cascade of sequential impacts. We sought to answer two specific questions about implementation and children’s SEL skill growth: What is the impact of QIS exposure on program quality (i.e., best practices, low staff turnover, great content), particularly for programs that have lower program quality at baseline? What is the impact of exposure to high program quality on student SEL skills?

Citation: Smith, C., & Peck S. C. (2019). Impact evaluation for Palm Beach County quality improvement system (QIS) using fully pattern-centered analytics. Ypsilanti, MI: QTurn.