The multilevel person in context ~ neuroperson (MPCn) model: Guidance for quality improvement systems (QIS) focused on socio-emotional skill growth and transfer outcomes

Reviewing theory and research on socio-emotional learning (SEL) highlights the importance of a wide range of psychological and behavioral skills, ranging from very specific psychological processes that occur on the order of milliseconds (e.g., updating working memory) to broad patterns of behavior that occur over minutes, days, and months (e.g., teamwork and relationship skills). Attempts to organize this vast array of skills into a coherent theoretical or measurement framework has yielded dozens of unique but overlapping frameworks. For example, a recent review of SEL theory, research, and practice by the American Institutes for Research (Berg et al., 2017) found over 100 different SEL frameworks.

The multilevel person-in-context model of youth development programs (Smith, McGovern, Peck, et al., 2016) facilitates thinking about how the SEL skills being developed at the point of service (POS) are both (a) embedded within the wider context of policy decisions, family background, and out-of-school time (OST) program quality and (b) related subsequently to shorter-term youth outcomes (e.g., SEL skill growth) and longer-term youth achievements (e.g., graduation and employment). The multilevel person-in-context model, and the corresponding neuroperson model that focuses on the structure and dynamics of SEL skill growth (described below), together constitute the multilevel person-in-context, neuroperson (MPCn) framework that was developed to improve the precision, validity, and comprehension of performance data used in lower-stakes quality improvement systems (QIS) in the OST sector (Smith, McGovern, Larson, et al., 2016; Smith, McGovern, Peck, et al., 2016; Smith et al., 2019).

Citation: Peck, S. C., Smith, C., & Smith, L. (2019). The multilevel person-in-context ~ neuroperson (MPCn) model: Guidance for quality improvement systems (QIS) focused on socio-emotional skill growth and transfer outcomes. Ypsilanti, MI: QTurn.

Promoting Healthy Development of Young People: Outcomes Framework 2.0

In the summer of 2018, the Local Government Association (LGA) in England commissioned the Centre for Youth Impact to produce an outcomes framework to help partners across the English youth sector to develop and agree on mutual aims to support young people in their local areas. The work was in response to LGA’s consultations that led to its vision statement described in the report, Bright Futures: our vision for youth services, published at the end of 2017. In that report, the authors noted:

“A clear outcomes framework can help to effectively monitor the impact of a service at key milestones to spot where things aren’t working and provide opportunities to make changes where needed. It can also support evidence of collective impact across the system.”

The proposed framework was intended to support partners’ efforts to track and understand the short-, medium-, and longer-term impacts of their work on the lives of young people. The framework needed to be simple and adaptable to provision for different groups of young people and for diverse approaches.

This document is an update on the framework and is the result of two phases of work: an initial phase including desk research and widespread consultation with practitioners, commissioners and elected members, and a second phase to test the proposed framework in action. The work was undertaken by the Centre’s network of regional impact leads and its central team.

Measure Once, Cut Twice: Using Data For Continuous and Impact Evaluation in Education Programs

Frustration and confusion often occur when practitioners require detailed information about program processes for continuous quality improvement (CQI) while policy-makers require evidence of outcome effects for accountability and funding.  Impact studies are often preferred over continuous improvement studies, but they seldom offer useful information to practitioners.  Per the conference theme, this situation leads to a worldview that emphasizes the limitations of social science methods for achieving practical purposes and welcomes arbitrary decision making (i.e., Type-2 error) in the absence of better evidence and arguments. 

This paper describes a generic quality-outcomes design (Q-O design) that meets the need for performance measurement methodology for concurrent and integrated impact evaluation and continuous improvement in the same organization; that is, measure once, cut twice

Citation: Smith, C., Peck S., Roy, R., & Smith, L. (2019).  Measure once, cut twice: Using data for continuous improvement and impact evaluation in education programs.  Meetings of the American Education Research Association, Toronto, ON, CA.