Tag: Self-Assessment

Youth Program Quality Self-Assessment Pilot Study

Summary

Overall 24 sites within 17 grantees participated in the self-assessment pilot study by assembling staff teams to collect data and score the Youth Program Quality Assessment (PQA).

At each site an average of 5 staff spent an average of 13 staff hours to complete the self-assessment process.

Whether using an absolute standard or group norms as a benchmark for interpretation of data from the Youth PQA Self-Assessment Pilot Study (hereafter called the Pilot Study), quality scores were very positive for participating programs and also reflected the tendency of self-assessment scores to be biased toward higher quality levels.

The quality scores followed the same pattern as outside observer scores in other samples, highest on for issues of safety and staff support and lowest on higher order practices focused on interaction and engagement.

Youth PQA data collected using the self-assessment method demonstrated promising patterns of both internal consistency and concurrent validity with aligned youth survey responses.

Two thirds or more of sites reported that the observation and scoring process helped the selfassessment team to have greater insight into the operation of their programs, talk in greater depth about the program quality than usual, and have more concrete understanding of program quality.

Site directors and local evaluators said that the self-assessment process was a source of good conversations about program priorities and how to meet them. In almost all cases, concrete action followed from the self-assessment process.

Site directors and local evaluators demonstrated the ability to improvise the self-assessment method to fit local needs.

Program directors, site coordinators, and local evaluators have used the Youth PQA and statewide Youth PQA data to generate statewide program change models, suggesting that the instrument and data are useful for setting system-level improvement priorities.

Subjective Well-Being (SWB)

In OST organizations that focus on student’s socio-emotional skills and wellbeing, pandemic conditions created new urgency to staff’s desire to know how students and caregivers were doing. While many OST organizations know how to ask families about basic needs created by the pandemic – health care, housing, food, transportation – sometimes staff are less comfortable checking in on mental health issues.

The concept of subjective-well-being (SWB) has traditionally been assessed using measures of happiness and satisfaction with life. Most of the measurement instruments developed to assess SWB use several items to assess each of several dimensions. However, for simplicity and efficiency, we recommend using only one item for each of the three dimensions: positive feelings, negative feelings, and life satisfaction.

Positive Feelings

On a scale of 1 to 10: During the past month, how often have you had positive feelings (e.g., happiness, joy, interest, calmness, and love)?

Negative Feelings

On a scale of 1 to 10: During the past month, how often have you had negative feelings (e.g., sadness, fear, boredom, anxiety, and hate)?

Life Satisfaction

On a scale of 1 to 10: During the past month, how satisfied have you felt with your life?

Download

Management Practices Self-Assessment (MPSA)

The Management Practices Self-Assessment (MPSA) is a measurement instrument created to empower Michigan 21st CCLC Project Directors, who manage at least one program site. The MPSA provides Project Directors with the opportunity to check-in with themselves and their team by identifying practices and policies that are well-documented and working well, or that could be updated or improved, and areas where they need additional assistance to reach their full potential.

Download the Guidebook
Download the Instrument

Citation: Smith, Thoreson, Smith, Peck, Sipes (2020). Management Practices Self-Assessment (MPSA) for Project Directors in Michigan’s 21st Century Community Learning Centers. Michigan Department of Education and QTurn LLC.

These materials were developed under a grant awarded by the Michigan Department of Education.

Findings from the Self-Assessment Pilot in Michigan 21st Century Learning Centers

Overall 24 sites within 17 grantees participated in the self-assessment pilot study by assembling staff teams to collect data and score the Youth Program Quality Assessment (PQA).

At each site an average of 5 staff spent an average of 13 staff hours to complete the self-assessment process.

Whether using an absolute standard or group norms as a benchmark for interpretation of data from the Youth PQA Self-Assessment Pilot Study (hereafter called the Pilot Study), quality scores were very positive for participating programs and also reflected the tendency of self-assessment scores to be biased toward higher quality levels.

The quality scores followed the same pattern as outside observer scores in other samples, highest on for issues of safety and staff support and lowest on higher order practices focused on interaction and engagement.

Youth PQA data collected using the self-assessment method demonstrated promising patterns of both internal consistency and concurrent validity with aligned youth survey responses.

Two thirds or more of sites reported that the observation and scoring process helped the self-assessment team to have greater insight into the operation of their programs, talk in greater depth about the program quality than usual, and have more concrete understanding of program quality.

Site directors and local evaluators said that the self-assessment process was a source of good conversations about program priorities and how to meet them. In almost all cases, concrete action followed from the self-assessment process.

Site directors and local evaluators demonstrated the ability to improvise the self-assessment method to fit local needs.

Program directors, site coordinators, and local evaluators have used the Youth PQA and statewide Youth PQA data to generate statewide program change models, suggesting that the instrument and data are useful for setting system-level improvement priorities.