Throughout the development of the HMM, community needs assessment, performance measure monitoring and evaluation activities have informed the model's guiding principles, logic model, implementation activities, and HM practices.
Evaluation Activities, Measures and Analysis
Annually, the BTH team monitors available public health data sets to identify inequities and trends over time to focus HMM programming and supports in communities with greatest need (priority communities). The following HC secondary data sets are included in this annual needs assessment surveillance:
- Birth rates by age, city, neighborhood, and race/ethnicity (Minnesota Department of Health Health Statistics)
- Sexually transmitted infection rates by age, gender, city, and race/ethnicity (Minnesota Department of Health Health Statistics)
- Sexual activity by school district, grade, gender, and race/ethnicity (source: Minnesota Student Survey)
- Contraception use by school district, grade, gender, and race/ethnicity (source: Minnesota Student Survey)
- Well-being indices by school district, grade, gender, and race/ethnicity (source: Minnesota Student Survey)
- School enrollment records by age, gender, race/ethnicity, and free and reduced-price lunch (source: Minnesota Department of Education)
- Population demographics by city, age, race/ethnicity, family income below 100% poverty, and % adults less than HS education (source: Census)
- Population mental well-being and health care utilization (source: HC SHAPE Survey)
In addition to reviewing available public health data, the BTH team reviews available qualitative data gathered by partners and engages community stakeholders (youth, clinic partners, school staff and stakeholders) in assessment activities to interpret available data and provide recommendations for programming, supports and resource allocation.
BTH also collects and monitors the following performance measures to inform program implementation:
- HMM activities completed by program type
- HMM reach by program activity and participant demographics
- Referrals to supportive services including health care
- Saturation of evidence-based interventions at OPA funded HMM sites
- Quality of implementation through program observations and fidelity monitoring at OPA funded HMM sites
- Youth, parent, and community engagement
In addition to ongoing secondary data surveillance and performance measure monitoring, in 2021 BTH worked with external evaluators to develop and implement an evaluation plan for primary data collection. This work deeply engaged BTH stakeholders including young people, utilized an equity framework, and focused on feasibility and sustainability in any products developed and recommendations identified. This work resulted in:
- 21 stakeholder interviews and 2 listening sessions with HMs, clinic partners and school staff to inform the evaluation plan and identify HMM impacts
- Revised and annotated HMM logic model
- Five online instruments for classroom education participants, 1 to 1 session participants, school/site staff, health mentors, and clinic partners to measure impacts, satisfaction, and improvement areas
- Data analysis and interpretation plan with HMM site specific results and feedback loop process for continuous quality improvement
- Identified how the HMM utilizes a racial equity framework to address disparities in adolescent sexual health in HC
With support from HCPH's assessment team, BTH team manages the collection and analysis of all primary data sources including the above-mentioned performance measures and evaluation instruments. BTH utilizes the following tools for data analysis:
- Health Mentor Model database reports of quantitative reach data
-Qualtrics quantitative and qualitative analysis on performance measures and evaluation instruments
- Stakeholder Engagement focused reflection with BTH Community Advisory Group, HMs and other stakeholders to interpret evaluation results and consider strategic directions
Evaluation Results
The following results demonstrate how the HMM met program goals and objectives outlined previously in this application over the past year.
Surveillance of community needs assessment data demonstrated that the following geographic communities in HC experience the greatest disparities in sexual health and well-being indices compared to other HC communities, HC in general, and MN: Brooklyn Center; Brooklyn Park; Minneapolis Central; Minneapolis North; Richfield; Robbinsdale.
Additionally, needs assessment activities confirmed that BTH and the HMM needs to continue to support building health equity among communities of color in HC as they experience disproportionate rates of STIs, teen births, and well-being challenges as compared to their peers.
Monitoring of performance measures demonstrated the following:
- 7 community clinic partners
- 18 HMM sites in priority communities
- 5000+ young people reached by HMs (15 & 16-year-old is largest proportion)
- 700+ referrals to sexual health services by HMs
- 250+ referrals to mental health and well-being services by HMs
- 94 program observations with mean quality rating of 4.35 out of 5.0
- 24% saturation of evidence-based interventions across Office of Population Affairs funded sites
Evaluation activities over the past year demonstrated the following results. Young people (N=666) from 16 schools agreed that the classroom sex education provided by the HM:
- Made them feel better prepared to make decisions about their health (93% agreed)
- Created a comfortable environment to talk about sexual health (95% agreed)
- Information shared was clear and understandable (95% agreed)
- Presentation kept them interested (79% agreed)
- Information was relevant based on their personal identity (gender, sexual orientation, race/ethnicity) (89% agreed)
- Information shared was helpful to them (93% agreed)
Young people (N=85) from 13 schools agreed that the 1 to 1 sessions provided by the HM:
- Made them feel better prepared to make decisions about their health after visiting the HM (77% agreed)
- Would recommend the HM to a friend (81% agreed)
- Would go back to the HM if needed future support (80% agreed)
- Felt heard by the HM (82% agreed)
- Created a safe and comfortable environment to talk (86% agreed)
School staff (N=289) from 10 schools reported:
- 92% were aware of the HM services at their school
- 84% agreed the HM meets the needs of a diverse array of students (16% did not know)
- 81% agreed the HM effectively engages with students (18% did not know)
- 83% agreed the HM is knowledgeable about adolescent sexual health (16% did not know)
- 39% had made a referral to the HM
- 13% reported that they co-facilitated education with the HM
Staff reported overwhelming support for the HMM at their school including:
The health mentor understands the barriers that our community faces day to day. They can meet the students where there are and give them what they need.”
Best program I've experienced in over a decade working in schools.”
Results from the HM and HM manager surveys were merged with the stakeholder interviews and listening sessions and the following themes were identified:
- The HMM works and participants are highly satisfied
- Schools are supportive of the HMM
- Clinic and school partnerships need to be strong for HMM success
- HMs need ongoing support, training, and guidance
- HMM needs to be operationalized and consistent across sites
- Data collection is strenuous but valuable
Based on these themes and other findings from evaluation activities and reflection with BTH stakeholders, BTH is developing a HMM Manual. This manual identifies the theoretical and evidence base underpinning the model, operationalizes model components and expectation, and identifies policies and procedures relevant to HMM activities. This manual will allow uniform practices for onboarding and supporting HMs, articulate shared expectations for all partners, maintain consistency in the model activities across sites, and build in sustainability for future replication.
In summary, assessment data confirmed that the HMM addresses the current needs identified by young people, parents, community stakeholders, and adolescent health professionals and that the communities in which the HMM is being offered, remain the priority communities. It was also clear from evaluation activities that the programming implemented by the HMM reflect strategies the community identified as important to meeting the identified needs, especially the inclusion of well-being within the framework of adolescent sexual health promotion.
BTH will sustain the focus on continued assessment, monitoring, evaluation, and continuous quality improvement when implementing the HMM. Specifically, BTH will continue to engage the community and stakeholders in assessment, evaluation and planning related to the implementation of the model to maintain relevant, high quality, community-driven, equity-focused programming in HC communities.