The most deadly mass school shooting in US history was at Sandy Hook Elementary School in Connecticut in December 2012. Because this and many other mass shootings have involved people with serious mental health challenges, the Obama administration decided to enhance the nation’s capacity to head off such events through prevention and early identification and treatment. The main initiative in the public schools is the Advancing Wellness and Resilience in Education (AWARE) program.
The Center on Disability Studies (CDS) took a lead role in writing the Hawaii State Department of Education’s successful proposal for an AWARE grant, and is now serving as project evaluator. Hawaii’s project is called HI-AWARE, and it is funded at about $12 million over five years through 2019 to cover the school complexes of Nanakuli-Waianae, Leilehua-Mililani-Waialua, and Kau-Keeau-Pahoa.
HI-AWARE is designed to enhance the prevention and early intervention components of the Hawaii public school system’s three-tier system of student services and supports, which is mainly focused on Level 3 students already diagnosed with serious mental health challenges.
Evidence-based practices being tested to better prevent or reduce student mental health challenges include: creating school-based teams that collect and use data to identify and address local challenges; increasing school and community early intervention capacity; addressing social-emotional learning in academic curricula; identifying and attending to symptoms of trauma; and training hundreds of personnel and community members in Youth Mental Health First Aid (recognizing symptoms of distress in youth and knowing when and where to make referrals).
In conducting the evaluation, the CDS Evaluation Team worked with the three complex HI-AWARE teams to develop an innovative use of the Youth Risk Behavior Survey (YRBS). The YRBS is an initiative of the US Centers on Disease Control. The survey consists of over 100 items on health-related topics, ranging from diet and exercise to use of drugs and alcohol to symptoms of depression. The survey is conducted every two years with representative samples of student across the country, and is one of the main sources of information about how the risk behaviors of the nation’s youth are changing over time.
Hawaii’s AWARE grant proposal had used YRBS data for Hawaii for 2014 to identify priority areas, including general needs to reduce bullying, create a higher sense of safety and inclusion for all in the schools, and address relatively high rates of suicidal thoughts. The evaluation challenge is to assess any impact that HI-AWARE activities might have on these areas. The YRBS itself is not suited for measuring impact because it is only conducted once every two years, and then only with a relatively small sample of students.
The solution was to develop the HI-AWARE Risk Behavior Survey, or HARBS, which uses only about 20 YRBS items that directly relate to the project’s objectives. There are two versions, one for middle school and the other for high school. A great advantage of using YRBS items is that they have been shown over the years to yield valid and reliable information.
The HARBS is being conducted by most of the HI-AWARE target schools for all students in the Fall semester every year, in order not to conflict with the YRBS which is done in the Spring semester. An online system allows students to anonymously complete the survey, and the results can be compiled according to school. In this way each school can explore which risk behaviors are most in need of attention, such as bullying or suicidal thoughts.
There are substantial advantages of all students completing the survey. One is that we can have more confidence in the reliability of the results because sampling errors can be expected to be minimal. The other advantage of all students participating is that they can be considered cohorts according to grade to be followed longitudinally. By contrast, the YRBS random sampling design means that the cross-section of students surveyed one year may not be entirely comparable to the cross-section of students surveyed two years later.
The CDS Evaluation Team has helped summarize the HARBS results for each school using graphs. Schools report that the results of the first administration of the HARBS in the Fall 2016 semester have been valuable in their self-assessment and planning activities, and the CDS Evaluation Team is preparing for the next round of administration in Fall 2017.
The CDS Evaluation Team for HI-AWARE is staffed by David Leake firstname.lastname@example.org, Leslie Okoji, and Marla Arquero.
David Leake is a researcher, evaluator, project director, and principal investigator for a range of research and demonstration projects in the areas of transition to adulthood, child and adolescent mental health, and development of curricula that are culturally responsive for students including those of Native Hawaiian heritage. As an East-West Center fellow, Leake received a PhD in medical anthropology and Masters of Public Health specializing in international health.