Toolkits & Guides

Section 3: Evaluating the Effectiveness of Your Threat Assessment Program

What is an Effective Program?

Fundamentally, a threat assessment program is effective if it keeps everyone safe. However, safety is more than the absence of school shootings, which are statistically rare and will never occur in most schools.99 School safety includes both physical and psychological safety.100 Physical safety is concerned with physical acts of violence like fighting and assault, while psychological or social-emotional safety is concerned with problems such as verbal bullying and harassment. School threat assessment teams should consider multiple indicators of physical and psychological safety and focus on prevention and mitigation efforts such as lowering rates of fighting and bullying and making use of multi-tiered systems of supports and interventions.

School threat assessment aims to help students resolve problems and to support social-emotional and academic learning needs that might be identified in the threat assessment process. Therefore, an additional measure of effectiveness is to examine the services and supports provided to these students and the relevant outcomes. A part of case management is to monitor and promote the student’s behavioral and academic status. 

As noted in the U.S. Secret Service guide, “a crucial component of preventing targeted violence at school relies on developing positive school climates built on a culture of safety, respect, trust, and social and emotional support.”11 Threat assessment teams should encourage school-wide efforts to help students feel connected to the school. Students should have supportive peer groups and trusting relationships with their teachers. School climate surveys and other measures can be used to assess the healthy and protective qualities of the school. School discipline should be fair and equitable and concerned with facilitating student growth and responsibility. 

Implementation Fidelity

Developing a school threat assessment program is a dynamic process, requiring regular review and adjustment as necessary. One way to evaluate the effectiveness of your threat assessment program is to examine whether it has been implemented with fidelity. This section describes the importance of implementation fidelity and presents some general procedures and a scoring protocol that can be applied across threat assessment models. Fidelity includes making sure that the program is conducted in a fair and equitable manner that protects student rights.

The Importance of Implementation Fidelity

Knowing whether a program has been implemented with fidelity allows practitioners to understand how and why an intervention works. When a program is unsuccessful, it is difficult to judge whether the program was ineffective or was not properly implemented. Therefore, evaluating the fidelity of implementation is essential to evaluating a program and achieving maximum effectiveness.

School threat assessment programs should demonstrate implementation fidelity. Failure to carry out threat assessments effectively can have tragic consequences. Investigations of school shootings in Colorado and Florida revealed that the school’s threat assessment team had identified the student who subsequently carried out the shooting, but did not follow its threat assessment procedures with fidelity and did not take appropriate actions to prevent violence.102-104 These cases highlight the need for ongoing program evaluation to measure fidelity of implementation.

Factors Affecting Implementation Fidelity

Implementation fidelity is a concern for all kinds of programs and can be especially challenging in school settings.105 Staff need a solid understanding of the rationale and need for a program, and there must be strong support by the school leadership and staff who champion the particular program.106,107 Otherwise, the quality of program implementation will suffer because of the many competing demands and responsibilities in schools. Factors that negatively influence program implementation are insufficient staff, inadequate supervision, high staff turnover, heavy student caseloads, and lack of training.108,109  

One of the first hurdles to achieving implementation fidelity for schools implementing threat assessment is training all team members in each school. The team members need high-quality training that includes active learning, role-playing, and feedback. Training must be scheduled at a time when all team members can attend, which often means pulling staff from their regular school duties for a full-day workshop. In districts with many schools, the training must be coordinated across schools. After the initial training, there will be a need to train new team members each year due to staff turnover. Beyond training team members, the school should provide an orientation to threat assessment for its staff so that they understand and support the program. All staff members must understand the need to report threats promptly so that they can be investigated. Parents and students also need an orientation to threat assessment for the same reasons.

Another challenge to implementation fidelity occurs when staff have heavy caseloads and do not feel they can devote enough time to conducting a thorough threat assessment or following up with students after a threat assessment has been conducted. The team leader or another administrator with responsibilities for school safety must provide oversight and support so that teams follow their protocol.

The school administration and threat assessment team must be in alignment for implementation fidelity. For example, one school trained its threat assessment team and implemented its program, but the next year a new principal joined the school who did not understand the threat assessment approach. The team, who had received threat assessment training teaching them to effectively critique the current risk, evaluated a middle school student who drew a picture of a figure holding a very large knife in a threatening manner. The team found no evidence that the student had such a knife, a current peer conflict, or an intention to stab anyone. The information gathered in the assessment suggested that the boy was trying to impress his peers with a dramatic drawing. During the assessment the student recognized that his drawing could have been misinterpreted and was apologetic. The team decided that the threat was not serious and resolved it as a minor incident. However, the school principal applied a zero-tolerance approach and decided that the student should be given a long-term suspension in order to set an example and deter other students from aggressive behavior.

A further complication is that outside evaluators cannot always be available to monitor implementation, given that threat assessments typically occur infrequently and unexpectedly.49,86 However, local supervisors could be called upon to monitor or supervise cases. In one district, the head of the threat assessment program routinely reviews the digital records of ongoing cases and consults with the team on the most serious or complex cases. 

Measuring Implementation Fidelity

Schools need to regularly review the quality of their threat assessments. This evaluation should examine whether school teams are conducting threat assessments consistent with their guidelines and whether they are using evidence-based practices. Schools should review protocols to ensure that there are clearly defined roles and expectations for all team members.26

The evaluation of threat assessment is particularly challenging because threat assessment requires some degree of flexibility and professional judgment. Threat assessment models offer guidelines to assist a team’s decision-making rather than a prescriptive process. Threat assessments are intended to produce an individualized safety plan or intervention that depends on the student and the nature of the threat. 

Program implementation literature recommends that program developers specify core components of an intervention that are directly related to a program’s theory of change. This approach allows collaborators some flexibility to adapt a program to individual circumstances and helps ensure that the intended outcomes are achieved.107,110

There are several core components of threat assessment practice identified by threat assessment experts.10,43,45,80,103 These include:

  • Establishment of a multidisciplinary team, including training for all team members.
  • Education for students, parents, and staff about threat reporting and the school’s use of threat assessment.
  • Consistent use of standard threat assessment procedures (e.g., information gathering, threat classification, management strategies).
  • Regular team meetings to monitor cases and assess the effectiveness of risk reduction efforts.
  • Fair and equitable outcomes for students, including disciplinary consequences, law enforcement actions, and supports and interventions.

Case Outcomes

Another important measure of the effectiveness of your threat assessment program is to consider its influence on students. School teams should be monitoring the effectiveness of their safety plans as part of their case management efforts. For those students who remain on their caseload, the team should reviews the student’s well-being and safety on a regular basis, with more frequent review where there is greater concern. 

This section describes three important measures of student outcomes following a threat assessment: safety, support (services to the student), and equity and fairness. General considerations for each outcome measure are discussed, and a scoring tool is included.

Outcome One: Safety

Physical Safety

Threats to physically injure someone should be prevented or averted without anyone being harmed. Schools cannot realistically prevent all threats of violence from being carried out but should have a very low rate of violent incidents following a threat assessment. Violent incidents can range from simple assaults and fights to more serious injuries and, of course, school shootings. School shootings are so rare that it is not scientifically feasible to claim that an absence of a shooting can be attributed to threat assessment, but controlled studies have shown threat assessment results in a decline in bullying and other forms of victimization.111,112 Nevertheless, it is important for schools to document the number of threat cases they have investigated, what kinds of violent acts were threatened, and the number that resulted in some type of violent act (most often, a fight). 

A stack of paperwork on a desk that someone is filing through.

Psychological Safety

Researchers found that victims of aggression, especially bullying, suffer from impaired concentration, motivation, and engagement in learning that compromises academic achievement.113-115 Research also shows that school violence is negatively associated with teachers’ self-efficacy and professional engagement and that teachers who feel unsafe are more likely to leave the profession.116-119

A welcoming, supportive school environment that fosters respect, communication, and trust is foundational to school safety. A large body of research associates a positive school climate with several positive student outcomes, including better social-emotional adjustment and less peer aggression, misconduct, and weapons carrying.108,120-124 A positive school climate, characterized by high structure and high support, “is the foundation for a safe school.”125 The NTAC and the threat assessment models highlighted throughout this toolkit emphasize the importance of a positive school climate to threat assessment implementation and school safety.

Outcome Two: Student Support

Students should receive interventions and services to address the problems or concerns that are identified in the threat assessment. Teams will want to examine student outcomes to gauge the effectiveness of their efforts and identify ways to improve. Researchers found that schools have implemented a wide range of non-disciplinary supports for most students following a threat assessment.42 These included mental health supports, a behavior plan, and a modified schedule, among others. Strategies such as academic supports and mental health services for students are effective in addressing student needs and in improving student behavior following a threat assessment.31

Effective supports and services can mitigate the need for exclusionary discipline, which has well-established negative consequences for students.125 For example, the Texas study Breaking Schools’ Rules found that the widespread practice of removing students from school for minor misconduct did not improve school safety, had no apparent benefits to the students, and increased their risk of school dropout and juvenile court involvement.23

A Virginia study found that high school suspension rates were associated with higher dropout rates beyond the effects of school demographics and student attitudes toward school rules.126 The discipline gap is closely associated with the achievement gap and the “school-to-prison pipeline.”127 Schools serving high numbers of disadvantaged students are more likely to use exclusionary discipline and to have lower academic achievement and higher dropout rates, ultimately leading to higher juvenile crime rates.23,127 It is critical to track the relationship between threat assessment and school discipline over time at each school by evaluating threat assessment outcomes. 

It is also important to document whether students continue to make threats, engage in further aggressive behavior, or have disciplinary problems in the months and years after a threat assessment. Although students rarely carry out their threats, students who make more serious, substantive threats are much more likely to attempt to carry out the threat than students whose threats are classified as not serious.128 We also know from case studies of school shootings and averted shootings that the students most at risk for extreme violence had multiple incidents of conflict or misbehavior before a serious act of violence occurred.10,20 Although a threat assessment might avert an immediate crisis, some students remain at risk for continued problems and require further support.

An important goal of threat assessment is that the student can continue in school with continued behavior and academic progress. Threat assessment is able to generate support for students in need of services. Schools can document the extent to which students receive counseling, mental health services, academic support, or other interventions using the Student Outcome Tool. If students are receiving special education services, the threat assessment should be coordinated with the student’s IEP and comply with all special education requirements. Finally, there should be a follow-up assessment to determine whether the student was able to continue successfully in school.

Outcome Three: Fairness and Equity

Schools implementing threat assessment should use it as part of a comprehensive approach to building safe and positive school climates that reduce their use of exclusionary discipline, except in the most serious cases where school removal is indicated for safety reasons. 

Although threat assessment teams generally do not make disciplinary decisions, their assessments should inform the disciplinary process. Schools using threat assessment should be able to avoid major disciplinary actions (such as long-term suspension or expulsion) for minor student misbehavior that does not pose a serious threat. Schools should consider law enforcement outcomes as well as disciplinary outcomes. Although threat assessment teams do not make law enforcement decisions about students, the threat assessment process should inform law enforcement decision-makers (for a detailed discussion of information sharing during the threat assessment process, see Student Rights in Section 2 of this toolkit). 

There should be a relatively low rate of students who are arrested, charged, or placed in a detention facility as a result of a threat. In a study of 1,865 cases in Virginia schools, only 1% of students were arrested and fewer than 1% were placed in juvenile detention. Court charges were registered in approximately 5% of cases.100 However, court charges do not necessarily result in convictions and sometimes are resolved with a referral for services.134 A study of 22,694 Florida threat assessment cases found an arrest rate of 0.7%, incarceration of 0.1%, and court charges of 1.8%.38

As noted above, the threat assessment team does not make disciplinary decisions or undertake law enforcement actions; however, schools should examine the disciplinary decisions and law enforcement outcomes for their cases.51 One of the major concerns in American education is that students from some minority groups and students with disabilities are subject to disproportionately higher rates of exclusionary discipline. Outcomes should be equitable across student demographic groups as defined by race, ethnicity, or disability status.

Several studies have found little or no disparity across Black, Hispanic, and White students in the use of school suspension or law enforcement actions among students who received a threat assessment.36,42 However, in light of the prevalence of disparities in exclusionary discipline observed in the general student population (not limited to students receiving a threat assessment) as reported by the U.S. Department of Education, it is important for schools to monitor and review disciplinary and law enforcement outcomes for students receiving a threat assessment.51,96

One concern is that students of color and students receiving special education services might be referred for threat assessment at a higher rate than other students. The threat assessment process uses multiple sources of information and multiple perspectives of team members to help protect against bias in the decision-making process. Research indicates that these students do not receive disproportionate disciplinary or law enforcement outcomes when referred for a threat assessment.27,32 Studies have shown that students are much more likely to be excluded from school for a threat if the school does not use threat assessment. 

When appropriate, a referral for threat assessment is preferable to exclusionary discipline such as suspension or expulsion, especially when the threat assessment program is being evaluated with fairness and equity of outcomes in mind.

As the National Association of School Psychologists concluded, “When BTAM best practices are followed, the process helps prevent or reduce the overuse of restrictive placements and punitive measures for students with disabilities and students of color.”51

Procedures for Determining Equitable Outcomes

An important aspect of determining equitable outcomes for threat assessment is accurate record keeping, described in Section 2 of this toolkit. If a school keeps accurate threat assessment records and discipline records, it can determine whether there are disparities in referral rates by race, ethnicity, disability status, or other demographic characteristics of interest. A school can compare overall discipline rates (e.g., suspension rate) to the discipline rates for students receiving a threat assessment.

Table 5, below, provides an example of how differences in suspension by race within the group of students referred for threat assessment can be determined. A hypothetical school with 1,000 students reported 25 threat cases with the following student demographics and suspension outcomes.

Table 5: Hypothetical Differences in Suspension by Race

Threat Case DataTotal (N)White (N)Black (N)Hispanic (N)
Number of cases2510114
Number of cases resulting  in suspension7331
Percent suspended in group28%30%27%25%
Risk ratio*.90.83
*The risk ratio tells us how the risk for one racial/ethnic group compares to the risk for a comparison group. In this example, we used White as the comparison group (the risk ratio for Black students compared to White students is 27/30, the number of Black students suspended/number of Black students referred for TA *100 / the number of White students suspended/number of White students referred for TA *100).

Note that this is a hypothetical example. Most schools have relatively few (<10) threat assessment cases per year, but many more disciplinary referrals. Statistical analyses will be more reliable in larger samples, so that it is desirable to aggregate data across schools, perhaps at the district level. 

Suspension Rates Following Threat Assessment

Within the 25 threat cases, 7 resulted in suspension. Of the 10 White students referred for threat assessment, 3 (30%) were suspended. Of the 11 Black students referred for threat assessment, 3 (27%) were suspended, and 1 of the Hispanic students referred for threat assessment was suspended (25%). This suggests that there is parity in the rate of suspension following threat assessment for Black and White students; approximately equal proportions of White and Black students were suspended following a threat assessment.

Two school staff members stand in empty school atrium, looking through a folder of documents.
Two school staff members stand in empty school atrium, looking through a folder of documents.

References

These references cover the entire toolkit, not just this section.

  1. Posner, K., Brown, G. K., Stanley, B., Brent, D. A., Yershova, K. V., Oquendo, M. A., Currier, G. W., Melvin, G. A., Greenhill, L., Shen, S., & Mann, J. J. (2011). The Columbia–suicide severity rating scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. American Journal of Psychiatry, 168(12), 1266–1277. https://doi.org/10.1176/appi.ajp.2011.10111704
  2. Erbacher, T., Singer, J., & Poland, S. (2014). Suicide in schools: A practitioner’s guide to multi-level prevention, assessment, intervention, and postvention (1st ed.). Routledge. https://www.taylorfrancis.com/books/mono/10.4324/9780203702970/suicide-schools-terri-erbacher-jonathan-singer-scott-poland
  3. Preventing suicide: Guidelines for administrators and crisis teams. (n.d.). National Association of School Psychologists (NASP). Retrieved January 31, 2022, from https://www.nasponline.org/resources-and-publications/resources-and-podcasts/school-safety-and-crisis/mental-health-resources/preventing-youth-suicide/preventing-suicide-guidelines-for-administrators-and-crisis-teams
  4. The school counselor and suicide prevention/awareness—American School Counselor Association (ASCA). (n.d.). Retrieved January 31, 2022, from https://www.schoolcounselor.org/Standards-Positions/Position-Statements/ASCA-Position-Statements/The-School-Counselor-and-Suicide-Prevention-Awaren
  5. Wang, K., Chen, Y., Zhang, J., & Oudekerk, B. (2020). Indicators of school crime and safety: 2019 (NCES 2020063). National Center for Education Statistics. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2020063
  6. Nekvasil, E. K., & Cornell, D. G. (2012). Student reports of peer threats of violence: Prevalence and outcomes. Journal of School Violence, 11(4), 357–375. https://doi.org/10.1080/15388220.2012.706764
  7. Modzeleski, W., & Randazzo, M. R. (2018). School threat assessment in the USA: Lessons learned from 15 years of teaching and using the federal model to prevent school shootings. Contemporary School Psychology, 22(2), 109–115. https://doi.org/10.1007/s40688-018-0188-8
  8. O’Toole, M. E. (1999). The school shooter: A threat assessment perspective. For full text:
    http://www.https://eric.ed.gov/?id=ED446352
  9. Fein, R. A., Vossekuil, B., Pollack, W. S., Borum, R., Modzeleski, W., & Reddy, M. (2004). Threat assessment in schools: A guide to managing threatening situations and to creating safe school climates. U.S. Department of Education. https://eric.ed.gov/?id=ED515943
  10. National Threat Assessment Center. (2018). Enhancing school safety using a threat assessment model: An operational guide for preventing targeted school violence. CISA. U.S. Secret Service, Department of Homeland Security. https://www.cisa.gov/publication/enhancing-school-safety-using-threat-assessment-model-operational-guide-preventing
  11. National Threat Assessment Center. (2019). Protecting America’s schools: A U.S. Secret Service analysis of targeted school violence (p. 68). U.S. Secret Service, Department of Homeland Security. https://www.secretservice.gov/sites/default/files/2020-04/Protecting_Americas_Schools.pdf
  12. Alathari, L., Drysdale, D., Driscoll, S., Ed, M., Blair, A., Carlock, A., Cotkin, A., Johnston, B., Foley, C., Mauldin, D., McGarry, J., & Nemet, J. (n.d.). This report was authored by the staff of the U.S. Secret Service National Threat Assessment Center (NTAC). 68.
  13. Silver, J., Simons, A., & Craun, S. (2018). A study of pre-attack behaviors of active shooters in the United States between 2000 and 2013 [File]. Federal Bureau of Investigation. https://www.fbi.gov/file-repository/pre-attack-behaviors-of-active-shooters-in-us-2000-2013.pdf/view
  14. Meloy, J. R., & Hoffmann, J. (2013). International handbook of threat assessment. Oxford University Press.
  15. Woitaszewski, S., Crepeau-Hobson, F., Conolly, C., & Cruz, M. (2018). Rules, requirements, and resources for school-based threat assessment: A fifty state analysis. Contemporary School Psychology, 22(2), 125–134. https://doi.org/10.1007/s40688-017-0161-y
  16. Wang, K, Kemp, J., & Burr, R. (2022). Crime, violence, discipline, and safety in U.S. public schools in 2019–20: Findings from the school survey on crime and safety (NCES 2022-029). U.S. Department of Education. https://nces.ed.gov/pubs2022/2022029.pdf
  17. Holland, K. M., Hall, J. E., Wang, J., Gaylor, E. M., Johnson, L. L., D. Shelby, Simon, T., & School-Associated Violent Deaths Study Group. (2019). Characteristics of school-associated youth homicides. Morbidity and Mortality Weekly Report, 68(3), 53–60.
  18. Cornell, D. G. (2015). Our schools are safe: Challenging the misperception that schools are dangerous places. American Journal of Orthopsychiatry, 85(3), 217–220. https://doi.org/10.1037/ort0000064
  19. Mann, A., Whitaker, A., Torres-Gullien, S., Morton, M., Jordan, H., Coyle, S., & Sun, W.-L. (2019). Cops & no counselors: How the lack of school mental health staff is harming students. Showcase of Faculty Scholarly & Creative Activity. https://digitalcommons.unf.edu/facultyshowcase/2019/showcase/11
  20. Langman, P., & Straub, F. (2019). A comparison of averted and completed school attacks from the Police Foundation Averted School Violence Database. Washington, DC. Office of Community Oriented Policing Services.
  21. American Psychological Association Zero Tolerance Task Force. (2008). Are zero tolerance policies effective in the schools?: An evidentiary review and recommendations. American Psychologist, 63(9), 852–862. https://doi.org/10.1037/0003-066X.63.9.852
  22. Losen, D. J., & Martinez, P. (2020). Lost opportunities: How disparate school discipline continues to drive differences in the opportunity to learn. N.A.(n.a.). https://escholarship.org/uc/item/7hm2456z
  23. Fabelo, T., Thompson, M., Plotkin, M., Carmichael, D., Marchbanks III, M.P., & Booth, E. (2011). Breaking school rules: A statewide study of how school discipline relates to students’ success and juvenile justice involvement. Council of State Governments Justice Center.
  24. Morgan, E., Salomon, N., Plotkin, M., & Cohen, R. (2014). School discipline consensus report: Strategies from the field to keep students engaged in school and out of the juvenile justice system. https://www.ojp.gov/library/publications/school-discipline-consensus-report-strategies-field-keep-students-engaged
  25. Maeng, J. L., Cornell, D., & Huang, F. (2020). Student threat assessment as an alternative to exclusionary discipline. Journal of School Violence, 19(3), 377–388. https://doi.org/10.1080/15388220.2019.1707682
  26. Louvar Reeves, M. A., & Brock, S. E. (2018). School behavioral threat assessment and management. Contemporary School Psychology, 22(2), 148–162. https://doi.org/10.1007/s40688-017-0158-6
  27. Cornell, D. G., Sheras, P., Gregory, A., & Fan, X. (2009). A retrospective study of school safety conditions in high schools using the Virginia threat assessment guidelines versus alternative approaches. School Psychology Quarterly, 24(2), 119–129. https://doi.org/10.1080/15388220.2019.1707682
  28. Cornell, D. G., Gregory, A., & Fan, X. (2011). Reductions in long-term suspensions following adoption of the Virginia Student Threat Assessment Guidelines. NASSP Bulletin, 95(3), 175–194. https://doi.org/10.1177/0192636511415255
  29. Cornell, D. G., Sheras, P. L., Kaplan, S., McConville, D., Douglass, J., Elkon, A., & McKnight, L. (2004). Guidelines for student threat assessment: Field-test finding. School Psychology Review, 33(4), 527–554.
  30. Cornell, D., & JustChildren. (2013). Prevention v. Punishment: Threat assessment, school suspensions, and racial disparities. University of Virginia.
  31. Strong, K., & Cornell, D. (2008). Student threat assessment in Memphis city schools: A descriptive report. Behavioral Disorders, 34(1), 42–54. https://doi.org/10.1177/019874290803400104
  32. Allen, K., Cornell, D., Lorek, E., & Sheras, P. (2008). Response of school personnel to a student threat assessment training. School Effectiveness and School Improvement, 19(3), 319–332. https://doi.org/10.1080/09243450802332184
  33. Cornell, D. G. (2011). A developmental perspective on the Virginia student threat assessment guidelines. New Directions for Youth Development, 129, 43–59. https://doi.org/10.1002/yd.386
  34. Stohlman, S., Konold, T., & Cornell, D. (2020). Evaluation of threat assessment training for school personnel. Journal of Threat Assessment and Management, 7(1–2), 29–40. https://doi.org/10.1037/tam0000142
  35. Civil Rights Data Collection. (n.d.). Retrieved July 27, 2021, from https://ocrdata.ed.gov/estimations/2017-2018
  36. Cornell, D. G., Maeng, J., Shukla, K., & Konold, T. (2018). Racial/ethnic parity in disciplinary consequences using student threat assessment. School Psychology Review, 47(2), 183–195. https://doi.org/10.17105/SPR-2017-0030.V47-2
  37. Cornell, D., & Lovegrove, P. (2015). Student threat assessment as a method for reducing student suspensions. In Race and Gender Disparities in School Discipline (pp. 180–191). Teachers College Press. https://escholarship.org/uc/item/18h2929c
  38. Maeng, J. L., Cornell, D. G., Kerere, J., Huang, F., Konold, T., & Afolabi, K. (2023). School threat assessment in Florida: Technical report of 2021-2022 case data.
  39. Miller, C., & Meyers, S. (2015). Disparities in school discipline practices for students with emotional and learning disabilities and autism. Journal of Education and Human Development, 4. https://doi.org/10.15640/jehd.v4n1a23
  40. Sullivan, A. L., Klingbeil, D. A., & Norman, E. R. V. (2013). Beyond behavior: Multilevel analysis of the influence of sociodemographics and school characteristics on students’ risk of suspension. School Psychology Review, 42(1), 99–114. https://doi.org/10.1080/02796015.2013.12087493
  41. Kaplan, S. G., & Cornell, D. G. (2005). Threats of violence by students in special education. Behavioral Disorders, 31(1), 107–119. https://doi.org/10.1177/019874290503100102
  42. Crepeau-Hobson, F., & Leech, N. (2021). Disciplinary and nondisciplinary outcomes of school-based threat assessment in Colorado schools. School Psychology Review 51(5), 609-618. https://doi.org/10.1080/2372966X.2020.1842716
  43. Amman, M., Bowlin, M., Buckles, L., Burton, K. C., Brunell, K. F., Gibson, K. A., Griffin, S. H., Kennedy, K., & Robins, C. J. (2017). Making prevention a reality: Identifying, assessing, and managing the threat of targeted attacks. U.S. Department of Justice. https://www.fbi.gov/file-repository/making-prevention-a-reality.pdf/view
  44. Calhoun, F. S., & Weston, S. W. (2021). All Threat, All Reporting Criteria.
  45. Cornell, D. (2018). Comprehensive school threat assessment guidelines. School Threat Assessment Consultants LLC.
  46. Colorado School Safety Resource Center, Department of Public Safety. (2020). Essentials of school threat assessment: Preventing targeted school violence. https://cdpsdocs.state.co.us/safeschools/CSSRC%20Documents/CSSRC_Essentials_of_TA_2020.pdf
  47. Kelly, S. R. (2018). The school psychologist’s role in leading multidisciplinary school-based threat assessment teams. Contemporary School Psychology, 22(2), 163–173. https://doi.org/10.1007/s40688-017-0153-y
  48. U.S. Department of Education, Privacy Technical Assistance Center. (2019). School Resource Officers, School Law Enforcement Units, and the Family Educational Rights and Privacy Act (FERPA). https://files.eric.ed.gov/fulltext/ED595430.pdf
  49. Cornell, D., & Maeng, J. (2020). National Center for School Safety Initial Survey of School Threat Assessment Experts. 15.
  50. Behavior Threat Assessment and Management (BTAM) Best Practice Considerations for K–12 Schools. (n.d.). National Association of School Psychologists (NASP). Retrieved January 31, 2022, from https://www.nasponline.org/resources-and-publications/resources-and-podcasts/school-safety-and-crisis/systems-level-prevention/threat-assessment-at-school/behavior-threat-assessment-and-management-(btam)-best-practice-considerations-for-k%E2%80%9312-schools
  51. Reeves, M., & McCarthy, C. (2021). Upholding student civil rights and preventing disproportionality in behavioral threat assessment and management (BTAM). National Association of School Psychologists. http://www.nasponline.org/btam-sped
  52. Averting targeted school violence: A U.S. secret service analysis of plots against schools | United States Secret Service. (n.d.). Retrieved January 31, 2022, from https://www.secretservice.gov/newsroom/reports/threat-assessments/schoolcampus-attacks/details-0
  53. Mulvey, E. P., & Cauffman, E. (2001). The inherent limits of predicting school violence. American Psychologist, 56(10), 797–802. https://doi.org/10.1037/0003-066X.56.10.797
  54. Fox, B., Heide, K., Khachatryan, N., Michel, C., & Cochran, J. (2021). Juveniles arrested for murder: A latent class analysis of male offenders. Behavioral Sciences & the Law, 39(4), 470–491. https://doi.org/10.1002/bsl.2531
  55. Cornell, D. G., Miller, C., & Benedek, E. P. (1988). MMPI profiles of adolescents charged with homicide. Behavioral Sciences & the Law, 6, 401.
  56. Cornell, D. G. (1990). Prior adjustment of violent juvenile offenders. Law and Human Behavior, 14(6), 569–577. https://doi.org/10.1007/BF01044882
  57. Toupin, J. (n.d.). Adolescent murderers: Validation of a typology and study of their recidivism in: A. V. Wilson (Ed.), Homicide: The victim/offender connection (pp. 135-156).
  58. Cornell, D. G., Benedek, E. P., & Benedek, D. M. (1987). Juvenile homicide: Prior adjustment and a proposed typology. The American Journal of Orthopsychiatry, 57(3), 383–393. https://doi.org/10.1111/j.1939-0025.1987.tb03547.x
  59. Langman, P. (2009). Rampage school shooters: A typology. Aggression and Violent Behavior, 14(1), 79–86. https://doi.org/10.1016/j.avb.2008.10.003
  60. Kazdin, A. E. (2015). Psychosocial treatments for conduct disorder in children and adolescents. In a guide to treatments that work, 4th ed (pp. 141–173). Oxford University Press.
  61. Van der Kolk, B. A. (2014). The body keeps the score: Brain, mind, and body in the healing of trauma. Viking.
  62. Wamser-Nanney, R., Nanney, J. T., Conrad, E., & Constans, J. I. (2019). Childhood trauma exposure and gun violence risk factors among victims of gun violence. Psychological Trauma: Theory, Research, Practice and Policy, 11(1), 99–106. https://doi.org/10.1037/tra0000410
  63. Overstreet, S., & Chafouleas, S. M. (2016). Trauma-informed schools: Introduction to the special issue. School Mental Health, 8(1), 1–6. https://doi.org/10.1007/s12310-016-9184-1
  64. Calhoun, F. S., & Weston, S. W. (2003). Contemporary threat management: A practical guide for identifying, assessing, and managing individuals of violent intent. Specialized Training Services.
  65. Langman, P. (2021). Warning Signs: Identifying School Shooters Before They Strike | School Shooters .info. https://schoolshooters.info/warning-signs-book
  66. Woodward, W., & Goodrum, S. (n.d.). Report on the Arapahoe High School Shooting. 141.
  67. Meloy, J. R. (2021). International Handbook of Threat Assessment (J. Hoffmann, Ed.; 2nd edition). Oxford University Press.
  68. Pollack, W. S., Modzeleski, W., & Rooney, G. (2008). Prior knowledge of potential school-based violence: Information students learn may prevent a targeted attack. In U.S. Department of Education (p. 15). U.S. Secret Service and US Department of Education. https://eric.ed.gov/?id=ED511645
  69. Planty, M., Banks, D. C., Lindquist, C. H., Cartwright, J. K., & Witwer, A. R. (2020). Tip lines for school safety: A national portrait of tip line use. https://www.rti.org/publication/tip-lines-school-safety
  70. Stohlman, S. L., & Cornell, D. G. (2019). An online educational program to increase student understanding of threat assessment. The Journal of School Health, 89(11), 899–906. https://doi.org/10.1111/josh.12827
  71. Madfis, E. (2020). How to stop school rampage killing: Lessons from averted mass shootings and bombings. https://doi.org/10.1007/978-3-030-37181-4
  72. Stallings, R., & Hall, J. C. (2019). Averted targeted school killings from 1900-2016. Criminal Justice Studies, 32(3), 222–238. https://doi.org/10.1080/1478601X.2019.1618296
  73. Singh, J. P., Grann, M., & Fazel, S. (2011). A comparative study of violence risk assessment tools: A systematic review and metaregression analysis of 68 studies involving 25,980 participants. Clinical Psychology Review, 31(3), 499–513. https://doi.org/10.1016/j.cpr.2010.11.009
  74. Meloy, J. R., & O’Toole, M. E. (2011). The Concept of Leakage in Threat Assessment: The concept of leakage in threat assessment. Behavioral Sciences & the Law, 29(4), 513–527. https://doi.org/10.1002/bsl.986
  75. Millspaugh, S., Cornell, D., Huang, F., & Datta, P. (2015). Prevalence of aggressive attitudes and willingness to report threats in middle school. Journal of Threat Assessment and Management, 2, 11–22. https://doi.org/10.1037/tam0000031
  76. Perkins, J. M., Perkins, H. W., & Craig, D. W. (2020). Norms and attitudes about being an active bystander: Support for telling adults about seeing knives or guns at school among greater London youth. Journal of Youth and Adolescence, 49(4), 849–868. https://doi.org/10.1007/s10964-019-01127-7
  77. Crichlow-Ball, C., & Cornell, D. (2019, August). Does School Climate Facilitate Student Threat Reporting? [Poster]. 2019 American Psychological Association National Convention, Chicago, IL. https://education.virginia.edu/sites/default/files/images/YVP/Crichlow-Ball%20APA%202019%20Does%20School%20Climate%20Facilitate%20Student%20Threat%20Reporting.pdf
  78. Cameron, K. (2018). VTRA – North American Center for Threat Assessment and Trauma Response. Violence Threat Risk Assessment (VTRA). https://www.nactatr.com/vtra.html
  79. Leuschner, V., Schroer-Hippel, M., Bondü, R., & Scheithauer, H. (2012). Indicated prevention of severe targeted school violence: NETWorks Against School Shootings (NETWASS). In School Shootings (pp. 401–420). Springer. https://doi.org/10.1007/978-1-4614-5526-4_18
  80. Van Dreal, J. (Ed.). (2016). Assessing student threats: Implementing the salem-keizer system, 2nd Edition. Rowman and Littlefield Publishers. https://rowman.com/ISBN/9781475830507/Assessing-Student-Threats-Implementing-the-Salem-Keizer-System-2nd-Edition
  81. Oksanen, A., Kaltiala-Heino, R., Holkeri, E., & Lindberg, N. (2015). School shooting threats as a national phenomenon: Comparison of police reports and psychiatric reports in Finland. Journal of Scandinavian Studies in Criminology and Crime Prevention, 16(2), 145–159.
    https://doi.org/10.1080/14043858.2015.1101823
  82. Wisconsin Department of Justice Office of School Safety. (2019). Wisconsin school threat assessment protocol: A guide for school personnel and law enforcement officers. Wisconsin Department of Justice. https://www.doj.state.wi.us/sites/default/files/school-safety/WI_School_Threat_Assessment_Protocol.pdf
  83. South. (2020). School-based behavioral threat assessment & management: Best practices guide for South Carolina K-12 schools. State of South Carolina Department of Education.
  84. Deisinger, G. (2020). Threat assessment and management in Virginia public schools: Model policies, procedures, and guidelines. Virginia Department of Criminal Justice Services: Virginia Center for School and Campus. https://www.dcjs.virginia.gov/sites/dcjs.virginia.gov/files/publications/law-enforcement/threat-assessment-model-policies-procedures-and-guidelinespdf_0.pdf
  85. Hall, C. M., Bertuccio, R. F., Mazer, T. M., & Tawiah, C. O. (2020). Google kt: A component analysis of free online violent threat assessment tools for schools. The Rural Educator, 41(1), 21. https://doi.org/10.35608/ruraled.v41i1.680
  86. Cornell, D. G., Allen, K., & Fan, X. (2012). A randomized controlled study of the Virginia Student Threat Assessment Guidelines in kindergarten through grade 12. School Psychology Review, 41(1), 100–115. https://doi.org/10.1080/02796015.2012.12087378
  87. Nekvasil, E. K., & Cornell, D. G. (2015). Student threat assessment associated with safety in middle schools. Journal of Threat Assessment and Management, 2(2), 98–113. https://doi.org/10.1037/tam0000038
  88. Cornell, D. G., Maeng, J. L., Burnette, A. G., Jia, Y., Huang, F., Konold, T., Datta, P., Malone, M., & Meyer, P. (2017). Student threat assessment as a standard school safety practice: Results from a statewide implementation study. School Psychology Quarterly, 33(2), 213–222. https://doi.org/10.1037/spq0000220
  89. Leuschner, V., Fiedler, N., Schultze, M., Ahlig, N., Göbel, K., Sommer, F., Scholl, J., Cornell, D., & Scheithauer, H. (2017). Prevention of targeted school violence by responding to students’ psychosocial crises: The NETWASS program. Child Development, 88(1), 68–82. https://doi.org/10.1111/cdev.12690
  90. Goss v. Lopez, 419 U.S. 565 (1975). (n.d.). Justia Law. Retrieved October 6, 2021, from https://supreme.justia.com/cases/federal/us/419/565/
  91. Alexander, K., & Alexander, M. D. (2018). Alexander and Alexander’s the law of schools, students and teachers in a nutshell, 6th. West Academic Publishing. https://www.westacademic.com/Alexander-and-Alexanders-The-Law-of-Schools-Students-and-Teachers-in-a-Nutshell-6th-9781640204249
  92. Protecting Student Privacy. (n.d.). Does FERPA permit the sharing of education records with outside law enforcement officials, mental health officials, and other experts in the community who serve on a school’s threat assessment team? Retrieved February 1, 2022, from https://studentprivacy.ed.gov/faq/does-ferpa-permit-sharing-education-records-outside-law-enforcement-officials-mental-health
  93. Rights (OCR), O. for C. (2008, May 7). Summary of the HIPAA Privacy Rule [Text]. HHS.Gov. https://www.hhs.gov/hipaa/for-professionals/privacy/laws-regulations/index.html
  94. Alder, S. (2022, January 9). Does HIPAA apply to schools? HIPAA Journal. https://www.hipaajournal.com/does-hipaa-apply-to-schools/
  95. Johnson, R., Persad, G., & Sisti, D. (2014). The Tarasoff rule: The implications of interstate variation and gaps in professional training. The Journal of the American Academy of Psychiatry and the Law, 42(4), 469–477.
  96. Johnson, R., Persad, G., & Sisti, D. (2019). The Tarasoff rule: The implications of interstate variation and Gaps in Professional Training. FOCUS, 17(4), 435–442. https://doi.org/10.1176/appi.focus.17402
  97. Protecting Student Privacy. (n.d.). FERPA. Retrieved August 19, 2022, from https://studentprivacy.ed.gov/ferpa
  98. 20-255 Mahanoy Area School Dist. V. B. L. (06/23/2021). (2021). https://www.supremecourt.gov/opinions/20pdf/20-255_g3bi.pdf
  99. Borum, R., Cornell, D. G., Modzeleski, W., & Jimerson, S. R. (2010). What can be done about school shootings?: A review of the evidence. Educational Researcher, 39(1), 27–37. https://doi.org/10.3102/0013189X09357620
  100. Cornell, D. G., Mayer, M. J., & Sulkowski, M. L. (2021). History and future of school safety research. School Psychology Review, 50(2–3), 143–157. https://doi.org/10.1080/2372966X.2020.1857212
  101. Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, J., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2(1), 40. https://doi.org/10.1186/1748-5908-2-40
  102. National Implementation Research Network. (n.d.). Module 4: Implementation Stages. Frank Porter Graham Child Development Institute. https://nirn.fpg.unc.edu/ module-1/implementation-stages
  103. Cornell, D. G., & Maeng, J. L. (2021). School threat assessment experts training and practice standards survey. Charlottesville, VA: School of Education and Human Development.
  104. Reeves, M., Kanan, L. M., & Plog, A. E. (2010). Comprehensive Planning for Safe Learning Environments: A School Professional’s Guide to Integrating Physical and Psychological Safety – Prevention through Recovery. Routledge. https://www.routledge.com/Comprehensive-Planning-for-Safe-Learning-Environments-A-School-Professionals/Reeves-Kanan-Plog/p/book/9780415998352
  105. Crosse, S., Williams, B., Hagen, C. A., Harmon, M., Ristow, L., DiGaetano, R., Broene, P., Alexander, D., & Tseng, M. (2011). Prevalence and Implementation Fidelity of Research-Based Prevention Programs in Public Schools: Final Report.
  106. Fixsen, D., Naoom, S., Blase, K., Friedman, R., & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. The National Implementation Research Network. https://nirn.fpg.unc.edu/resources/implementation-research-synthesis-literature
  107. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41(3–4), 327–350. https://doi.org/10.1007/s10464-008-9165-0
  108. Gottfredson, D. C., & Gottfredson, G. D. (2002). Quality of school-based prevention programs: Results from a national survey. Journal of Research in Crime and Delinquency, 39(1), 3–35. https://doi.org/10.1177/002242780203900101
  109. Payne, A. A., Gottfredson, D. C., & Gottfredson, G. D. (2006). School predictors of the intensity of implementation of school-based prevention programs: Results from a national study. Prevention Science: The Official Journal of the Society for Prevention Research, 7(2), 225–237. https://doi.org/10.1007/s11121-006-0029-2
  110. National Implementation Research Network. (n.d.). Module 2: Implementation Drivers. Retrieved September 9, 2022, from https://implementation.fpg.unc.edu/wp-content/uploads/Implementation-Drivers-Overview.pdf
  111. CHDS School Shooting Safety Compendium. (n.d.). CHDS School Shooting Safety Compendium. Retrieved September 14, 2022, from https://www.chds.us/ssdb/
  112. Reporting on guns and gun violence in America. (n.d.). The Trace. Retrieved November 19, 2021, from https://www.thetrace.org/
  113. Graham, S., Bellmore, A. D., & Mize, J. (2006). Peer victimization, aggression, and their co-occurrence in middle school: Pathways to adjustment problems. Journal of Abnormal Child Psychology, 34(3), 363–378. https://doi.org/10.1007/s10802-006-9030-2
  114. Juvonen, J., Nishina, A., & Graham, S. (2000). Peer harassment, psychological adjustment, and school functioning in early adolescence. Journal of Educational Psychology, 92(2), 349–359. https://doi.org/10.1037/0022-0663.92.2.349
  115. Cornell, D. G., Gregory, A., Huang, F., & Fan, X. (2008). Virginia High School Safety Study. University of Virginia, Youth Violence Project, 132.
  116. Yang, C., Manchanda, S., Lin, X., & Teng, Z. (2021). An intersectional examination of the effects of race/ethnicity and immigrant status on school victimization in predominantly hispanic/latinx high schools. School Psychology Review, 50(2–3), 303–315. https://doi.org/10.1080/2372966X.2020.1840262
  117. Ingersoll, R. M. (2002). The teacher shortage: A case of wrong diagnosis and wrong prescription. NASSP Bulletin, 86(631), 16–31. https://doi.org/10.1177/019263650208663103
  118. Ronfeldt, M., Loeb, S., & Wyckoff, J. (2013). How teacher turnover harms student achievement. American Educational Research Journal, 50(1), 4–36. https://doi.org/10.3102/0002831212463813
  119. Sutcher, L., Darling-Hammond, L., & Carver-Thomas, D. (n.d.). A Coming Crisis in Teaching? 107.
  120. Gage, N. A., Larson, A., Sugai, G., & Chafouleas, S. M. (2016). Student Perceptions of School Climate as Predictors of Office Discipline Referrals. American Educational Research Journal, 53(3), 492–515. https://doi.org/10.3102/0002831216637349
  121. Loukas, A., & Robinson, S. (2004). Examining the Moderating Role of Perceived School Climate in Early Adolescent Adjustment. Journal of Research on Adolescence, 14(2), 209–233. https://doi.org/10.1111/j.1532-7795.2004.01402004.x
  122. Thapa, A., Cohen, J., Guffey, S., & Higgins-D’Alessandro, A. (2013). A Review of School Climate Research. Review of Educational Research, 83(3), 357–385. https://doi.org/10.3102/0034654313483907
  123. Wang, M.-T., & Degol, J. L. (2016). School Climate: A Review of the Construct, Measurement, and Impact on Student Outcomes. Educational Psychology Review, 28(2), 315–352. https://doi.org/10.1007/s10648-015-9319-1
  124. Cornell, D. G., Huang, F. L.,Konold, T. R., Shukla, K., Malone, M., Datta, P., Jia, Y., Stohlman, S., Burnette, A. G., & Meyer III, J. P. (2017). Development of a Standard Model for School Climate and Safety Assessment. (n.d.). Final Report, 147.
  125. Noltemeyer, A. L., Ward, R. M., & Mcloughlin, C. (2015). Relationship Between School Suspension and Student Outcomes: A Meta-Analysis. School Psychology Review, 44(2), 224–240. https://doi.org/10.17105/spr-14-0008.1
  126. Lee, T., Cornell, D., Gregory, A., & Fan, X. (2011). High Suspension Schools and Dropout Rates for Black and White Students. Education and Treatment of Children, 34(2), 167–192. https://doi.org/10.1353/etc.2011.0014
  127. Gregory, A., Skiba, R. J., & Noguera, P. A. (2010). The achievement gap and the discipline gap: Two sides of the same coin? Educational Researcher, 39(1), 59–68. https://doi.org/10.3102/0013189X09357621
  128. Burnette, A. G., Datta, P., & Cornell, D. (2018). The distinction between transient and substantive student threats. Journal of Threat Assessment and Management, 5(1), 4–20. https://doi.org/10.1037/tam0000092
  129. Vossekuil, B., Fein, R. A., Reddy, M., Borum, R., & Modzeleski, W. (2002). Final report and findings of the safe school initiative: Implications for the prevention of school attacks in the United States | Office of Justice Programs (195287; p. 63). U.S. Department of Education and United States Secret Service Safe School Initiative. https://www.ojp.gov/ncjrs/virtual-library/abstracts/final-report-and-findings-safe-school-initiative-implications