School‐based interventions for reducing disciplinary school exclusion: a systematic review

Authors: Sara Valdebenito, Manuel Eisner, David P. Farrington, Maria M. Ttofi, Alex Sutherland

License: © 2018 Valdebenito et al. CC BY 4.0 This is an open access article under the terms of the http://creativecommons.org/licenses/by/3.0/ License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Article links: DOI: 10.4073/csr.2018.1  |  PubMed: 37131379  |  PMC: PMC8533648

Relevance: Moderate: mentioned 3+ times in text

Full text: PDF (4.9 MB)

1. Background

1.1 THE PROBLEM, CONDITION OR ISSUE

1.1.1 School discipline

Discipline problems are frequent in schools and they may have a harmful effect on pupils’ learning outcomes. A lack of discipline and the subsequent potential increase in school disorder (e.g. bullying, substance misuse) can seriously threaten the quality of instruction that teachers provide, hamper pupils acquisition of academic skills and subsequently reduce their attachment to the education system (ref. Gottfredson, Cook, & Na, 2012).

As such, discipline represents a serious concern for parents and teachers, demanding significant efforts and resources from schools (ref. Kaplan, Gheen, & Midgley, 2002). The PISA 2009 report (OECD, 2010) stated that schools registering higher levels of disciplinary problems result in teachers spending less time on learning in order to deal with such issues. In its 2012 version, the PISA report asked students about school discipline. Results found that “28% of students reported that teachers had to wait a long time to quiet down every class, or almost all classes”(OECD, 2013). Being more precise, the Teaching and Learning International Survey (TALIS) revealed that teachers spend at least 20% of lesson time dealing with disruption and administrative tasks. In the United Kingdom, the Behaviour Survey 2010 states that 80% of school teachers felt their ability to teach effectively was impaired by students’ poor behaviour (ref. Massey, 2011). On a global level, evidence suggests that 13% of teachers’ time is, on average, spent maintaining order (OECD, 2009).

Schools use different procedures to manage discipline, including a range of punitive responses (e.g., loss of privileges, additional homework or detention). Among these, exclusion is normally seen as one of the most serious punishments. Although types and lengths vary from country to country, school exclusion (also known as school suspension in the United States)¹ can be broadly defined as a disciplinary sanction imposed in reaction to students’ behaviour (i.e. violations of school policies) by the responsible authority. In concrete terms, exclusion entails a removal from regular teaching for a period of time during which students are also not allowed to be present on school premises. Specifically, fixed‐term exclusions consist of a limited number of hours or days (ref. Cornell, Gregory, & Fan, 2011), whereas permanent exclusion (i.e., expulsion) involves the pupil being transferred to a different school, or educated outside of the regular education system (ref. Spink, 2011; ref. Webb & Vulliamy, 2004).

Even if school policies suggest that exclusion should be used as a last resort, reserved for only the most serious and persistent offences (ref. Gregory & Weinstein, 2008a; Skiba & Peterson, 1999; ref. Skiba, Trachok, Chung, Baker, & Hughes, 2012), research evidence suggests that minor offences can also provoke this type of punishment (ref. Munn, Cullen, Johnstone, & Lloyd, 2001; ref. Skiba, 2014). ref. Fenning et al., (2012) provide a case in point: their research concluded that suspension and expulsion were the most common types of punishment for minor problems such astardiness and school truancy. These findings were also confirmed by ref. Liu (2013) who found that 48% of suspensions lasting a maximum of five days targeted minor disorder or disruptive behaviours.

In terms of prevalence, data provided by the UK Department for Education (academic year 2011/12) shows that in England fixed‐term exclusion affects 3.5% of the school population whereas permanent exclusion applies to only 0.06%. The national figures suggest that students in secondary‐level education (6.8% of the school population) as well as those in special education (14.7%) are the most likely to experience fixed‐term exclusion (DfE, 2013). In the United States, data provided by the Department of Education (academic year 2011/12) concluded that 7.4% (3.5 million) of students were suspended in school, 7% (3.45 million) were suspended out‐of‐school, and less than one per cent were subject to expulsion (around 130,000 students). Black students and those presenting disabilities are, respectively, three and two times more likely to be excluded compared to White and non‐disabled pupils (U.S. Department of Education, 2012).

International comparisons of exclusion prevalence rates are not available in the literature examined. Indeed, differences in use, extent and recording (i.e., unreported exclusions) make an international estimation challenging. In Table 1, the reader will find information regarding the use of exclusion in a sample of high‐ and middle‐income countries. The information is limited to a convenience sample involving twelve different cases to allow an overview of i) the types of exclusion used in these countries, ii) the length of the sanction, iii) the authority responsible for determining this sanction, iv) the behaviours for which school exclusion is permissible, and, in cases where information was available, the table also includes v) the local prevalence of exclusion. This does not claim to be a representative sample of all countries, but as an initial approach will help provide a more complex picture of the phenomenon. In addition, this comparison was intended to help with searches for studies that could be potentially included in the systematic review. For instance, by comparing exclusion in different countries, it was found that the same school sanction had different names in different countries (e.g., “stand‐down” in New Zealand, “exclusion” in the UK and “suspension” in the US).

The comparative data incorporated in the table above suggests heterogeneity in the application of exclusion. For instance, in the US, Norway and England, educational systems distinguish between fixed and permanent exclusion. However, in some educational systems, such as Finland’s, the law only permits fixed‐term exclusion. Concerning length, England limits fixed‐term exclusions to a maximum of 45 days per school year while New Zealand’s legislation allows exclusions for a maximum of 10 days per year. On the other hand, it is important to note that in some countries – such as France – specific laws define and regulate exclusion, whereas in others – like Chile and Colombia – the ability to set the length of the sanction is granted to each school.

Although the data on prevalence is limited to a few countries, the percentage of in‐school exclusion seems to be larger than out‐of‐school exclusion and expulsion. In New Zealand, the data suggests that the use of exclusion is marginal particularly when compared with some areas in the US and the UK.

1.2 PREDICTORS AND OUTCOMES

The research concerning predictors and outcomes of school exclusion has some limitations it is necessary to address before arriving at any final conclusions. Regarding predictors, only ethnicity seems to have a clear role in predicting exclusion. For other variables of interest such as sex, age or socio‐economic status most of the evidence is limited to bivariate associations.

Regarding the outcomes, while there is a stark link between misbehaviour (e.g., school drop‐out and delinquency) and school exclusion, there is no clear causal relationship. Notwithstanding decades of research on school exclusion and its impact on later behaviour, we are still at an initial stage for testing causal associations in these matters. The association between exclusion and these negative outcomes may simply reflect underlying behavioural tendencies that lead to conduct problems, exclusion and poor outcomes later in life – that is, the antisocial syndrome described by ref. Farrington (1997). In fact, school exclusion and the behaviours outlined here as “negatives” could be explained by the personality traits of the syndrome.

As stated by ref. Sutherland and Eisner (2014) “it is currently unclear whether the disciplinary action itself has a causal effect over and beyond the social, familial and behavioural characteristics of the affected children. To date, studies have used analytical approaches that are unable to reliably establish a robust link between exclusion and outcomes such as criminal behavior.” Some longitudinal studies have attempted to deal with this matter by controlling for previous behavioural characteristics that could alter the impact of the sanction. When this is the case, the methodological details are explicitly presented in this review.

Keeping these reservations in mind, the following section describes variables associated with the prediction of school exclusion, as well as some negative outcomes linked to exclusion.

1.2.1 Predictors of school exclusion

From a normative point of view, school exclusion is a punitive response for misbehaviour. In that sense, behavioural problems seem to be the most obvious empirical predictor for exclusion. ref. Reinke, Herman, Petras, & Ialongo, (2008) illustrate the role of problem behaviour in exclusion by conducting a latent class analysis. Participants in the subclass of boys exhibiting behavioural problems only (i.e., isolating other academic/learning difficulties) were almost 4 times more likely to be suspended (OR = 3.42; 95%CI 1.36 to 8.58; p < .05) than their non‐problematic peers. Similarly, ref. Pas, Bradshaw, Hershfeldt, & Leaf, (2010) found that after controlling for student, teacher, classroom, and school level covariates, the strongest predictor for out‐of‐school suspension was disruptive behaviour (OR = 4.83; 95%CI 4.10 to 5.68; p < .05).

Despite the role of behaviour in school exclusion, research suggests that it is not the sole or even the most prominent predictor. In fact, previous findings show a more complex scenario where exclusion is also strongly predicted by gender, ethnicity, age, economic background, and special educational needs (ref. Costenbader & Markson, 1998a; Mcloughlin & Noltemeyer, 2010; ref. Monroe, 2005; ref. Nickerson & Spears, 2007; ref. Noltemeyer & Mcloughlin, 2010; ref. Skiba et al., 2011; ref. Yudof, 1975). In the following paragraphs, we offer an overview of the role of these variables in predicting school exclusion.

Gender as a predictor of exclusion

Data provided by the Department for Education in England (DfE) 2011/12 suggests that male pupils are around three times more likely to be punished by exclusion than female pupils (DfE, 2013). The same trend can be observed in the study published by ref. Liu (2013) based on longitudinal data from 13,875 American students. The study reports the predominance of males being excluded, but recognises that the proportion of females excluded tends to increase from elementary (23.7%), to secondary (32.7%), to high school (35.2%). More specifically, ref. Bowman‐Perrott et al. (2013, p. 91) concluded that, based on a sample of 2,597 pupils, the predominance of males in exclusion rates (OR = 2.28) was even larger in the case of pupils with learning disabilities (OR = 4.31).²

Ethnicity

Research outcomes suggest a clear and consistent disproportionality in the prevalence of ethnic minorities as a target for disciplinary exclusion (ref. Anyon et al., 2014; ref. Gregory, Skiba, & Noguera, 2010). In the US, different sources of data show that school exclusion overly affects minorities such as Afro‐Caribbean (ref. Noltemeyer & Mcloughlin, 2010), Latino (ref. Skiba et al., 2011) and American Indian students (ref. Gregory et al., 2010) in comparison with their White peers. In the UK, data from the (DfE, 2012) showed that: “The rate of exclusions was highest for Travellers of Irish Heritage, Black Caribbean and Gypsy/Roman ethnic groups. Black Caribbean pupils were nearly 4 times more likely to receive a permanent exclusion than the school population as a whole and were twice as likely to receive a fixed period exclusion.” Notably, recent multivariate analysis points out that racial disproportionality in exclusion still remains significant after controlling by behaviour, number and type of school offences, age, gender, teacher’s ethnicity, and socio‐economic status (ref. Fabelo et al., 2011; ref. Noltemeyer & Mcloughlin, 2010; ref. Rocque & Paternoster, 2011; ref. Skiba, Michael, Nardo, & Peterson, 2002). Consider, for instance, a substantial longitudinal report produced by ref. Fabelo et al., (2011) in Texas (N=928,940), intended to isolate the effect of race alone on disciplinary actions. The study used a multivariate analysis controlling for 83 different variables. The findings suggest that African‐American students were 31% more likely to be removed from classrooms compared to White and Hispanic students. In the same vein, ref. Skiba (2015) has argued that, in the United States at least, racial disproportionality in school discipline is ubiquitous. In his opinion, ethnic minorities are overrepresented in almost all types of school punishment. Even more worryingly, instances of exclusionary discipline among African Americans have continued to increase over the years.

Possible reasons for this overrepresentation of Black students, even when controlling for demographic and risk factors, have been addresses by some scholars, who suggest that a racist bias could explain the phenomenon (Losen, 2011; ref. Skiba et al., 2002; ref. Skiba, 2015). In particular, Simson (2014) asserts that racial stereotyping (conscious or unconscious) as well as a cultural mismatch between teachers and students can explain at least some part of the existing racial disproportionality in school discipline.

It is important to say that, as stated by ref. Theriot, Craun, & Dupper (2010, p.14), “the over‐representation of ethnic minority students, especially African American students, in school suspension and expulsion is one of the most consistent—and perhaps most controversial—findings in the extant literature on school discipline.” In general, studies using solid and strong multivariate models highlight the discrimination against racial minorities compared to White students.

Age as a predictor of exclusion

The likelihood of being punished by exclusion increases with age, being more frequent during adolescence. In England, 52% of permanent exclusions are imposed on pupils aged between 13 and 14 (DfE, 2013). In the case of American students, the results follow a similar trend. In fact, data reported by ref. Liu, (2013) pointed out that suspensions reach a peak in ninth grade (i.e., 14 to 15 years of age). Also based on a sample of American students, ref. Raush & Skiba, (2004) concluded that the number of out‐of‐school suspensions was significantly higher in secondary schools compared to elementary schools.

Socio‐economic status (SES)

Low SES has also been identified as a predictor of high rates of disciplinary exclusion. The UK Department of Education (DfE, 2012) compared the rates of exclusion by eligibility for free school meals (FSM). Those eligible for FSM were 4 times more likely to be punished by a permanent exclusion and around 3 times more likely to get a fixed‐period exclusion than children who were not eligible. In the US, ref. Nichols, (2004) using a sample of 52 schools (37,000 students), found a similar pattern – but the correlation between FSM and exclusion was higher and more significant for pupils in middle school (r = .84; p <. 01) than for elementary (r = ‐.12) or high school pupils (r = .48). In Australia, ref. Hemphill et al., (2010), using multilevel mixed‐effects logistic regression (N = 8,028 students), concluded that pupils settled in low SES neighbourhoods were exposed to higher rates of exclusion (8.7%) when compared with pupils in high SES areas (2.9%).

However, the evidence still seems to be inconclusive in this respect. Recently, ref. Skiba et al., (2012), using a multilevel approach, tested data from 365 schools and a total number of 43,320 students. They concluded that when comparing those students eligible for free or reduced‐cost lunches with their non‐eligible peers, the first were more likely to get out‐of‐school exclusions (OR= 1.27; p<.05). However, contrary to expectations, the eligibility for free or reduced meals resulted in a negative predictor of permanent exclusion (OR= 0.03; p<.05).

Special educational needs (SEN)

Although an increasing amount of research has focused on predictors of school exclusion, analysis of the role of SEN still seems to be limited. In 2007, Achilles, McLaughlin, and Croninger differentiated the role of three different SEN, namely emotional/behavioural disorders (EBD), attention‐deficit/hyperactivity disorders (ADHD), and learning disabilities (LD). Higher rates of exclusion were more likely among those with EBD (OR = 1.49; p<. 001) compared with ADHD (OR = 2.58; p < .001) or LD (OR = 5.44; p < .001). Recently, ref. Bowman‐Perrott et al. (2013), using three waves from the Special Education Elementary Longitudinal Study (SEELS), confirmed that children with emotional or behavioural disorders (OR = 3.95; p <.05) and attention‐deficit or hyperactivity disorders (OR = 4.96; p <.05) were more likely to get suspended or expelled from school than children with learning disabilities (OR = 2.54; p <.05). In a study involving 2,750 students and 39 American schools, ref. Sullivan, Van Norman, & Klingbeil (2014) also observed differences between types of disabilities: those presenting an EBD were at a far greater risk of exclusion (OR = 6.78; SE=0.21) than those presenting other health impairments (i.e., a specific learning disability, intellectual disability, speech and language impairment). When controlling for race and gender, and parents’ education, this trend remained stable and significant. It is important to emphasize that the associations between this disability and exclusion mainly reflect differences in behaviour, respectively psychological or chronic behavioural problems.

1.2.2 Negative outcomes linked to school exclusion

Supporters of zero tolerance policies have pointed out that the use of exclusion can persuade students to account for their behaviour and lead to a decrease in rule‐breaking (ref. Bear, 2012). However, most of the research has consistently documented the negative impact of these types of sanctions (APA Zero Tolerance Task Force, 2008; ref. Chin, Dowdy, Jimerson, & Rime, 2012; ref. Hemphill, Toumbourou, Herrenkohl, McMorris, & Catalano, 2006; Sharkey &Fenning, 2012). In particular, previous research suggests that school exclusion is related to serious negative outcomes in at least three dimensions of young people’s development: behavioural, academic, and future social inclusion.

Behaviour

Some literature related to the relationship between exclusionary punishments and behaviour suggests that such harsh punishments could result in a spiral into more defiant behaviour by students. ref. Raffaele‐Mendez, (2003), for instance, found a moderate and significant correlation (r = .39) between out‐of‐school exclusion (grades 4 to 5) and subsequent exclusion (grade 6). Similarly, ref. Theriot, Craun, & Dupper, (2010) found that pupils punished by in‐school and out‐of‐school exclusion were slightly more likely to get the same punishment again (OR_in‐school = 1.25; p < .001; N = 9706 and OR_{out‐of‐school} = 1.32; p < .001; N = 9706).

Using longitudinal data, ref. Arcia, (2006:366) concluded that school dropout was another behavioural consequence of exclusion. In fact, “43% of students who were suspended 21 or more days dropped out 3 years after their ninth‐grade enrolment.” Similarly, ref. Cratty (2012:649) found a positive correlation between out‐of‐school suspensions and dropout rates. In particular, “those who had an early record of multiple exclusions registered 60% dropout during high school” when compared with non‐excluded students.

The use of exclusion, in turn, is linked with more serious behavioural outcomes such as antisocial conduct, delinquency and entry into the juvenile justice system. Longitudinal research carried out by ref. Hemphill et al. (2006:736) argues that “school suspensions significantly increased antisocial behaviour 12 months later, after holding constant established risk and protective factors (OR = 1.5; 95%CI 1.1 to 2.1; p< .05; N = 3655)”In terms of the involvement of school excludees in the criminal justice system, ref. Costenbader & Markson, (1998) found significant differences between excluded students and those never excluded. In their view, “while 6% of the students who had never been suspended reported having been arrested, on probation, or on parole, 32% of the externally suspended subsample and 14% of the internally suspended subsample responded positively to this question. Males reported significantly more involvement with the legal system than did females.” (p.67). Meanwhile, ref. Challen & Walton, (2004), studying a population of males in the criminal justice system, concluded that more than 80% had been previously excluded from school³.

Academic achievements

Evidence suggests that periods of exclusion may have detrimental effects on pupils’ learning outcomes. Exclusion is accompanied by missed academic activities, alienation, and demotivation in relation to academic goals (ref. Brown, 2007; ref. Michail, 2011). In particular, ref. Hemphill et al., (2006) found that excluded pupils were slightly more prone to fail in the academic curriculum when compared with non‐excluded students (OR = 1.3, 95% CI 1.1 to 1.5, p< .01). Along similar lines, ref. Arcia, (2006) produced a longitudinal retrospective study regarding the associations between exclusions and achievements from fourth to seventh grade. After three years, non‐excluded students displayed substantially higher reading achievement scores when compared with their non‐excluded peers. In fact, seventh‐grade students who were excluded for 21 days or more achieved scores similar to fourth‐grade students that had not been excluded. Finally, Raffaele‐Mendez, (2003) added that those excluded were also less likely to graduate from high school on schedule.

Future social inclusion

Some studies have pointed out that young people excluded from school can also register a high risk of becoming “Not in Education, Employment, or Training” (NEET) in the future. In 2007, Brookes, Goodall, & Heady stated that students who had been excluded were 37% more likely to be unemployed during adulthood. ref. Spielhofer et al. (2009) showed that among individuals with long‐term status as NEET, the majority had previous experienced of exclusions and truancy. More precisely, ref. Massey (2011) argued that approximately one out of two excluded children will be NEET within two years of their exclusion.

Research has also illustrated the long‐term implications of exclusion for society as a whole. In economic terms, the cost of excluding children from school places a demand on public resources. Although the literature on this matter is still limited, ref. Brookes et al. (2007) produced a report regarding the costs of permanent exclusion in the United Kingdom. The analysis encompasses an estimation of costs for the individual as well as for the educational, health, social and criminal justice services. Overall the cost, in 2005 prices, of permanently excluding a student was estimated at £63,851 per year to society.

While there is a stark link between the aforementioned negative outcomes and school exclusion, these should not be regarded as causal. Notwithstanding decades of research on school exclusion and its impact on later behaviour, we are still at an initial stage for testing causal associations in these matters. The association between exclusion and these negative outcomes may simply reflect underlying behavioural tendencies that lead to conduct problems, exclusion and poor outcomes later in life – that is, the antisocial syndrome depicted by ref. Farrington (1997). In fact, school exclusion and the behaviours described here as “negative outcomes” could be explained by the same underlying factors or personality traits characterising the syndrome.

Despite the lack of empirical support for a causal association, some criminological theories provide a plausible explanatory framework to understand the connection between punishment and the persistence of deviant behaviour. Labelling theory, for example, suggests that those punished (by exclusion) and labelled as “deviant” may start behaving in ways that conform to their newly formed self‐image: by limiting their interactions with integrated students, for example, and shunning conventional social systems such as school (ref. Krohn, Lopes, & Ward, 2014, p. 179). Likewise, Sherman’s defiance theory (1993) elucidates the circumstances in which a punishment can produce more antisocial behaviour, such as defiance, instead of compliance with rules. In his view, punishment can increase the prevalence, incidence or seriousness of future offending when offenders deny responsibility, and when they perceive sanctions as unfair, stigmatising and imposed by an illegitimate authority.

Finally, in addition to all these findings and the rationale around the negative outcomes linked to school exclusion, it is important to mention that, so far, there is no evidence demonstrating that exclusion is effective for improving school discipline (ref. Skiba, 2014). What is more, in the short term, exclusion seems to directly deny students’ right to access education as well as reducing adult supervision for those who are most at risk of further deviant behaviour, or most in need of teachers’ support.

1.3 THE INTERVENTION

1.3.1 School‐based programmes

The prevalence of exclusion and its adverse correlated consequences have caught the attention of policy makers and programme developers. As a result, a range of interventions have been designed and implemented to improve school discipline. In the present review, we include different types of school‐based intervention aimed at reducing school exclusion as a punishment for inappropriate behaviour. These interventions include those targeting individual risk factors or school‐related factors, as well as those using a more comprehensive strategy that includes parents, teachers, school administrators, and the community.

Interventions targeting individual risk factors include, for instance, cognitive‐behavioural approaches such as anger management programmes or skills training for children (e.g., ref. Humphrey & Brooks, 2006). Another type of intervention focusing on student behaviour – or, more precisely, students’ skills for conflict resolution – are restorative justice programmes (e.g., ref. Schellenberg & Parks‐Savage, 2007; ref. Shapiro, Burgoon, Welker, & Clough, 2002)In general, these interventions target motivated children and train them in practical skills to deal with anger, solve conflicts or become more assertive in social relationships. Such interventions are normally organised within a curriculum and implemented during school hours. The curriculum involves a package of group or one‐to‐one sessions using a wide range of techniques such as instruction, modelling, role‐play, feedback, and reinforcement, among others (ref. Gottfredson, Cook, & Na, 2012; ref. Schindler & Yoshikawa, 2012).

At the classroom level, interventions may target teachers’ abilities in classroom management (ref. Pane, Rocco, Miller, & Salmon, 2013). The training for teachers encompasses instructional skills, such as guidelines for teaching rules and maintaining attendance, and non‐instructional skills, such as group management techniques, reinforcing positive conduct, and techniques to explain expected behaviour. Both skill sets are aimed at improving the learning process, preventing misbehaviour and encouraging positive participation by pupils (ref. Averdijk, Eisner, Luciano, Valdebenito, & Obsuth, 2014).

Some schools offer mental health services independently or via community agencies. Experienced clinicians are located in schools in order to deliver individual, group, and/or family therapy. Clinicians may also be available for teacher consultation on matters related to students’ behavioural and emotional issues. All these interventions may target a reduction in out‐of‐school exclusion (ref. Bruns, Moore, Stephan, Pruitt, & Weist, 2005).

Alternatively, comprehensive prevention strategies target students, families, teachers and school managers as well as the community as a whole (ref. Bradshaw, Waasdorp, & Leaf, 2012; ref. Flay & Allred, 2003; ref. Colin Pritchard & Williams, 2001; ref. Snyder et al., 2010). A well‐known comprehensive programme is the School‐Wide Positive Behavioural Interventions and Supports (SWPBIS). The programme aims to provide support for positive conduct by building proactive school‐wide disciplinary procedures (i.e., improving school climate and reducing problem behaviours). SWPBIS incorporates a multi‐level approach: from whole school prevention, to group‐based intervention for problematic pupils, and personalised, tailored interventions for high‐risk students. The basic elements of the programme are: i) building a school culture for both social and academic attainment, ii) early prevention of problem behaviours, iii) teaching social skills to all students, iv) using behaviour support practices, and v) actively using data for decision‐making. Research reports promising results, although further and stronger evaluation designs need to be undertaken (ref. Gottfredson et al., 2012; ref. Maag, 2012).

Previous reviews

In 2013/14 we conducted a systematic search of reviews and meta‐analyses assessing the effectiveness of school‐based programmes for promoting early prevention of risks (ref. Averdijk et al., 2014). The results suggested there had been no previous meta‐analysis aimed at assessing the effectiveness of different types of interventions for reducing disciplinary school exclusion. Probably the most similar study is one published by ref. Burrell, Zirbel, & Allen, (2003) who conducted a meta‐analysis on the effectiveness of mediation programmes in educational settings. Among many other outcomes, the analysis suggested that these interventions had a desirable effect (r = ‐.287, k = 17, N = 5,706, p < .05) on administrative suspensions, expulsions and disciplinary actions. However, in this meta‐analysis suspension was reported along with other disciplinary actions, and the study did not compare mediation with any other intervention (as proposed in the present meta‐analysis). The authors also call for a cautious interpretation given the high heterogeneity of primary results. A similar type of analysis was followed by ref. Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, (2011) and ref. Gottfredson, Wilson, & Najaka, (2002). In both studies, school exclusion was coded as an outcome, but the final meta‐analysis did not report on the impact of the intervention specifically in relation to this targeted outcome.

ref. Likewise, Solomon, Klein, Hintze, Cressey, & Peller, (2012) conducted a meta‐analysis exclusively testing the effectiveness of School‐Wide Positive Behavioural Interventions and Supports (SWPBIS) programme. Despite a small number of included studies reporting data on exclusion, the review does not report effect sizes by measuring their increase/decrease. Rather, the review reports effect sizes on the reduction of office discipline referrals and problematic behaviour.

In addition, two narrative reviews have recently been produced looking at intervention as a means of reducing disciplinary exclusion. ref. Spink, (2011) explored qualitative, quantitative and mixed methods studies. Overall, 10 reports were found. The review concluded that multi‐agency interventions were the most frequent and that they could have a positive effect on reducing exclusion of pupils who are at risk. As expected, the study did not report a meta‐analysis of effect sizes. In 2012, Johnson produced another narrative review identifying programmes that may be an alternative for suspension in school systems. The search strategies were not clear enough to allow replication and, again, the nature of the design does not allow for the calculation of effect sizes.

1.4 WHY IT IS IMPORTANT TO DO THE REVIEW

Despite a growing body of research on the negative side effects of exclusion, no previous meta‐analysis based on a comprehensive systematic review has been conducted to synthesize evidence assessing the impact of school‐based interventions in reducing disciplinary exclusion. The current review addresses this gap by meta‐analysing results from existing published and unpublished studies, providing a statistical assessment of the overall effect of school‐based interventions at reducing exclusion.

This meta‐analytic investigation has clear implications for policy making. The results provided by the present study would produce a much‐needed evidence base for school managers, policymakers and researchers alike. These results can contribute to tackling the adverse developmental, social and economic effects of school exclusion mentioned in the previous pages, as well as potentially identifying alternative and less punitive approaches to school discipline.

2. Objectives

The main goal of the present research is to systematically examine the available evidence for the effectiveness of different types of school‐based interventions for reducing disciplinary school exclusion. Secondary goals include comparing different types of interventions (e.g., school‐wide management, classroom management, restorative justice, cognitive‐behavioural interventions) and identifying those that could potentially demonstrate larger and more significant effects.

We also aim – potentially – to run analysis controlling for characteristics of participants (e.g., age, ethnicity, level of risk); interventions (e.g., theoretical bases, components); implementation (e.g., facilitators’ training, doses, quality), and methodology (e.g., research design).

The research questions underlying this project are as follows:

• Do school‐based programmes reduce the use of exclusionary sanctions in schools?
• Are some school‐based approaches more effective than others in reducing exclusionary sanctions?
• Do participants’ characteristics (e.g., age, gender, ethnicity) affect the impact of school‐based programmes on exclusionary sanctions in schools?
• Do characteristics of the interventions, implementation, and methodology affect the impact of school‐based programmes on exclusionary sanctions in schools?

3. Methods

3.1 TITLE REGISTRATION AND REVIEW PROTOCOL

The title of the present review was registered in The Campbell Collaboration Library of Systematic Reviews on January 2015. The final version of the review protocol was approved in November 2015. The title registration and the respective protocol are available at: https://www.campbellcollaboration.org/library/reducing‐school‐exclusion‐school‐based‐interventions.html

3.2 CRITERIA FOR CONSIDERING STUDIES FOR THIS REVIEW

3.2.1 Research design

Our original proposal was to include both randomised controlled trials and high‐quality quasi‐experimental studies (defined as studies using a comparison group, pre‐post testing and a statistical matching approach). To be eligible for inclusion, we stated that manuscripts must clearly report the method used to ensure equivalence between treatment and control groups, taking into account major risk factors (e.g. behavioural measures) and demographic characteristics.⁴

In this review, we only present results from randomised controlled trials (RCTs). There were three reasons for our decision:

• First, even though a number of quasi‐experimental studies initially fulfilling our inclusion criteria were found by our searches, many of them fail to report baseline measures (e.g., ref. Guardino, 2013; ref. Hawkins, Catalano, Kosterman, Abbott, & Hill, 1999; ref. Munoz, Fischetti, & Prather, 2014), the matching procedures were not described (e.g., ref. Hasson, 2011; ref. Risner, Vanderhaar, Muñoz, & Abati, 2005; ref. St James‐Roberts & Singh, 2001) or the balance procedures did not produce statistical equivalence (e.g., ref. Gao, Hallar, & Hartman, 2014).
• A number of the school‐based intervention programmes included in this review presented several studies, involving quasi‐experiments as well as RCTs. Some examples involve interventions such as the Positive Action Program or the School‐Wide Positive Behavioural Interventions and Supports (e.g., ref. Lewis et al., 2013; ref. Snyder et al., 2010). In both cases, RCTs (e.g., ref. Lewis et al., 2013; ref. Snyder et al., 2010) were preceded by quasi‐experimental studies (Barrett, Bradshaw, & Lewis‐Palmer, 2008; ref. Flay & Allred, 2003) . In this context, we decided to keep the strongest study design.
• RCTs are regarded to be the most compelling methodological design to test the impact of a particular treatment. This type of study has the strengths of isolating confounding factors, reducing the likelihood of alternative explanations for observed effects (ref. Shadish, Cook, & Campbell, 2002; ref. Sherman, Farrington, Welsh, & Mackenzie, 2002). We believe that by selecting only these studies we will achieve a more precise final estimation of the effect of school‐based interventions.

To offer a broad overview of the research testing the impact of school‐based intervention at reducing school exclusion, a list of the quasi‐experimental studies can be provided on request.

Qualitative studies were excluded from the present review as stated in the published protocol.

3.2.2 Types of participants

The present review is focused on the general population of students in primary and secondary schools irrespective of nationality, ethnicity, language, and cultural or socio‐economical background. By targeting primary and secondary schools, participants could theoretically be aged from 4 to 18 years of age.

Reports involving students who presented special education needs, disabilities or learning problems but were educated in mainstream schools were included in this review. However, reports involving students with serious mental disabilities or those in need of special schools were excluded. The rationale for this is that the results of this review are intended to be generalisable to mainstream populations of students in non‐specialised schools from all the included countries.

Students in college or higher levels of education have been excluded. Their exclusion from the review is based on previous evidence suggesting the largest number of exclusions affect pupils aged about 10 to 15 (e.g., ref. Liu, 2013; ref. Raush & Skiba, 2004; DfE, 2012).

3.2.3 Included interventions

We include interventions defined as school‐based: that is, delivered on school premises, or supported by schools with at least one component implemented in the school setting. In the present review, we include interventions explicitly aimed at preventing/reducing school exclusion or those measuring exclusion as an outcome.

Interventions in the present review cover a wide range of psychosocial strategies for targeting students (e.g., ref. Cook et al., 2014), teachers (e.g., ref. Ialongo, Poduska, Werthamer, & Kellam, 2001), or the whole school (e.g., ref. Bradshaw, Waasdorp, & Leaf, 2012). Types of intervention include, for example, those focused on:

• instructing students to identify risky behaviours and expanding their alternatives for responding appropriately to risks or harms (e.g., social skills training)
• developing teachers’ skills to improve the quality of their classroom management (e.g., reward schemes)
• cognitive‐behavioural treatment, such as anger management, counselling, social work, and mentoring programmes;
• school‐wide interventions.

Since there was no previous review analysing school‐based prevention programmes for reducing exclusionary discipline, we wanted to include a wide range of school‐based interventions that could be effective for reducing exclusionary practices.

3.2.4 Excluded interventions

We excluded studies where the intervention was not school‐based or school supported. Even though some of these interventions targeted school students, they were community programmes or purely focused on mental health issues without any connection to schools (e.g., ref. Henderson & Green, 2014; ref. Schwartz, Rhodes, Spencer, & Grossman, 2013; ref. Wiggins et al., 2009).

We also excluded interventions designed for children or adolescents who have committed a crime, that is, specialised interventions aimed at reducing reoffending or reconviction. Although suspended students may commit offences, such specialised interventions were excluded from the present review because they exceed the strategies used by schools to prevent misbehaviour and their levels of complexity make them too specific for a general population of students. School‐based prevention programmes targeting outcomes related only to students’ physical health (e.g., AIDS/ HIV prevention programmes, programmes to develop healthy eating programmes) were also excluded.

3.2.5 Types of outcome measures

Primary outcomes

Eligible studies addressed school exclusion as an outcome. As mentioned in the background section, school suspension or exclusion is defined as an official disciplinary sanction imposed by an authority and consisting of the removal of a child from their normal schooling. This removal happens as a reaction to student behaviour that violates the school rules. We included studies testing fixed or permanent, long‐term or short‐term suspension as well as in‐school and out‐of‐school suspensions.

We excluded studies testing other disciplinary sanctions implemented in schools if they do not share the criteria described above. For instance, we excluded disciplinary sanctions such as loss of privileges, extra work, break/lunch detention, and after‐school detentions. These interventions do not involve exclusion from school or exclusion from regular teaching hours, and as such they are not covered by this review.

Secondary outcomes

For any identified study that reported findings on school exclusion as an outcome, we also coded the effects of the intervention on specific behaviour domains, focusing on internalising (e.g., inhibition, social withdrawal, anxiety or depression) and externalising (e.g., defiant or delinquent behaviours or aggressive behaviours such as bullying)problem behaviour (ref. Achenbach & Edelbrock, 1979; ref. Achenbach, 1978; ref. Farrington, 1989)

By coding secondary outcomes, we aimed to assess the extent to which reductions in problem behaviour are a mediator of treatment effects on school exclusion. Indeed, interventions may affect exclusion in two different ways. The first is by improving behaviour that might otherwise lead to an exclusionary measure. The second possibility is that behaviour stays the same, but that the school develops an alternative strategy to deal with the disciplinary problems.

3.2.6 Included literature

Databases and journals were searched from 1980 onwards with the aim of comprising more contemporary interventions or prevention programmes. Eligible studies included both published and unpublished book chapters, journal articles, government reports, and Doctoral theses. When the same data was published in more than one source (e.g., a book chapter and a journal article) we used all the linked manuscripts but the most complete report or the report measuring suspension was defined as the main source of data (see Section 3.4.5). That way we kept as much information as possible from a specific study but avoided over‐estimation of effect sizes. In cases where it was not clear if the manuscripts referred to the same study, we contacted the main author for further information (e.g., email communication with ref. Barnes, Bauza, & Treiber, 2003; ref. Bradshaw et al., 2012; ref. Lewis, Romi, & Roache, 2012; ref. Snyder et al., 2010; ref. Sprague, Biglan, Rusby, Gau, & Vincent, 2016).

3.3 SEARCH METHODS FOR IDENTIFICATION OF STUDIES

The electronic searches were conducted between September 1 and December 1, 2015. In order to reduce the effect of publication bias, an attempt was made to locate the most complete collection of published and unpublished papers.

3.3.1 Electronic searches

Below we list details of the 27 electronic databases searched. As noted above, these databases included both published (e.g., ISI web of knowledge, PsycINFO) and unpublished reports (e.g., Dissertation Abstracts, EThOS) as well as reports from Latin‐American countries (e.g., Scientific Electronic Library Online – SciELO).

For each database, we ran pilot searches including the key terms described in Table 3. Four categories of key words were used, including: i) type of study; ii) type of intervention; iii) population; and iv) outcomes. The pilot searches were useful to adjust the terms, synonyms and wildcards as appropriate. They were also helpful in creating combinations of terms that capture relevant sets of studies in each database.

We kept a record with the date of searches, number of reports found, number of reports retrieved, key terms included, synonyms, and wildcards used when appropriate. Further details of electronic searches are presented in Section 13.

3.3.2 Other resources searched

As planned, we contacted key authors requesting information on primary studies that could potentially be integrated in this systematic review and meta‐analysis. We also reviewed reference lists of previous primary studies or reviews related to the intervention/outcomes (e.g., ref. Burrell, Zirbel, & Allen, 2003; ref. Gottfredson, Cook, & Na, 2012; ref. Johnson, 2012; ref. Mytton, DiGuiseppi, Gough, Taylor, & Logan, 2006; ref. Wilson, Tanner‐Smith, Lipsey, Steinka‐Fry, & Morrison, 2011).

3.4 DATA COLLECTION AND ANALYSIS

3.4.1 Selection of studies

Eligible studies met the following criteria:

• Reported results of interventions from 1980 onwards
• Tested the impact of a school‐based intervention on different types of exclusion (e.g., in‐school, out‐of‐school, expulsion)
• Included students from primary and secondary school levels settled in mainstream schools
• Based on an experimental design, where participants are randomly allocated treatment or control conditions
• Reported statistical results for computed an effect size

3.4.2 Data extraction and management

Data extraction was the responsibility of two researchers (AC & SV). Descriptive data of all studies potentially includable in the meta‐analysis was extracted using the data collection instrument presented in Section 12.2. The instrument facilitated the extraction of the following information:

• Bibliographical data (e.g., type of publication, year of publication, name of the publication, main author discipline)
• Ethics (e.g. declaration of conflicts of interest, use of informed consent)
• Research methods (e.g., type of design, units of randomisation, unit of analysis, variables used for matching)
• Sample selection (e.g., methods to select sample, attrition)
• Primary outcome coding (e.g., type of exclusion, duration of exclusion)
• Secondary outcomes coding (e.g., internalising and externalising behaviours, name of the instrument used to measure the outcome data)
• Base‐line measurements (e.g., source of data, quantitative measure of the primary outcome)
• Programme delivery (e.g., programme deliverer, training, type of intervention, frequency of the intervention)
• Post‐intervention and follow‐up measurement (e.g. official records, surveys)
• Data for calculation on effect sizes

The same two researchers extracted data for effect‐size calculations. The process was carried out independently. In general, discrepancies were solved by agreement but when the information reported was contentious, we asked for input from the more senior members of the team (ME & DF). Details on the data extracted from each included report can be found in section 9.1, 9.2 and 9.3 of the present review.

When data for calculation of effect sizes was incomplete we used two different strategies. First, we tried to find more details in other sources (e.g., published protocols or reports). Secondly, the lead researcher or members of the research team were contacted regarding the additional data needed.

Endnote X7 software was used to manage references, citations and documents. Data extracted to characterise studies was inputted in STATA v.13 in order to produce inferential/descriptive statistics. Effect sizes were inputted in Version 3.0 of the Comprehensive Meta‐Analysis software.

3.4.3 Strategy to test inter‐rate reliability

To check code consistency across studies, or inter‐rate reliability, we use Cohen’s Kappa coefficient (ref. Cohen, 1960). In the event of two coders making inclusion/exclusion decisions, Cohen’s Kappa coefficient computes a standardised index across studies based on cross‐tabulated ratings. The index is given by the difference between the observed percentage of agreement in ratings across studies (P_o ), and the probability of expected agreement due to chance (P_e ), divided by 1‐P_e .

Cohen’s Kappa was calculated based on the following formula:

To a good approximation, we calculated the standard deviation of Cohen’s Kappa, following the expression (ref. Cohen, 1960):

3.4.4 Assessment of risk of bias in included studies

We planned to assess the risk of bias in included studies by using two different instruments. In the case of RCTs, we intended to use the Cochrane Collaboration risk of bias tool (ref. Higgins & Green, 2011). To analyse risk of bias of studies involving quasi‐experimental designs we planned to use the ACROBAT‐NRSI, another Cochrane Risk of Bias Assessment Tool for Non‐Randomised Studies (ref. Stern, Higgins, & Reeves, 2014). Both instruments were supposed to assist the research team in evaluating the external validity of the included reports.

At the end of January 2016, we began the assessment with the originally proposed instruments. We soon realised that the instruments and their categories seemed more suited to medical trials than school‐based experiments. Therefore, possible alternatives were explored. In consultation with the coordinating group editor, we selected the EPOC risk of bias tool suggested in the methods section of the Campbell Collaboration website (see the tool in section 12.3). The instrument proposes the following eight criteria for the assessment of quality bias:

• Sequence generation
• Allocation concealment
• Baseline outcome equivalence
• Baseline characteristics equivalence
• Incomplete outcome data
• Blinding of outcome assessment
• Protection against contamination
• Selective outcome reporting

Each of these domains was judged on a 3‐point scale (i.e., low risk, high risk, unclear risk). EPOC tool provides guidance and examples for each domain that facilitate the decision of assigning low, high or unclear risk. Two members of the team performed the assessment of risk of bias (AS & SV). Assessment of bias was performed independently and the final results represent the agreement of both evaluators.

3.4.5 Criteria for determination of independent findings

Since violations of the assumptions of independence in meta‐analysis would lead to incorrect estimates of the variance for pooled effect sizes (ref. Higgins & Green, 2011b; Romano & Kromrey, 2009),we used some strategies to deal with dependency in the data extracted from primary studies.

First, since we included book chapters, journal articles, government reports, and academic PhD theses, we anticipated the case where the same results would be published in more than one source (e.g., a book chapter and a journal article). The protocol stated that in those cases we would code only one outcome (e.g., the most complete, or the most outdated). In practice, we excluded 11 reports whose results were reported in more than one publication. They are grouped in four cases:

• The study did not report enough statistical data for effect size calculation. For instance, Vincent, Sprague, Pavel, Tobin, & Gau, (2015) did not report enough data for meta‐calculation. Although the main author was contacted, we were not able to access more details. For that reason we decided to include ref. Sprague, Biglan, Rusby, Gau, & Vincent, (2016). This latter case used the same data but reported enough information for meta‐analysis. There was no overlap.
• RCT data was merged with quasi‐experimental data. For instance, ref. Allen & Philliber, (2001) merged the RCT sample with another sample of students. As a result, the study was not an RCT anymore (For further details see ref. Allen & Philliber 2001, p. 641).
• The same results were reported in more than one manuscript. That was the case of Panayiotopoulos & Kerfoot. The results of the study were reported in two papers (2004 and 2007). Results were identical in both publications (same dataset, same analysis and same outcomes). There was no overlap.
• The study did not report the outcome measured (i.e., suspension). A case in point is ref. Arter, (2007). The author reported the results of her thesis in a journal. The journal article did not describe the outcome suspension, probably because no effect was found. We included the thesis since it reported all the outcomes measured.

Based on that, those 11 manuscripts were not included in the meta‐analysis. There was no overlap among them, consequently, no dependence in the outcomes was observed.

Secondly, included studies reported multiple time points, for instance, multiple follow‐up measures. The inclusion of multiple follow‐ups would create statistical dependence because the different measures are based on the same subjects (i.e., correlated with each other). We calculated effect sizes separately for those studies reporting short‐term and long‐term follow‐up measures. We also corrected variances estimation (see 3.4.8).

3.4.6 Measures of treatment effect

We use Standardised Means Differences (SMD or Cohen’s d) to measure the treatment effects of the school‐based interventions included in the review. The decision to use this specific effect size is based on the fact that most of the included manuscripts report results where measurements are expressed in continuous scales (see section 9.3). The standardised mean effect size for a non‐clustered study is given by

where X¯e and X¯c represent the experimental and control group means, respectively, S_p is the pooled sample standard deviation given by

where n_e and n_c are the sample size in each group, and se2 and sc2 are the experimental and control group standard deviation, respectively.

3.4.7 Issues with the unit of analysis

In the present review, we anticipated the inclusion of primary studies involving individually randomised as well as cluster‐randomised unit (e.g., schools or classrooms). One key issue emerges when meta‐analyses include cluster‐randomised studies: participants nested in the same cluster tend to be more similar to one another (as measured by the intra class correlation – ICC). Furthermore, when units of randomisation are clusters instead of individuals, we need to deal with the fact that the data presents different levels of variation (i.e., within clusters variation, between clusters variation and the total variance). This issue needs to be taken into account when computing effect size estimates. When this correlation is not accounted for, standard errors, confidence intervals and p‐values will tend to be too small. These conditions affect the meta‐analysis in two different ways. Firstly, the primary trial gets a mistakenly large weight. Secondly, the pooled result produces estimated effect sizes with an overly small standard error (ref. Borenstein, Hedges, Higgins, & Rothstein, 2009; ref. Higgins et al., 2011).

For the case of clustered data with dichotomous outcome measures (e.g., odds ratios), we followed the strategy proposed by the Cochrane Handbook for Systematic Reviews of Interventions (ref. Higgins & Green, 2011), which corrects standard errors of effect sizes. The handbook suggests that the effective sample size in a cluster‐randomised trial can be obtained by dividing the original sample size by the design effect, which is calculated via

In this equation, M is the average cluster size (units per cluster) and ICC is the intra‐cluster correlation coefficient. Once we were able to identify the design effect, the squared root of the design effect could be multiplied by the original standard error of the log Odds Ratio. Since ICC is rarely reported in primary studies, we have assumed a value of .05, based on the review of multiple meta‐analyses testing similar populations, produced by ref. Ahn, Myers, & Jin, (2012).

In the case of clustered studies with continuous outcomes (e.g., school level means and standard deviations), we followed the strategy suggested by ref. Hedges (2007) and ref. Spier et al. (2013). Effect sizes were computed using d_T2 assuming equal cluster sample size:

In this equation X¯¯■E and X¯¯■C represent the overall means of the experimental and control group and S_T is the total sample standard deviation estimated from the pooled sample standard deviation across the experimental and the control group. Rho (is the notation used to represent the intra class correlation. N is the total sample size and the sample size of the clusters is represented by n. Based on the characteristics of our data and following ref. Spier et al., (2013) we assume equal cluster size in our calculations.⁵ When the clusters have different sizes, we will take a conservative approach, including the smallest cluster size in our calculation.

The variance of the effect size will be calculated by

In this equation, N^E and N^C represent the experimental and control group sample across clusters. As suggested by ref. Higgins & Green, (2011), in the event that the value of ρ is not reported, analysts are advised to assume a reasonable value based in previous studies with similar population (ref. Ahn et al., 2012). As detailed previously, we have assumed a value of (ρ)= .05.

3.4.8 Dealing with missing data

In those reports where key statistical information was missing, we attempted to obtain data from principal investigators. When that was not possible, the study was excluded from calculations of effect sizes. All the studies excluded for this reason have been identified and systematically reported in section 4.1.3.

3.4.9 Time points within a study

In the present review, many of our selected studies involved repeated measures of the outcome exclusion. In fact, manuscripts reported measures of exclusion at baseline and post treatment (e.g., ref. Hawkins, Doueck, & Lishner, 1988) or posttreatment and follow‐up (e.g., ref. Farrell, Meyer, & White, 2001). In these cases, we have calculated a synthesis index or effect size of the difference, representing the change between those different measures (i.e., time points). Consequently, the change in exclusion from the baseline is computed by subtracting the means (X) as follows:

One issue arises when pursuing this strategy. Because measures at baseline and posttreatment are positively correlated, the calculation of the variance must be corrected. If we avoid the correction, assuming the two measures to be independent (correlation equals zero), we could be overestimating the variance and underestimating the precision of the difference (ref. Borenstein et al., 2009). For a fair approximation of the value of the variance, we would need to know the correlation between the pre‐ and post‐measures (covariation). However, the covariance is not usually reported in primary research (and this was commonly the case in our set of included studies). Consequently, we proceed to assume a value for that correlation. After checking previous meta‐analysis of similar populations, testing school‐based interventions with estimates for the stability of serious problem behaviours (e.g., ref. Farrington & Ttofi, 2009; ref. Wilson et al., 2011), we concluded that the value of the pre‐post correlation should be assumed to be equal to .75. We then calculated the variance of the difference by using equation 9 below (ref. Borenstein et al., 2009), where V₁ and V₂ represent the variances of the original point estimates and r represents the pre‐post correlation value:

As suggested by ref. Higgins & Green, (2011), we undertook sensitivity analyses to determine whether the overall result of the analysis is robust in the use of imputed correlation coefficient.

3.4.10 Assessment of heterogeneity

We report weighted mean effect sizes, under a random model using 95% confidence intervals and accompanied with graphical representation (i.e., forest plots). For investigating heterogeneity, we use the estimates suggested by ref. Borenstein et al. (2009), specifically; Tau‐squared, Q‐statistic and I² .

Tau‐squared, or the difference between the total variance or variance observed and the within‐studies variance, will be estimated and reported.

The final calculation of the Q‐statistic includes reporting its value, degrees of freedom and p‐values. Significant p‐values provide evidence of heterogeneity in intervention effects.

Bearing in mind that Q can appear distorted when the number of studies meta‐analysed is small (ref. Higgins, Thompson, Deeks, & Altman, 2003), we also report I2. I2 “is the proportion of observed dispersion that is real rather than spurious” (ref. Borenstein et al., 2009). High percentages will be interpreted as an indication of high heterogeneity, meaning that the study‐to‐study dispersion is due to real differences in true effect size and not attributable to random error.

3.4.11 Data synthesis

Since our review has a wide scope, we use the random effect inverse variance weighted models for meta‐analytical calculations. The random effect model is the most appropriate when effect sizes are not homogeneous or consistently coming from a single population (ref. Borenstein et al., 2009). Under a random effects model the variance includes the original (within‐studies) variance plus the between‐studies variance, Tau² .

Following ref. Chandler, Churchill, Higgins, Lasserson, & Tovey, (2013), effect sizes will be coded such that a positive effect will reflect the outcomes favouring the treatment group. To illustrate our analysis, we provide summary forest plots displaying the estimated effect sizes along with their 95% confidence intervals.

3.4.12 Subgroup analysis and investigation of heterogeneity

In the present review, we use moderator analysis involving categorical variables estimating models analogous to ANOVA. Analyses are run under a random‐effect model assuming separate variance components for each group. Meta regression has been run in order to explore heterogeneity.

3.4.13 Outliers

The distribution of SMD effect sizes was examined to determine the presence of outliers. Following ref. Lipsey & Wilson, (2001), outliers were defined as those values which are more than two standard deviations from the overall mean of effect sizes. One outlier was detected (i.e., ref. Collier, 2002) and it was windsorised to the next closest value (ref. Lipsey & Wilson, 2001).

3.4.14 Sensitivity analysis

Since the present meta‐analysis involved a wide range of decisions, we conducted sensitivity analysis to test the robustness of these decisions (ref. Higgins & Green, 2011). Specifically, we ran sensitivity analysis for the pre‐post correlations (i.e., covariance) assumed to be .75. We re‐ran the analysis using a correlation equal .50. As expected there was no change in the effect sizes and no relevant difference in standard errors. We also ran sensitivity analysis testing the impact of the outlier and the impact of the windsorisation (see section 4.7).

As stated in our protocol, we ran sensitivity analysis testing differences between published and unpublished reports.

3.4.15 Duration of follow‐up

Included studies reported multiple time points data, for instance, multiple follow‐up measures. Since the inclusion of multiple follow‐ups would create statistical dependence due to the different measures based on the same subjects (i.e., correlated with each other), we calculate effect sizes separately for those studies reporting short‐term (i.e., post‐treatment) and long‐term (i.e., follow‐up) measures.

4. Results

4.1 DESCRIPTION OF STUDIES

4.1.1 Results of the searches

We attempted to identify and retrieve the body of published and unpublished studies that met our inclusion criteria. Figure 1 shows a PRISMA flow diagram describing the results of our searches.

At the beginning, different combination of terms produced a total of 42,749 references from different electronic databases, of which we kept 1,474 relevant hits.

The 1,474 hits were selected based on screening the title, abstract and key words. We targeted studies broadly defined as evaluations, testing the impact of interventions on school exclusion.

As originally planned, we complemented electronic searches for papers with two extra strategies: i) reviewing lists of references from retrieved manuscripts, and ii) communication with key authors. Based on these steps, an additional 22 manuscripts were added to our sample. The full list of studies and their references were imported into Endnote X7. After removing duplicates, a total of 516 unique manuscripts were saved for further assessment.

Efforts were made to retrieve the full text copies of all 516 selected manuscripts. Notably, a high percentage of them corresponded to unpublished reports (48.6%), mainly PhD theses from American universities and Technical or Governmental reports. In the end, we were able to retrieve almost all full text manuscripts. Only eight studies out of 516 were categorised as “studies awaiting classification” and they are reported in Table 6.

The next round of evaluation was based on reading the information available in abstracts, methods and results sections using the eligibility checklist (see Section 12.1). The checklist included the basic criteria for inclusion in this systematic review. We concluded the second round of evaluation with 471 manuscripts excluded for reasons laid out in section 4.1.3. Thirty‐seven papers presented enough statistical data for inclusion in our meta‐analysis.

4.1.2 Inter‐rater reliability

Two trained researchers (AC & SV) independently assessed the 516 pre‐selected manuscripts for inclusion or exclusion. We calculated Cohen’s Kappa for testing inter‐rater reliability (ref. Cohen, 1960; ref. Sim & Wright, 2005). The value of Kappa ranges between 0 and 1, where a value of 1 represents perfect agreement between the two raters and a value of 0 indicates no more agreement than that expected by chance. We obtained a Cohen’s Kappa= .76; SD=.81; SE=.036, reflecting a high level of agreement between coders. After calculating the agreement between coders, they went through the papers where they found disagreement. Differences were solved by further analysis and discussion. When needed, a more senior member of the team was consulted.

4.1.3 Excluded studies

Following our published protocol, we excluded a total of 471 manuscripts. Table 4 summarises the reasons for the exclusion of each report. References of the excluded papers are availablein section 7.2.

Outcome. We excluded 52 reports (11%) because they did not present a suitable measure of school exclusion. In some specific cases the primary outcome was not reported (e.g., ref. Gage, Sugai, Lewis, & Brzozowy, 2015; ref. Webster‐Stratton, Rinaldi, & Reid, 2011) or it was reported in a composite measure along with other disciplinary measures which did not involve any type of exclusion from school (e.g., De Blank, 2009; Wright, Offord, John, Duku, & DeWit, 2005). Since it was not possible to isolate our primary outcome we excluded those reports. In other exceptional cases exclusion was measured as a predictor instead of an outcome (e.g., ref. Rosenbaum, 2012).

Type of intervention. Following our protocol, we excluded 53 (11.2%) studies because the tested intervention was not delivered in schools, supported by schools or with at least one component implemented in school settings. Consequently, we excluded community programmes when they had no alliance with a school (.e.g., ref. Henderson & Green, 2014; ref. Wiggins et al., 2009). Alternative schools for high‐risk students (e.g., ref. Rhea, 2010)and intervention in special schools (e.g., ref. Kutash, Duchnowski, Green, & Ferron, 2013) were also excluded. Since our protocol stated that targeted intervention must be an “alternative to school exclusion”, we dismissed studies testing for instance the impact of restorative justice in the context of an in‐school exclusion programme (e.g., ref. Brown‐Kersey, 2011), or conflict resolution in the context of an in‐school exclusion programme (e.g., ref. Devlin, 2006). In both cases, the tested intervention was delivered in addition to exclusion more than being an alternative to it.

Methodological design. 339 (72%) studies were excluded because they did not satisfy the methodological characteristics defined in the protocol. We excluded studies lacking a control group (e.g., ref. McDaniel‐Campbell, 2011; ref. Nocera, Whitbread, & Nocera, 2014) and those studies where the control group was not equivalent in demographics and risk factor variables (e.g., ref. Kilian, Fish, & Maniago, 2006; ref. May, Stokes, Oliver, & McClure, 2015).

During the searches, we kept 28 manuscripts which corresponded to literature reviews (e.g., ref. Fronius, Persson, Guckenburg, Hurley, & Petrosino, 2016; ref. Gonzalez, 2012), systematic reviews (e.g., ref. Wolgemuth, Cobb, & Dugan, 2007) or meta‐analysis (e.g., ref. Noltemeyer & Ward, 2015) related to school exclusion or behavioural problems in schools. These types of manuscripts were initially retained on the understanding that they could be a source for identifying extra primary research reports. All 28 of these studies were excluded in the second round of assessment once we had checked their citation lists.

We identified nine manuscripts evaluating the impact of obligatory use of school uniform on levels of school exclusion (i.e., ref. Draa, 2005; ref. Gentile & Imberman, 2012; ref. Gouge, 2011; ref. Johnson, 2010; Samuels & Bishop, 2003; ref. Shimizu & Peterson, 2000; Stevenson III & Brooks II, 1999; ref. Vaughan, 2001; Washington‐Labat & Ginn, 2003). Even if they represented a particular type of intervention that could be interesting to meta‐analyse and compare, none of these eight interventions presented the research design targeted by our review. For that reason, they were excluded.

Finally, under the method design criteria we excluded a number of qualitative studies (ref. Huston, 1999; ref. Maguire, Macrae, & Milbourne, 2003; ref. Rose, 2008) and case studies (e.g., ref. Navarro, Aguilar, Aguilar, Alcalde, & Marchena, 2007) since none of them contributed with statistical data for meta‐analysis.

Participants. Five reports (1.1%) presented data focused on students with special needs (e.g., ref. Cramer, 1990), or young offenders (e.g., ref. Parkes, 2008). Since these participants were not targeted in our protocol, all these reports were excluded from our analysis.

Time span. Five studies (1.1%) were excluded because they were published before 1980 (i.e., ref. Rogers, 1972)or because they reported the evaluation of an intervention implemented before 1980 (i.e., ref. Feldi, 1980; ref. Fisher, 1980; ref. Herzog, 1980; ref. Safer, Heaton, & Parker, 1981).

Reports based on the same dataset. Finally, we excluded 11 reports (2.3%) because they presented additional results based on the same data reported elsewhere (e.g., ref. Allen & Philliber, 2001; ref. Arter, 2007; ref. Panayiotopoulos & Kerfoot, 2007; Vincent, Sprague, Pavel, Tobin, & Gau, 2015). In these cases, we kept the most complete report and the additional manuscripts assisted in a better understanding of the included research; they were however defined as excluded.

Pilot study. We excluded a single report presenting data from a pilot study (ref. Bonell et al., 2015). In page 12 of the cited report, the author states that the aim of the pilot was “to evaluate feasibility and acceptability of the intervention and trial methods, and not to estimate intervention effects.” For those reasons, even though it was reporting results, the study was excluded. We kept references to the report in the category of ongoing research (see section 4.1.4).

Not enough data for meta‐calculations. Five reports (1.1%) were excluded because they did not present enough data for calculation of effect sizes (e.g., ref. Grinage, 2005).

4.1.4 On‐going studies

We identified the protocol of four ongoing studies whose outcomes had not been published at the end of the searches in December 2015. As observed in Table 5, all of them are cluster randomised control trials testing the impact of school‐based interventions and measuring school exclusion as an outcome.

4.1.5 Studies awaiting classification

We were unable to classify eight studies. We selected them based on title, abstract and key words but we have not been able to locate the full text copies. A list of studies that could be potentially included in a future updated version of this review is given in Table 6.

4.1.6 Included studies

Thirty‐seven studies reporting 38 interventions’ effect sizes were included in this meta‐analysis. As we mention before, all of them were randomised controlled trials. In 23 studies the control group received no‐treatment (62.2%); six studies reported controls receiving intervention or business as usual (16.2%), four experiments offered a placebo to the control group (10.8%) and four studies allocated controls in a waiting list (10.8%).

Following sections provide a general overview of 37 included studies. Data is organised by characteristics of the included studies, characteristics of participants and characteristics of the delivered interventions.

Characteristics of the included studies

We included studies presenting interventions (M_date =2003; SD=9.5) carried out and reported between 1980 and December 1, 2015 when we finished our searches. Exceptionally the review involves three manuscripts published in 2016. In the first case, ref. Obsuth et al., (2016), the registered protocol of the study had been identified in our electronic searches and we were waiting for the published version, which was released in March 2016. The second case involves two different papers: ref. Okonofua, Pauneskua, & Walton, (2016) and ref. Sprague, Biglan, Rusby, Gau, & Vincent, (2016). Both studies were sent to us from key authors in the field. As they were recent studies, matching our inclusion criteria, we decided to retain them in our analysis. No other study was sent to us after the end of the searches in December 2015. Figure 10.1 (in appendix) presents the distribution of studies per year. Results show that the number of RCTs has increased over the specific years involved in this report.

As shown in Table 7, we included published and unpublished reports in almost equal proportion (51% published versus 49% of unpublished reports). All of them were written in English, and represent studies implemented in the United States and the United Kingdom only. Although we explored global databases, in particular databases from Latin‐American countries we were not able to find studies conducted in other locations.

We coded data on the main author’s discipline. Our data shows that more than 60% of studies testing interventions intended to reduce exclusion have been carried out by researchers in the fields of Education and Psychology.

Interestingly, only seven studies (36.8%) of experimental evaluations published in peer‐reviewed journals disclosed a personal or organisational Conflict of Interest (CoI). This percentage is coherent with the findings of ref. Eisner, Humphreys, Wilson, & Gardner, (2015) who found limited attention to full CoI disclosure in the evaluation of psychosocial interventions. They argue that even if “transparency about CoI in itself does not necessarily improve the quality of research, and researchers with a CoI should not be presumed to conduct less valid scholarship, transparency is needed for readers, to assess the study findings and their particular context” (ref. Eisner et al., 2015, p. 10).

In addition to the presence/absence of CoI statements, we evaluated studies on their potential conflict of financial interest (CoFI) by using a scale developed by ref. Eisner & Humphreys, (2012). It is a trichotomous scale that helps to identify three levels of conflict. The three categories in the scale are defined as follows: i) Unlikely conflict of interest: none of the study authors are programme developers or licence holders; ii) Possible: a study author is a programme developer or collaborator with a programme developer AND the programme is not (yet) commercially available OR the business model is ‘not‐for‐profit’; or iii) Likely: study author, is a programme developer or collaborator with a programme developer AND programme is commercially available AND business model is ‘for‐profit’. For details on the instrument see section 12.4.

We found 18 studies (48.6%) where the CoFI was defined as ‘unlikely’. Essentially, in this set of studies, none of the programme evaluators were involved (i.e., directly or as a collaborator) in the development of the intervention or were licence holders. However, we found 13 studies (35.1%) where we assessed a ‘possible’ CoFI. In those cases, the evaluator was a programme developer/deliverer or a previous collaborator with a programme developer; the programme was not commercially available, or the business model was defined as ‘not‐for‐profit’. Finally, only two studies (5.4%) in our evaluation transparently declared information that allowed us to classify them as ‘likely’ to present a potential financial conflict of interest. In both cases the authors reported that one of the members of the evaluation team was related to the holder of a licensed programme evaluated. The evaluation of each study is reported in section 9.2.

In terms of methodological design, all included studies were randomised controlled trials. While 70.3% of our studies randomised individuals, almost 30% (11 studies) randomised clusters of students, that is, entire schools (i.e., ref. Bradshaw et al., 2012; ref. Cornell, Allen, & Fan, 2012; ref. Lewis et al., 2013; ref. Obsuth et al., 2016; ref. Snyder et al., 2010; ref. Sprague et al., 2016; ref. Ward & Gersten, 2013; ref. Wyman et al., 2010) or classrooms within schools (i.e., ref. Brett, 1993; Farrell, Meyer, & White, 2010; ref. Ialongo, Poduska, Werthamer, & Kellam, 2001). Where necessary, we corrected data in clusters in order to combine it with individual level data. For further details on corrections, see section 9.3.

The most frequent analysis of the data was differences in means such as chi‐squared, ANOVA, ANCOVA or MANOVA (59.4%) followed by regression (21.6%), and with a minority of the studies running Multi‐level analysis (10.8%).

Additionally, measures of disciplinary exclusion were mainly based on official records (81%) provided by schools or other official institutions. Table 7 offers a general description of studies’ sample size. The average sample size was M_size=1,168 (SD=3107.3) participants. But this average should be cautiously interpreted since the included studies range widely from 20 to 13,498 participants. This is an important issue for the statistical power of the calculated impacts.

Characteristics of participants in included studies

As observed in Table 8, in the present meta‐analysis, sampled students most frequently come from the higher school grades (i.e., middle and high schools), with a M_age=12.9; SD= 2.8.

In addition, data shows that students were nested in schools with a high percentage of Black and Latino students and, subsequently, with a low percentage of White‐Caucasian peers (24.6%). In fact, five of the included studies are based in schools where a 100% of the pupils were Black (i.e., ref. Barnes, Bauza, & Treiber, 2003; ref. Brett, 1993; ref. Collier, 2002; ref. Mack, 2001; ref. Reese, Murphy, & Filipczak, 1981).

As stated by ref. Hobbs & Vignoles, (2010) and ref. Sutherland, Ilie, & Vignoles, (2015), in this report the percentage of students with access to free school meals was understood as an index of the vulnerability of the population. Across our selected studies the pupils receiving meals funded by the school was predominant (M_fsm =66.2; SD=23.9%).

Intervention characteristics: dosage, delivery and targeted change

The present review includes a wide range of school‐based interventions covering, for instance, strategies such as counselling, mentoring, community services, classroom management, after‐school academic support, and school‐wide strategies, among others. Taking a broad view, 27 % of the interventions were focused on a change at the level of the school or teacher, and a 73% anticipating that a change in pupils’ skills/behaviours could affect suspension/exclusion rates.

As shown in Table 9, school staff (32.4%) or school staff assisted by external facilitators (24.3%) delivered most of the interventions. This means that schools interested in reducing exclusion typically put resources into the intervention process. Most of those delivering the interventions were school psychologists/counsellors (32.2%) and, in two particular cases (5.4%), the intervention was delivered by a police or probation officers, which seems to be an increasing trend in the United States (e.g., ref. Gottfredson et al., 2012; ref. Hirschfield, 2008).

We coded data on the role of the evaluator. We identified four potential approaches taken by evaluators; i) delivering the programme, meaning that the researcher implemented the intervention as well as acting as evaluator (e.g., ref. Harding, 2011); ii) designing the programme, meaning that the evaluator defined or took part in the defining the theoretical base, aims and activities of the intervention (e.g., ref. Cornell et al., 2012), iii) both, designing and delivering the programme (ref. Mack, 2001) and iv) independent evaluation, referring to those researchers or research teams where any member was involved in any stage of the designed or delivery (e.g., ref. Bradshaw et al., 2012; ref. Dynarski et al., 2003). A high percentage of interventions were designed and/or delivered by the same researcher who evaluated the impact of the intervention. Independent evaluators conducted 48.6% of the RCTs included in this review.

A high percentage of the interventions were curricular (67.6%). For instance, social skills trainings or anger management training were based on a pre‐designed curriculum with detailed aims and activities for each session. In the present review, non‐curricular interventions are those targeting school‐wide change or those focused on counselling and individual therapies.

On average, included interventions lasted M=20; SD=11.5 weeks. 37.8% of the interventions lasted less than 12 weeks and an equal percentage were delivered over more than 24 weeks. In general, school‐wide interventions lasted longer in our sample (i.e., 35 weeks or more).

4.1.7 Description of the interventions

Table 10 provides a description of the included school‐based programmes targeting school exclusion as a primary or secondary outcome. In some specific cases this grouping could be restrictive, because some of the interventions involve multiple components, however we have attempted to create an exhaustive list of categories.

Enhancement of academic skills. We found two effect sizes targeting the enhancement of academic skills as a strategy in order to improve academic performance, increase motivation and promote more adaptive behaviour. ref. Edmunds et al., (2012) tested an intervention to boost the academic progress of students in order to facilitate their future access to college while in the case of ref. Cook et al., (2014), the intervention involved academic remediation plus social skills training.

After‐school programmes. Two effect sizes come from interventions that offered students after‐school activities. While the intervention tested by ref. Dynarski et al., (2003) was more focused on academic support and recreational activities, ref. Hirsch, Hedges, Stawicki, & Mekinda, (2011) tested an after‐school programme offering students paid apprenticeships.

Mentoring/monitoring programmes. Five effect sizes reported the impact of interventions focused on mentoring/monitoring. These programmes involved structured and supportive relationships between a young person who presents academic, emotional or behavioural difficulties and a non‐parental adult, their mentor. Mentoring entails a volunteer member of the community serving as a role model and providing support to a younger person over an extended period of time (e.g., ref. Brett, 1993; ref. Wyman et al., 2010). In other cases, adults served as tutors for the students, supervising their performance, providing advice or counselling, and assisting the students with academic tasks (ref. Johnson, 1983; ref. Peck, 2006; ref. Reese et al., 1981). Such tutors were normally schoolteachers or school counsellors.

Social skills training for students. We found nine effect sizes representing the impact of social skills training. These programmes were based on social learning and cognitive behavioural theories (e.g., ref. Burcham, 2002; ref. Collier, 2002; ref. Harding, 2011; Hostetler & Vondracek, 1995; ref. Shetgiri, Kataoka, Lin, & Flores, 2011; ref. Smith, 2004) and their goal is to enhance individuals’ socio‐cognitive, socio‐emotional, and behavioural skills in order to regulate maladaptive conducts. Social skills training programmes typically consist of a curriculum with focused training modules. Some more specific programmes target communication skills (e.g., ref. Obsuth et al., 2016) or approaches to reducing stress (e.g., ref. Barnes, Bauza, & Treiber, 2003). Group‐based sessions and occasionally one‐to‐one sessions (e.g., ref. Russell, 2007) offer the opportunity to implement specific techniques (e.g., instruction, modelling, role‐playing, feedback and reinforcement) in a “real‐world environment”.

Skills training for teachers. We found four independent interventions targeting teachers’ skills. The training provides preventive strategies and techniques that help to maintain classroom discipline, create a supportive educational environment, and enhance students’ positive behaviour. These involve training in facilitating mutual respect between teacher and student (e.g., ref. Okonofua, Pauneskua, & Walton, 2016) as well as training to establish clear classroom rules (e.g., ref. Hawkins, Doueck, & Lishner, 1988). Skills for teachers also involve strategies for working in an alliance with parents to promote students’ engagement to the school activities (e.g., ref. Ialongo et al., 2001).

School‐wide interventions. Six effect sizes represent comprehensive interventions targeting systemic changes across the whole school. They involve pupils, teachers, parents, and sometimes also the community where the school is based. These programmes aim to create positive environments, with clear rules that promote good behaviour, learning and safety. School‐wide interventions are capable of addressing the needs of schools as institutions as well as the particular needs of individual school children. We found six studies testing school‐wide strategies: ref. Bradshaw et al., (2012); ref. Cornell et al., (2012); ref. Lewis et al., (2013); ref. Snyder et al., (2010); ref. Sprague et al., (2016); ref. Ward & Gersten, (2013).

Violence reduction. We included three effect sizes measuring the impact of violence reduction programmes. Although these interventions could be classified as skills training, we have isolated them, because they are specifically targeted at increasing self‐control and reducing violence (e.g., ref. Feindler, Marriott, & Iwata, 1984; ref. Mack, 2001). We also included anger management programmes encouraging peaceful responses to conflict (e.g., Farrell et al., 2010).

Counselling and mental health interventions. We included three effect sizes primarily focused on the provision of counselling in schools (e.g., ref. Berlanga, 2004; ref. Tilghman, 1988)and on a more specialised provision from community mental health services (i.e., ref. Panayiotopoulos & Kerfoot, 2004).

Other interventions. Four effect sizes were classified in this general category. They encompass a community services programme (ref. Allen, Philliber, & Herrling, 1997), a multicomponent programme (ref. Arter, 2005), a career awareness intervention (ref. Bragdon, 2010), and a programme focused on character‐building education, promoting civic behaviour and national values (ref. Crowder, 2001).

4.2 RISK OF BIAS IN INCLUDED STUDIES

The methodological quality of each publication included in the review was evaluated using the EPOC risk of bias tool (see section 12.3). The instrument evaluates the internal validity of reported results. Two coders (SV & AS) independently applied the EPOC tool to each study at different locations. The following results represent the agreed rating of both coders.⁷ Below, we report the results for each of the eight criteria involved in EPOC. Figure 2 shows a summary of the overall result, and Table 9.2 (section 9.2 in appendix), offers a detailed evaluation of each study.

Adequate sequence generation

Adequate sequence generation is intended to produce comparable groups in an experimental evaluation. Accurate methods used to generate the allocation are, for instance, the use of random number tables or a computer random number generator (ref. Higgins & Green, 2011). Regardless of the selected methods, it should be clearly described in enough detail to allow an assessment of whether the sequence generation was adequate.

As shown in Figure 2, only three studies (8%) presented a high risk of bias with respect to allocation sequence. A case in point:ref. Allen, Philliber, Herrling, & Kuperminc, (1997) used three different methods of randomisation (i.e., picking names from a hat, coin toss and using an alphabetical list of names) as well as running randomisation at different levels in the same study (i.e., individuals and classrooms). In our opinion this mixture of methods could lead to a high risk of bias in the final results: in particular, the use of a list of names is known not to be appropriate for allocating cases. In the case of ref. Barnes, Bauza, & Treiber, (2003:2), the author mentions that “each school was alternately assigned to either TM or health education control.” Since the study involves only two schools, allocation would be predictable and at a high risk of bias.

Although we identify a low percentage of high‐risk cases, it is important to note that a large proportion of studies (59%) were categorised as “unclear risk”. This is because many reports presented succinct descriptions of the randomisation process without detailing the methods of sequence generation (e.g., ref. Arter, 2005; ref. Brett, 1993; ref. Feindler, Marriott, & Iwata, 1984; ref. Reese, Murphy, & Filipczak, 1981). 32% of manuscripts were defined as presenting a low risk of bias (e.g., ref. Bradshaw et al., 2012; ref. Hirsch et al., 2011; ref. Lewis et al., 2013).

Allocation concealment

Correct allocation concealment safeguards a rigorous implementation of the randomisation process by not allowing researchers or participants prior knowledge of the results of assignment (i.e., by using sealed envelopes or other procedure that prevent knowledge about the condition that the participant is going to be allocated). To achieve this aim it has been suggested that allocation should be centralised and executed at the beginning of the study (ref. Schulz & Grimes, 2002).

In the present meta‐analysis, only 5% of the studies were evaluated as having a high risk of bias for allocation concealment. Some examples of high‐risk studies are those where reports suggested that schoolteachers instead of researchers performed the random allocation (e.g., ref. Crowder, 2001) or that the randomisation was performed after the participants’ screening (i.e., ref. Mack, 2001). Even if we cannot be sure that the results of the screening biased the randomisation, this conduct could potentially interfere with the chances of participants being placed in the control or treatment group.

Once again, we found a high percentage of studies (54%) reporting minimal details of allocation concealment. In concrete terms, 20 out of 37 included reports were classified as “unclear risk” of bias. The remaining 41% (i.e. 15 studies) were classified as having a low risk of bias.

Baseline equivalence in the outcome measured

A key element of a randomised controlled trial is that it ensures, in theory, that participants (and their associated outcomes and characteristics) are distributed by chance in the control and treatment groups. Pre‐existing baseline differences between groups – particularly of outcomes – could suggest problems in randomisation, hence it is a key focus when assessing risk of bias (ref. Shadish et al., 2002). It is important to mention that we did not focus on whether there were statistically significant differences between groups, as that is a function of sample size; instead we reviewed means and distributions.

We found seven studies (19%) whose description of the baseline equivalence suggested a high‐risk. This was found, for instance, in cases where the control group presented higher levels of exclusion than the treatment group and there was not clear mention of adjustment (e.g., ref. Panayiotopoulos & Kerfoot, 2004). Another example concerns the imbalance of one specific type of exclusion: specifically, expulsion and out‐of‐school exclusion were equivalent in the treatment and the control groups, but display substantial imbalance in the case of in‐school exclusion (e.g., ref. Berlanga, 2004).

Again, it is important to mention that nearly 30% of assessed studies reported limited information, and for that reason they were assigned an unclear risk of bias. The remaining 49% of our included randomised controlled trials were evaluated as low risk of bias.

Baseline equivalence in other participants’ characteristics

Regarding this criterion, the instrument considers the balance in demographics or any other behavioural outcomes, which again should be equivalent in the treatment and the control group if randomisation has been successful.⁸ We found 11 studies (30%) displaying non‐equivalent results between treatment and control groups. Studies reporting imbalance in gender (e.g., ref. Berlanga, 2004), ethnicity (e.g., ref. Hostetler & Fisher, 1997), initial levels of problem behaviour, self‐control or size of the school (e.g., ref. Harding, 2011)were all classified as a potentially having a high risk of bias.

A further 22% (eight studies) did not report enough data for judgement (i.e., unclear risk) and around half (49%) of the studies were assessed as presenting a low risk of bias.

Addressing incomplete outcome data

The fact that our included studies pursued more than one measure across time makes it likely that attrition or other forms of missingnes affected sample sizes.

We assess a high risk of bias when i) substantial attrition was present in the study and the researchers did not mention a strategy to deal with that issue (e.g., ref. Peck, 2006); ii) when they used list‐wise deletion (e.g., ref. Russell, 2007), and iii) in those cases where the attrition affected the treatment or control group in an unequal proportion of missing cases across arms (e.g. ref. Hostetler & Fisher, 1997). Using these criteria, we judged that eight studies (22%) presented a high risk of bias.

As in previous cases, a high number of studies (30%) did not report enough data to be judged. Seventeen studies (46%) were evaluated as presenting a low risk of bias. Low risk cases involved i) those studies reporting zero attrition; ii) studies where attrition was represented by a low percentage of cases; iii) when missingnes was equivalent in the treatment and the control group, and iv) when the researcher reported attrition, analysed it and used methods to deal with attrition (e.g., multiple imputation, full information maximum likelihood or intention to treat rather than assessing the effect of treatment on the treated).

Blinding of outcome assessment

This criterion covers bias arising from the fact that those collecting outcome data are aware of the condition assigned to each participant (e.g. individuals, classrooms, schools). In our evaluation, 35% or 13 of the coded studies were assessed as having a high risk of bias. We assigned a high risk of bias when those in charge of delivering the intervention were also collecting data (e.g., ref. Harding, 2011) or when teachers rated the students’ behaviour while being aware of the allocated condition (e.g., ref. Tilghman, 1988). Although the outcome was in the majority of studies, based on official records, we did not assume this meant blind assessment. In most cases, teachers or school staff who imposed exclusion were not necessarily blind to the allocated condition of participants. In any case, it must be said that in school‐based experiments blinding is likely infeasible. Most of the studies require at least a minimal participation of school staff and allocated condition is mostly evident for participants (ref. Hutchison & Styles, 2010).

As in previous cases, the number of reports lacking the data to evaluate blindness of the outcome assessment (i.e., unclear risk) was high (54%) and only four studies (11%) were assessed as low risk bias.

Protection against contamination

Protection against contamination refers to the measures taken to avoid a spill‐over of the treatment into the control group. Specifically, contamination is the risk that the control group might accidentally receive the intervention, but also the risk that the two groups could influence each other’s outcomes. If the groups cannot remain isolated during the experimental study, contamination would threaten the validity of the results. In our meta‐analysis 76% of the studies presented a high risk of contamination. The main explanation for high risk of bias was the fact that schools in the trials host both the treatment and the control participants (e.g., ref. Cook et al., 2014; ref. Smith, 2004).

One single study (3%) did not report enough data for evaluation and it was categorised as unclear risk of bias. Eight studies (22%) were classified as low risk of bias. Normally low risk studies were cluster‐randomised experiments where control and treatment participants were in different schools (e.g., ref. Bradshaw et al., 2012; ref. Snyder et al., 2010).

Selective outcome reporting

Selective outcome reporting occurs when there is a difference between the proposed outcomes for evaluation and those finally reported. In our evaluation, four studies (11%) presented a high risk of selective outcome reporting (e.g., ref. Panayiotopoulos & Kerfoot, 2004; ref. Russell, 2007). In the case of thesis or trials without published protocols, the assessment was only based on discrepancies between outcomes proposed and reported in those documents⁹. Thirty‐two studies (86%) displayed a low risk of bias and only 3% did not report enough data for judgement.

Summary

Overall, our assessment demonstrates that a high number of the included studies lacked enough information to use the EPOC tool to judge risk of bias in all areas. Specifically, 22 studies were not clear on how they allocated units to treatment and control groups. Similarly, blinding to allocation was another area infrequently reported with enough detail to allow assessment (i.e., in the case of 21 studies). These points aside, contamination or spill‐over is probably the main threat to the validity of results among the RCTs included in our review.

Arguably the seven methodological strongest studies were ref. Bradshaw et al., (2012), ref. Cook et al., (2014), ref. Hirsch et al., (2011), ref. Johnson, (1983), ref. Lewis et al., (2013), ref. Obsuth et al., (2016) and ref. Wyman et al., (2010). All of them presented low risk in the randomisation process and most of them are clustered studies. In particular, these studies achieved low risk of bias in six or more of the eight EPOC criteria.

4.3 SYNTHESIS OF RESULTS

4.3.1 Primary outcome: overall impact of school‐based intervention

The present analysis incorporates 38 effect sizes across 37 studies producing enough statistical information for meta‐analysis. These studies represent a total sample of 31,273 students (M_age =12.5; SD=2.85) partaking in completed trials as treatment, control or placebo groups.

On average, school exclusion was significantly reduced in the experimental group compared with the control group, post‐treatment (i.e.,sixmonths on average). Under a random effects model, the standardised mean reduction was SMD=.30; (95% CI .20 to .41; p<.001). Figure 3 shows that results are positive and statistically significant, meaning that those participating in school‐based interventions were less likely to be excluded than those in the control group. Results exhibit significant heterogeneity (Q=301.3; df= 37; p<.001; I² = 87.7; τ² =.078) which was expected in this meta‐analysis bearing in mind that we include different school‐based programmes, administering different “doses of treatment” with participants from different locations and in different school grades.

As suggested by ref. Lipsey and Wilson (2001, p. 153) the U3 statistic for a SMD= .30 is 62%. This would indicate that 62% of the treatment group is above the median of the control group. Stated differently, if we assume a 50/50 success rate for both groups, the treatment group sees 62% success, versus 38% in the control group.

Long‐term effects

When we isolated studies measuring impact at follow‐up (i.e., 12 or more months after finishing the intervention), benefits of the interventions were less clear. In fact, Figure 4 demonstrate that the effect was reduced by half (SMD=.15; 95%CI ‐.06 to .35) and it was shown to be non‐statistically significant.

To increase precision in our results, we ran a meta‐analysis with a subset of studies reporting both post‐treatment and follow‐up measures. Only seven studies reported short and long‐term effect measures. As observed, the overall effect size at post‐treatment (Figure 5) and follow‐up (Figure 6) are lower than the ones initially reported but they follow the same direction.

At post‐treatment, under a random effect model, the standardised mean reduction was SMD=.21 (95%CI .11 to .30). However, when we ran the meta‐analysis including only the subset of seven studies, the average time involved under “post treatment” was 12 months. It implies that although the overall effect is slightly lower than the general measure reported in Figure 3, the impact lasts longer (i.e., 12 instead of 6 months on average). In the case of the effect at follow‐up, the subset of studies produced and overall impact that was almost null (SMD=.054; 95%CI ‐.04 to .15) and non‐significant. As shown in Table 11, heterogeneity was highly reduced when the analysis with the subset of studies was carried out.

4.3.2 Moderator analysis by type of exclusion

As proposed, we ran an analysis for the different outcome measures: in‐school exclusion, out‐of‐school exclusion, expulsion, and general exclusion. Results of the four independent meta‐analysesare summarised below in Table 12.

In‐school exclusion. Only six trials were concerned with the effect of school‐based intervention on reducing in‐school exclusion. Under a random model, the effect of the intervention remained positive and statistically significant. The final effect described in Figure 7 was SMD=.35 (95% CI .11 to .58; p<.005). Once again heterogeneity was tested, finding significant but smaller variability across studies (Q=11.62; df=5; p<.05; I² =57; τ² =.045).

Out‐of‐school exclusion. Correspondingly, nine studies reported data for the impact of interventions on out‐of‐school exclusion. Figure 8 shows an effect close to zero, non‐statistically significant of SMD=.02 (95% CI ‐.16 to .19; p>.05).

Expulsion. The impact of school‐based interventions on expulsion was significantly higher than any other impact described so far. Figure 9 shows that expulsion was reduced by SMD=.53 (95% CI .07 to .98; p<.05) with significant heterogeneity (Q=13.66; df=3; p<.05; I² =78; τ² =.14) but based on only four reports presenting data for analysis. Therefore, these results must be evaluated cautiously.

General Suspension. Finally, a number of studies presented data on suspension as a broad and general measure. These studies did not describe operational definitions about the type of disciplinary suspensions involved in the outcome. We have defined this category as General Suspension. In the aim of transparency, we report these results separately, although this measure could involve any of the previous outcomes reported above and, therefore, be a subset of the overall effect size reported at the beginning of this section.

Figure 10 shows 27 studies reporting 28 independent effect sizes concerned with the impact of targeted interventions on general suspension. The effect of school‐based interventions was positive SMD= .32 (95% CI .21 to .43; p<.001), it was statistically significant and similar to the overall effect size reported in Figure 3 above. In addition, heterogeneity remained substantial (Q=171.45; df=27 p<.001; I² =84; τ² =.056).

4.3.3 Secondary outcome: overall impact of school‐based intervention on internalising behaviours

As stated in our protocol, for any identified study reporting data on school exclusion, we also coded a secondary outcome referring to internalising and externalising behaviours. Data on internalising behaviour was not very prolific in our set of included studies. Only five trials presented statistical results but sometimes the data was insufficient for effect size calculations. For that reason, we are not running meta‐analysis on internalising behaviours, but we summarise the results in Table 13.

4.3.4 Secondary outcomes: overall impact of school‐based intervention on externalising behaviours

We found a diverse range of measures referring to externalising behaviours, such as substance misuse, violence, aggression, and problematic behaviour in school. To pool together comparable measures, we ran a meta‐analysis only on behaviours that could be categorised as antisocial such as aggression, physical fights, delinquency, bullying and conduct disorder¹⁰.

Fourteen studies reported complete data for a composite measure of antisocial behaviour (See Figure 11). The fourteen studies provided 15 independent effect sizes. Unusually, ref. Wyman et al. (2010) reported a measure of behaviour control (e.g., children accepting imposed limits). We reversed the results as a proxy of antisocial behaviour and included this information in our calculations. The same procedure was followed with ref. Feindler et al. (1984) who reported a measure of increase in self‐control. Once again, we reversed the effects size as a proxy of antisocial behaviour.

As reported in Figure 11, the overall impact of school‐based interventions on antisocial behaviour, under a random effects model, was not statistically different from zero SMD= ‐.005 (95% CI ‐.097 to .09; p >.05) indicating an overall null effect of these programmes at reducing antisocial behaviour.

It must be highlighted that some of the included studies in this overall measure reported negative effect sizes, meaning that in some specific cases the intervention had iatrogenic impact (ref. Hawkins et al., 1988; ref. Hostetler & Fisher, 1997; ref. Obsuth et al., 2016). For instance, ref. Hawkins et al., (1988) reported the impact of an intervention focused on interactive teaching and co‐operative learning targeting low achievers in mainstream schools. The treatment group showed a reduction in the number of exclusions but an increase in the mean value for serious crime. The study reported by ref. Hostetler & Fisher, (1997)described a similar case. For its part, ref. Obsuth et al., (2016) reported the results of a clustered randomized controlled trial testing the impact of “Engage in Education‐London”. The intervention targeted high risk students and it was aimed at improving communication and broader social skills. Results suggest that the iatrogenic effects were observable not only for exclusion but also for the case of antisocial behavior.

It is important to mention that ref. Feindler et al., 1984; as well as ref. Wyman et al., (2010) were reverted and used as proxies of antisocial behaviour. For this reason, they should not be understood as interventions which report iatrogenic results.

4.4 SUB‐GROUP ANALYSIS

A number of potential effect modifiers that could help to explain the expected heterogeneity in our results were initially identified in the published protocol. Based on previous research we pre‐defined moderators that covered four aspects, namely: i) participants’ demographic characteristics; ii) behavioural problems; iii) the theoretical basis of the interventions, and iv) the quality of the intervention. In this section we present sub‐group analysis results. The calculations have been run under a random effects model assuming a separate variance component. Forest plots with further details are reported in Section 11 (appendix).

4.4.1 Effects moderated by participants’ demographic characteristics

Of the 38 effects reported in this meta‐analysis, 11 were tested in schools whose population was predominantly male (i.e., more than 60% of students were male), and 19 effects were tested in schools presenting a mixed population (i.e., neither gender exceeded 60 per cent).

Post‐intervention effects were different for both groups. For studies targeting predominantly male schools, the standardised mean difference was SMD=.41; (95% CI .10 to .72; p<.05). In studies targeting mixed‐gender schools, the impact was lower (SMD=.17; 95% CI .02 to .32: p<.05). Differences between groups were not statistically significant (Q=1.84; df=1; p>.05).

In terms of age, the best proxy variable was school grade. This information was reported in 34 studies and 35 independent effects. To test the hypothesis that effect sizes vary by age, we ran sub‐group analysis for 12 studies involving elementary school students (SMD=.27; 95%CI .09 to .45; p<.05), 16 targeting middle schools (SMD=.23; 95% CI .04 to .41; p<.05), and eight targeting high schools (SMD=.45; 95%CI .18 to .72; p=.001). The effect was statistically significant in each sub‐group and larger in high school populations. However, the between effect difference was not statistically significant (Q=1.81; df=2; p=.41) meaning that there is no evidence that the effect of school‐based interventions differs by age (i.e., grade at school) in the present meta‐analysis.

Ethnicity was reported as a continuous variable (i.e., percentage of each ethnic group). For that reason, the role of ethnicity in explaining the effect of school‐based programmes will be explored in a meta‐regression analysis.

4.4.2 Effects moderated by participants’ behavioural problems

In our review, only a limited number of studies presented data on behaviour (i.e., internalising or externalising behaviours). In addition, the overall impact of school‐based interventions on antisocial behaviour was not statistically different from zero.

Pupils involved in included studies shared a similar high‐risk condition. They were registered in schools with a high percentage of ethnic minorities and more than 60% of the students in those schools received free school meals, which is an indicator of disadvantaged socio‐economic backgrounds. In that sense, participants do not display high variability.

4.4.3 Effect of different school‐based programmes on school exclusion

One of the aims of the present meta‐analysis was to compare the effect of different interventions on the reduction of school exclusion. Table 14 presents the standardised mean differences, confidence intervals and p‐values as well as measures of heterogeneity for each of the ninetypes of programmes included in the review. The typology is the same as described in section 4.1.7.

First of all, as observed in Table 14,most of the interventions are represented by a restricted number of effect sizes, and for this reason, these results should be interpreted cautiously.

Secondly, the standardised mean differences of only five types of programmes present positive (small to moderate effect sizes) and statistically significant results in favour of the reduction of school exclusion. Those programmes are: i) Enhancement of academic skills, ii) Mentoring/monitoring, iii) Skills training for teachers, iv) Counselling/mental health services, and v) Other programmes. Since “other programmes” involve a mixture of different interventions we believe they cannot be interpreted in the same way as the remaining four types. Similarly, when it comes to the number of studies included in each sub‐group, it seems that the most stable results are Mentoring/monitoring and Skills training for teachers since they are based on a larger number of studies.

Thirdly, to test the hypothesis that differences were significant among the compared sub‐types, we ran further analysis. The comparison demonstrates that differences are statistically significant (Q=18.4; df= 8; p<. 05), meaning that variation in effect sizes can be explained by the type of intervention implemented.

4.4.4 Theoretical bases

Reported information on the theoretical bases of the interventions was not very comprehensive. Selected studies described components of interventions more frequently than reporting the theory or set of theories framing the praxis. Based on the provided details, 20 interventions were clearly based on a cognitive behavioural frame, while another five were concerned with ecological ideas, targeting a change of the school system as a whole. The remaining effects refer to different theories such as emotional intelligence (i.e., ref. Smith, 2004), empathy (i.e., ref. Okonofua et al., 2016), civic values (i.e., ref. Crowder, 2001) or developmental theories (i.e., ref. Allen et al., 1997). A number of studies did not provide enough information to make a judgement. We tried to run sub‐group analysis on the theoretical bases of the interventions but it demonstrated low power, given that the number of effects was, in some cases, as low as one per sub‐group. In that scenario, we decided that a measure of the level of targeted change could inform more consistent data on the theory framing the interventions. We then divided the studies into those targeting a systemic change versus those targeting a change in pupils.

The standardised mean difference for the 10 evaluations targeting a change at school level provideda significant reduction with a value of SMD=.25 (95%CI .04 to .45), whereas the 28 evaluations targeting a change at the pupil’s level reported a significant reduction with a value of SMD=.33; (95%CI .19 to .48). Both independent effects are statistically significant; however, the between‐group comparison reported non‐significant differences (Q=.48; df=1; p>.05) meaning that there is no evidence that the effect differs by level of targeted change.

4.4.5 Moderator analysis: quality of implementation

Previous research demonstrates that well‐implemented programmes – those including training, monitoring and supervision – display larger and more consistent effect sizes (e.g., ref. Durlak et al., 2011; ref. Gottfredson & Wilson, 2003; ref. Lösel & Beelmann, 2006). We tested this hypothesis based on two variables, namely, “training before implementation” and “monitoring during intervention”.

Twenty‐five studies (reporting 26 independent effect sizes) clearly stated the presence of training hours before the intervention was delivered (i.e., training hours for those delivering the intervention). In the remaining 12 studies, authors did not mention any kind of training. We ran sub‐group analysis to test the hypothesis that those that reported training could produce a significantly different effect. In the end, those reporting prior training yielded a result equal to SMD=.29 (95% CI .16 to .43; p<.001) whereas those that did not report training produced an effect equal to SMD=.34 (95% CI .15 to .53; p<.001). Both effects were positive and significant, however the test of the difference between the two sub‐groups of studies yielded a result of Q=.16 with df=1 and p>.05, meaning that there is not enough evidence that effects differ by presence/absence of prior training.

Fifteen studies that reported monitoring the implementation of the programme during the trial yielded a result equal to SMD=.20 (95%CI .05 to .35; p<.05). In parallel, the 23 studies that did not report monitoring produced an SMD=.37 (95%CI .25 to .50; p<.001). Both results were positive and statistically significant. The test for differences between sub‐groups showed non‐significant differences (Q=2.89; df=1; p>.05).

4.4.6 Post‐hoc moderators

Based on descriptive analysis of data, and also based on previous research findings (ref. Beelmann & Lösel, 2006; ref. Farrington, Ttofi, & Lösel, 2016), we selected three post‐hoc moderators, namely: i) reasons for conducting the research; ii) evaluator role, and iii) risk of quality bias. Their results are described below:

Reasons for conducting the research. We coded data related to the reasons for conducting research. Two categories are explored: i) demonstration study, referring to those studies testing the impact of an intervention under highly controlled optimal conditions and ii) routine evaluation, focusing on testing established programmes under circumstances that approach real‐life conditions (ref. Singal, Higgins, & Waljee, 2014).

Sub‐group analysis produced different effects for the 16 studies conducted for demonstration (SMD=.43; 95% CI .26 t0.59; p<.001), and the 18 studies carried out for routine evaluation (SMD= .13; (95% CI .00 to .25; p<.05). Demonstration studies reported the highest effect. The between‐studies comparison results were significant (Q=8.15; df=1; p<.05), meaning that the effect varies depending on the reasons for conducting the research.

Evaluator role. We coded data identifying the role of the evaluator. Independent evaluators were those not taking part in the design or implementation of the evaluated programme. Dependent evaluators were those also contributing to the design and/or the implementation of the programme.

The 18 trials carried out by independent evaluators produced an SMD= .13 (95% CI .00 to .25; p<.05). Unsurprisingly, based on what is known about developer‐led trials, the 16 RCTs conducted by dependent evaluators (i.e., those who also developed and/or designed the intervention) yielded results where SMD=.47 (95% CI .32 to .62; p<.001). The between‐groups comparison demonstrated that effect sizes are lower for studies conducted by independent evaluators. These differences were statistically significant (Q=12.36; df=1; p<.001).

Risk of quality bias. As reported in section 4.2, we ran EPOC tool for assessing risk of quality bias. Results show that studies vary in the amount of information available for judging their level of risk as well as their risk of bias. We speculated that the quality of studies could explain some of the heterogeneity present in our results. In order to use the results reported by EPOC, we transformed the “low risk”, “unclear risk” and “high risk” into continuous 0, 2 and 3 values respectively. The sum of the results of the eight EPOC criteria resulted in an index that, in our case, was used as a representation of the level of risk of each single study. We tested the role of RoB in meta‐regression.

4.5 META‐REGRESSION

In order to explore heterogeneity, we ran meta‐regression using the moderators defined a priori (participants’ characteristics and intervention characteristics). Model I included participants’ characteristics only, to get a sense of the net value of these variables in the results. Students’ gender, grade at school or ethnicity (continuous variable), as reported in Table 15, did not explain heterogeneity in the study results. In model II, we introduced the intervention characteristics (i.e., interventions targeting a change at the individual level versus those targeting a change at the school level). Once again, none of the variables present significant results. This suggests that based on the present data, variability across effect sizes cannot be explained by the a priori defined modifiers.

We proceed with a meta‐regression model involving post‐hoc variables that could potentially explain the heterogeneity present in our results. Based on the descriptive results in section 4.1.6, and bearing in mind some previous findings, we hypothesise that the characteristics of the research, the ends of the research (demonstration versus routine), the role of the evaluator, and the risk of quality bias could play a role in explaining the results (see Table 16).

The results were significant (p<.05) only for the variable role of the evaluator. The coefficient is negative with the category of reference “dependent”, meaning that the effect is lower when an independent team runs research.

Figure 12 presents the value of R‐squared or percentage of explained variance between‐studies based on study‐level characteristics. To compute the total variance (of all studies about the grand mean), we run the regression with no covariates (a). To compute the variance not explained by the model (of all studies about the regression line), we run the regression with covariates (b). Finally, the difference between these values gives us the variance explained by the model which is R²=.58, meaning that the model explains 58% of the between‐studies variance.

4.6 PUBLICATION BIAS ANALYSIS

Publication bias in systematic reviews occurs when the included set of manuscripts fail to systematically represent the whole population of completed studies that should have been included. The whole population of studies can involve a range of results that must be present in a meta‐analysis to make it valid. However, consistent evidence indicates that studies presenting large effects are more likely to be published than those presenting null or modest effects (Rothstein, Sutton, & Borenstein, 2006). This means that publication bias can lead to an overestimation of the impact of a treatment when running meta‐analysis.

In the present study, much effort was spent in finding most complete collection of published and unpublished studies that test the impact of school‐based intervention on exclusion rates. In fact, almost 50% (see Table 7) of our included studies have never been published in books or peer reviewed journals (i.e., PhD thesis, technical or governmental reports).

As originally proposed, we use statistical procedures to quantify potential bias that could affect our analysis. First of all, we produce a funnel plot of standard errors by standardised differences in means, presented in Figure 13. In theory, symmetrical distributions of dots under the funnel represent a normal distribution of studies. As anticipated, our studies mostly fall under the funnel, and they are distributed around the main effect. However, since the evaluation of funnel plots can be subjective, we conducted additional statistical measures of publication bias, specifically, Duval and Tweedie’s trim‐and‐fill analysis.

Duval and Tweedie’s trim‐and‐fill analysis compares the differences in effect sizes that could potentially be attributed to bias. The technique imputes effect sizes until the error distribution gets close to normality. In this way, the test offers the best estimate of the unbiased effect (ref. Borenstein et al., 2009). Results of Duval and Tweedie’s trim‐and‐fill analysis suggest that there were no differences in effect sizes attributable to bias. Under the random effect model, the values were SMD=.30; 95%CI .20 and .41. Based on the parameter of Duval and Tweedie’s trim‐and‐fill, it seems that no studies are missing.

4.7 SENSITIVITY ANALYSIS

In the present meta‐analysis, the effect of the treatment was calculated as the difference between post‐treatment and baseline. We corrected the value of the variance by assuming a value of correlation equals to .75. In order to test the robustness of this assumption, we ran sensitivity analysis with a correlation equal to .50. Table 17, panel A shows that overall results remain stable when the correlation is smaller.

Another decision was related to the presence of outliers. We found one study presenting an effect size more than three standard deviations from the mean effect size, which was defined as an outlier (ref. Collier, 2002). We tested the impact of the outlier and also the impact of winsorization. The size of the effects, their direction and significance were not altered. See Table 17 below, panels B and C.

Finally, as stated in the protocol, we ran a sensitivity analysis to test differences between published and unpublished reports. The 20 independent effect sizes reported in 19 peer‐reviewed journalsyield a SMD=.31 (95%CI .17 to .45) whereas the 18 effect sizes extracted from unpublished reports reported a SMD=.29 (95%CI .11 to .47). The between effect difference was not statistically significant (Q=.028; df=1; p>.05).

5. Discussion

5.1 SUMMARY OF MAIN RESULT

In the following paragraphs we present a summary of our results based on the research questions leading this review.

• Do school‐based programmes reduce the use of exclusionary sanctions in schools?

The analyses reported in previous chapters suggest that school‐based interventions are capable of producing a small and significant (SMD=.30; 95% CI .20 to .41; p<.001) drop in exclusion rates. It means that those participating in interventions are less likely to be excluded than those allocated to control/placebo groups. These results are based on measures of impact collected on average, six months after treatment. When the impact was tested in the long‐term (i.e., 12 or more months after treatment), the effect of interventions was not sustained. In fact, the impact of school‐based programmes showed a substantial reduction (50%), and was no longer statistically significant.

We ran a further analysis including only the seven studies which reported post‐treatment and follow‐up measures. The overall impact of school‐based interventions at post‐treatment was SMD=.21 (95%CI .11 to .30). Even if the impact was slightly reduced, when we included those seven studies the average time involved under “post treatment” was 12 months on average, meaning that this measure of impact would be more precise and the impact would last longer. Coherent with the original results, the effect produced by combining the seven studies only was null and non‐significant in the long term (i.e., follow‐up measures).

In addition, our results suggest that school‐based interventions present different levels of impact depending on the type of exclusion. After running moderator analysis by type of exclusion, the evidence across studies demonstrated that school‐based interventions are more effective at reducing in‐school than out‐of‐school exclusion. The impact of programmes in the latter case was close to zero with p>.05, meaning that the result could have arisen by chance. Moreover, a larger impact was observed in the sanction expulsion, with a moderate and significant effect size (SMD=.53; 95%CI .07 to .98; p<. 05). However, these results must be cautiously interpreted due to the low number of manuscripts involved in the calculations.

We found 28 studies lacking an operational definition of the dependent variable. The studies did not specify the type of exclusion or exclusion tested, and in that scenario, we decided not to assume this general measure was out‐of‐school exclusion. The impact of school‐based interventions in those “general suspensions” was similar to the overall impact, which probably suggests that these studies are simply a subset of the overall measure.

As originally planned, in this review we ran independent meta‐analysis testing the impact of school‐based interventions ona secondary outcome, that is, internalising and externalising behaviours. It was hypothesised that a reduction in exclusion would be linked with variations in students’ behaviours.

In the case of externalising behaviours, we were not able to calculate a pooled effect size. Results suggest that only five out of the 37 included studies reported a measure for internalising behaviours. However, the narrative description suggests that school‐based intervention had a small effect on the reduction of the above symptoms.

We ran a meta‐analysis based on 15 effect sizes reporting the impact of school based intervention on externalising behaviours. Results show negative and non‐significant impact (SMD=‐ .005; 95%CI ‐.09 to .086; p>.05). It therefore follows that interventions aimed at reducing exclusion do not necessarily reduce antisocial behaviour. This could be in line with evidence suggesting that changes in school policies, rather than changes in behaviour, produce a reduction in disciplinary exclusion (ref. Noltemeyer & Fenning, 2013; ref. Skiba et al., 2015). Two questions arise from these results. First, how do we interpret them, and second, is it still worth implementing programmes that show no impact on antisocial behaviour?

Regarding the first question, one could hypothesise that if race, or more precisely being a racial minority in a given country, is a stronger predictor of school exclusion than other demographic and behavioural characteristics (as shown in Section 1.2), antisocial behaviour would not necessarily explain the rates of exclusion. What is more, it would not be the main cause of this punishment being used. A second possibility is that the behaviour outcome was not measured properly.

The second question considers whether the school‐based interventions tested in this review are still worth implementing if they do not reduce antisocial behaviour. The evidence produced by the present review shows that those programmes presenting positive results do not report negative side effects, in the sense that control groups do not increase the rate of punishment. If that is the case, we suggest that these efforts should be continued. However, based on our data, these ideas cannot yet be regarded as conclusive and more research is needed about the causes that explain the phenomenon of exclusion.

• Are some school‐based approaches more effective than others in reducing exclusionary sanctions?

The nine different types of interventions presented in the set of included studies were compared to test the hypothesis that some are more effective than others. There was a considerable variability across the programmes in overall effectiveness. Enhancement of academic skills, mentoring/monitoring, skills training for teachers and counselling/mental health services reported the largest and most significant effect sizes. Based on the number of studies included in each sub‐type, we believe that skills training for teachers and mentoring/monitoring represent the stronger and more reliable findings. The subgroup analysis reported that differences across types of programmes were significant, implying that effects vary depending on the programme implemented.

In line with previous findings, Tolan et al. (2008) examined 39 experimental and quasi‐experimental studies and specifically analysed the effectiveness of mentoring in reducing delinquency, aggression and drug use, and improving academic achievements. The largest positive effects were found for aggression (SMD = .40; 95%; CI = 0.06 to 0.74) and delinquency (SMD = .23; 95% CI .11 to .36). The meta‐analysis conducted by Eby, Allen, Evans, Ng, and ref. Dubois (2008) also found that mentoring programmes had a small and significant positive effect on academic performance.

Overall, mentoring programmes seem to be an effective strategy for reducing violence and conduct problems during adolescence. However, on average the effect sizes reported in relevant studies are small. In this respect, it is important to bear in mind the findings by Tolan et al. (2008). This study suggests that mentoring was more effective when (a) participants had greater pre‐existing behavioural problems or had been exposed to significant levels of environmental risk, (b) they were male, (c) the educational or occupational backgrounds of the mentors fitted the goals of the program, (d) mentors and youths were successfully paired, with similar interests, and (e) programmes were structured to support mentors.

As far as teachers’ skills for managing students’ behaviour is concerned, previous evidence suggests that such programmes canimprove teachers’ general instructional and behavioural management skills in planning, implementing and maintaining effective classroom practices. In fact, a meta‐analysis conducted by Gottfredson, Wilson, and ref. Najaka (2002) found that classroom or instructional management programmes (k=25) demonstrated a small and significant effect in reducing antisocial/aggressive behaviour (SMD= .13, p<.05). The most recent meta‐analysis carried out by ref. Oliver et al. (2011), concluded that classroom management practices (N=12 studies) had a significant, positive effect on reducing problem behaviour. Students taking part in the intervention display less disruptive, inappropriate and aggressive behaviour in the classroom compared to those in control classrooms. The overall effect of the intervention was g= .22; p<.05 (ref. Oliver et al., 2011). Evidence also suggests that programmes with the most positive effects tend to be of a longer duration and tend to combine classroom and instructional management strategies with some other major ingredient (e.g. parent training or social skills instruction) (ref. Gottfredson et al., 2002).

The positive relationship between teachers and students has been found to be a factor in promoting more prosocial and less aggressive behaviours later in life. A recent study (ref. Obsuth et al., 2016; p. 16), using a non‐bipartite propensity score matching technique, found that “teachers who reported having a more positive relationship with a student at age ten observed significantly fewer aggressive and defiant behaviours and more prosocial behaviours in the same student concurrently and one year later, at age 11. This was also associated with more prosocial behaviours two years later, at age 12 and also with less aggressive behaviour at age 13. Similarly, students who perceived a more positive relationship with their teacher at age 11 reported fewer aggressive behaviours and more prosocial behaviours concurrently and fewer aggressive behaviours two and four years later, at ages 13 and 15.” All these results make us believe that investing in teachers’ skills and positive relationships between students and teachers is worthwhile, with schools becoming target locations for preventing crime and promoting positive psychosocial development.

Finally, it is worth mentioning that we grouped interventions by those targeting a change at student level versus those expecting a change at school level. Even if the programmes targeting a change at student level display a larger effect size, the between‐group analyses do not allow us to conclude that the differences between these intervention strategies are statistically significant.

• Do participants’ characteristics (e.g., age, gender, ethnicity) affect the impact of school‐based programmes on exclusionary sanctions in schools?

Differences in post‐intervention effects were non‐significant when we compared schools whose population was predominantly masculine versus mixed schools. Thus, based on our findings, we could not confirm the hypothesis that the effect differs by the distribution of gender in schools. The same occurred with ethnicity and age. As both variables were continuous, we included them in our meta‐regression. Based on the available data, we could not confirm the hypothesis that the impact of the interventions varies when the intervention was implemented with younger or older students. Similarly, variation in the percentage of white students did not play a role in explaining the heterogeneity of the effect.

• Do characteristics of the interventions, implementation and methodology affect the impact of school‐based programmes on exclusionary sanctions in schools?

Twenty‐five studies (reporting 26 independent effect sizes) stated the presence of training hours before the intervention was delivered. In the remaining 12 studies, the authors did not mention any kind of training. When comparing the effect of both sets of studies, the between‐group analysis was non‐significant, meaning that there is no evidence that effect differs by presence/absence of training before implementation.

We ran an analysis to investigate the differences between those interventions that deployed monitoring during implementation and those which lacked monitoring. Interventions without monitoring produced significantly larger effects than those with it. In other words, programmes lacking monitoring produced larger effect sizes. This result is in line with previous findings indicating that a lack of monitoring, for instance in routine trials, tends to show smaller effects than demonstration evaluations (ref. Farrington, Ttofi, & Lösel, 2016).

On the other hand, we observed that interventions run by independent evaluators found significantly smaller effect sizes when compared to those studies carried out by researchers involved in the design and/or delivery of the programme. Again, this is not surprising. Previous research has called attention to this phenomenon as well as the advantages of running impendent trials for producing more precise and realistic results (e.g., ref. Eisner, 2009; ref. Manuel Eisner & Humphreys, 2012; ref. Lösel & Beelmann, 2006b; ref. Petrosino & Soydan, 2005; ref. Wilson, Lipsey, & Soydan, 2003).

5.2 OVERALL COMPLETENESS AND APPLICABILITY OF EVIDENCE

In the present review we screened a total of 42,749 citations across 27 electronic databases. Pre‐defined searches targeted published and unpublished reports from any country, in any language, as long as the abstract was presented in English. Only one study in a language other than English was identified by our searches. We purposively conducted electronic searches in databases involving manuscripts produced in Latin America and other Spanish and Portuguese‐speaking countries (e.g., SciELO‐Scientific Electronic Library Online). However, we did not find any evaluations whose characteristics make them includable in our review.

When compared to other contemporary reports (e.g., ref. Dymnicki, Weissberg, & Henry, 2011; ref. Mytton et al., 2006), our searches seemto be comprehensive enough to reduce the risk of publications bias. All in all, the extent of our searches and the high percentage of retrieved full reports make us confident that we have identified a substantially complete collection of the available relevant research.

However, this systematic review has allowed us to identify some gaps in the literature. First of all, studies testing the impact of school‐based interventions fail to disentangle the impact on different forms of disciplinary exclusion. In future, the availability of data calculating the impact of interventions across different types of exclusionary punishment could offer more detailed evidence in respect to what works in the field of school discipline.

Secondly, we did not find includable studies testing the impact of restorative justice strategies. One single ongoing study (i.e., ref. Acosta, 2015) was detected by our searches but its results are to be released in 2018. As recently stated by ref. Fronius et al., (2016: 17 and 19), the research evidence to support restorative justice in schools is still in a nascent state. In fact, the scarce evaluations produced so far rest in pre‐post designs lacking control comparison groups, which would likely have a serious impact on the internal validity of the empirical results. The potential for restorative principles to be applied in school settings needs to be explored in more detail.

Thirdly, we believe that the production of independent, high quality evaluations could contribute to more transparent and precise evidence regarding the impact of school‐based programmes. Previous research shows that demonstration programmes or those where designers take part in the evaluation tend to produce larger effect sizes (ref. Beelmann & Lösel, 2006).

Fourth, much of the evidence presented in this review has been produced in the United States where school safety and exclusion is a salient concern for researchers and policy makers alike. As shown in Table 1 of the present report, in a sample of high‐ and middle‐income countries, exclusion is a widespread school punishment. In European countries, exclusion is a more regulated sanction, while in some Latin‐American societies exclusion is under the discretional decision of each particular school (e.g., Chile). We think that research on the use of exclusion in those contexts needs more development.

Fifth, we observe a lack of research testing the long‐term impact of school‐based interventions. The evidence shows that the impact immediately after intervention tends to be larger than that measured several months later (ref. Farrington, Ttofi, & Lösel, 2016). Since in real life policy makers need to invest in effective programmes whose results endure long‐term, we think that an effort should be made to test the impact at least 12 months after intervention ends.

Finally, although we did not include cost‐benefit analysis in our aims, data was coded regarding the presence/absence of interventions’ costs. Two of our included studies reported the economic cost of programme implementation. We believe that cost‐benefit analyses are key for alerting decision makers about the advantages of early intervention. The prevention of exclusion and expulsion from school can reduce future social exclusion, violence and other negative correlates described in the research (ref. Noltemeyer, Ward, & Mcloughlin, 2015).

5.3 QUALITY OF THE EVIDENCE

The present review involves only randomised controlled trials. They are considered the best methodological design for isolating confounding factors and producing an accurate measure of intervention effects. However, as stated earlier in the results section, our included reports are lacking a considerable amount of information for judging the quality of the procedures carried out.

More than 50% of the reports fail to provide enough data to judge the precision of randomisation, that is, sequence generation and allocation concealment. Likewise, 30% of the studies did not report data on the equivalence of the groups after randomisation. As recently stated by ref. Roberts & Ker, (2016), missing data on those details can drive analysts to identify “false positive” RCTs. We cannot claim that was the case among our studies, but clearly the absence of detail imposes serious limitations on the assessment of quality in regard to potential threats to internal validity.

A similar lack of data affected the evaluation of blind outcome measures. Even if most of the exclusion measures in our report are based on official records, we cannot ignore the fact that teachers or school staff are in charge of imposing sanctions and could potentially be aware of a student’s participation in the experiment. More than 50% of the studies present few details for judging the level of risk involved in this criterion.

Randomised controlled trials of educational programmes are receptive to contamination (ref. Hutchison & Styles, 2010; ref. Simmons et al., 2015). As participants in the control arm may be surrounded by those receiving the treatment, because they share the same school, the likelihood of contamination is a matter of concern. Some of the effects of contamination have been identified by previous research (ref. Giraudeau & Ravaud, 2009; ref. Torgerson, 2001).

Contamination may reduce the effect of the intervention, leading to a type II error – that is, incorrectly retaining a false null hypothesis (a “false negative”) or, described in a different way, a “rejection of an effective intervention as ineffective because the observed effect size was neither statistically nor clinically significant” (ref. Torgerson, 2001). In the present meta‐analysis, 76% of the studies suggest a high risk of contamination bias. It could be the case that our estimation of the impact is underestimated.

In future research, it seems that the use of cluster randomised control trials would help ameliorate these biases. Even if clustered RCTs involve complex designs and demand huge efforts, the quality of the much‐needed evidence can be enhanced by using this methodological design (ref. Hutchison & Styles, 2010). However, the randomisation of clusters implies challenges for researchers. “The main issue is that observations from the same cluster are more similar than observations from two different clusters. This situation requires the use of both an inflated sample size and adapted statistical analysis to take into account this concern” (ref. Giraudeau & Ravaud, 2009, p. 1).

5.4 LIMITATIONS AND POTENTIAL BIASES IN THE REVIEW PROCESS

There are several limitations that could affect the results of the present review. It is important to acknowledge that even though we focused on randomised controlled trials, which are supposed to be the best evidence for measuring evaluation impact, included studies present limited information for judging quality bias.

Some 35% of the included studies reported results based on samples with less than 100 participants. The small size of the samples involved in some of the primary research could impose clear limitations on the ability to estimate the effects of interventions.

As reported in the moderator analysis, the independence of the evaluator explained the heterogeneity of effect sizes. Even if independent teams reported on a good number of our studies, close to 50% of them did not use independent evaluators. This fact could add some bias in to our results.

However, it is also important to elucidate some of the advantages of the present study. First of all, this systematic review and meta‐analysis is the first attempt to collect and statistically summarise interventions pursuing a reduction in school exclusion. As such, we believe that our report offers an overview of the amount, characteristics, limitations, and quality of the available evidence, as well as a measure of the size of the effect achieved by different types of intervention. The reader can find in this report an updated review of the evidence produced in the United States and the United Kingdom.

Secondly, we have endeavoured to use an exhaustive coding process to provide careful descriptions of the studies as well as a meticulous analysis of the statistical data available. Due to the inclusion of cluster data, corrections were introduced to make the information comparable at an individual level. Such corrections were carried out using the most recent strategies suggested by ref. Hedges, (2007) and Pigott, (n.d.). We therefore believe that our calculations are at a low risk of underestimating the size of standard errors.

5.5 AGREEMENTS AND DISAGREEMENTS WITH OTHER STUDIES OR REVIEWS

The present review is the first meta‐analytical effort to identify the impact of school‐based intervention at reducing school exclusion. In that sense, we have no similar evidence with which to contrast our findings.

However, the results provided by our review seem coherent with other studies testing the impact of school‐based intervention on behavioural outcomes. For instance, ref. Wilson et al., (2011) conducted a meta‐analysis on school‐based programmes looking for a reduction in drop‐outs. Our results more or less target similar programmes and the detected effect size follows a similar trend. In the same vein, ref. Mytton et al., (2006) conducted another review testing the impact of school‐based interventions in preventing violence. They found similar problems with quality assessment and reported an overall effect size slightly better than the one produced by the present review. However, in that last case, the long‐term effects are statistically stronger and last longer. The reduction in aggression reported by ref. Mytton et al., (2006) is not clear in our results.

In a more general examination, a recent review of reviews conducted by ref. Farrington, Ttofi, & Lösel, (2016) found a mean effect¹¹ of school‐based programmes equal to SMD=.184 (95%CI .16 to .20). Across the studies, effect sizes ranged between SMD=.091 to SMD=.631. Our findings are, therefore, more or less coherent with the impact of interventions in school settings evaluated by previous meta‐analysis.

6. Authors’ conclusions

The empirical evidence produced by this report suggests that non‐punitive school‐based programmes can reduce the use of exclusion. Even if the effects are not sustained for the long term, data shows that in the short term (i.e., six months on average) schools can opt for different and more effective approaches to managing discipline, rather than zero‐tolerance policies. This review, aimed at testing the effectiveness of school‐based programmes, offers a broad overview not only of the effectiveness of different interventions, but also uncovering findings that can guide public policy and future research.

6.1 IMPLICATIONS FOR PRACTICE AND POLICY

Research consistently reveals that school exclusion is disproportionately used as a punishment for ethnic minorities, males and those coming from low socio‐economic backgrounds. Exclusion as a disciplinary school measure seems to reduce school attendance, increase drop‐out rates and restrict future possibilities for inclusion in the labour market. Aside from that, research has also been consistent in finding a correlation between permanent exclusion and antisocial behaviour.

Though the causal link between exclusion and the above outcomes is still not clearly established, research based on observational data suggests that the consequences of exclusion affect not only students and their families but extend to the whole society. Indeed, as stated by ref. Rumberger and Losen (2016), exclusion may involve a high economic cost to the taxpayer. In their most recent report, the authors provide a conservative estimate of the economic impact of exclusion. They assert that, in the United States, tenth grade exclusions alone account for more than 67,000 school drop‐outs, at a cost of $35 billion.¹²

Although our results must be cautiously considered, the evidence produced by this review suggests that school administrators and policymakers do have alternatives to exclusion when dealing with disciplinary problems. In our findings, prevention strategies have a small but encouraging impact on exclusion rates, at least in the short term. When comparing different types of programmes, it appears that prevention programmes targeting teachers’ skills as well as those introducing mentoring/monitoring schemes can have a positive impact in reducing exclusion. These results are in line with previous research and also with previous narrative reviews which emphasize the importance of teachers’ skills and mentoring programmes in promoting prosocial behaviours and values (ref. Freiberg & Lapointe, 2006; ref. Gottfredson et al., 2002; ref. Oliver et al., 2011; Tolan et al., 2008).

It is important to clarify that this is most applicable to the United States, where the majority of the assessed evidence was collected. As expressed in the previous paragraphs, some flaws affect the contemporary primary evaluations present in this review. However, these results should encourage researchers to produce better quality evidence rather than abandoning their efforts to find strategies to replace exclusionary punishments.

6.2 IMPLICATIONS FOR RESEARCH

Clearly, more primary research evaluating the impact of prevention programmes targeting school discipline is needed. Ideally this research should be conducted under high methodological standards, accounting for mediating mechanisms that lead to a reduction in exclusion rates.

In particular, the results of this first review on the topic identify some implications for future research, laid out below:

• Addressing the racial gap. Most of the literature reviewed in the present study indicates that racial or ethnic identity plays a central role in predicting school exclusion, even after controlling for demographic and behavioural variables. More research needs to be developed for testing the mechanisms that produce different treatment for some racial minorities, such as African‐American and Latino students in the United States or Black Caribbean and Gypsy/Romany students in the United Kingdom. It is necessary to understand the individual and social processes that lead to the overrepresentation of minorities in school exclusion. As suggested by some scholars, the divergence between students and teachers’ cultural expectations could potentially contribute to misinterpretation of each other’s behaviour, fears and conflicts (ref. Gregory et al., 2010). Understanding the mechanisms that make race a predictor of exclusion could have implications for future policy and practice. For instance, as stated in previous research, greater diversity among staff could be helpful in promoting understanding of cultural differences and reducing bias (ref. Irvin, 2002).
• Causal effects of exclusion. The present research has addressed the lack of certainty about the genuine causal effects of exclusion. A review of previous research suggests that exclusion could be a risk factor for a set of negative outcomes already described in the above paragraphs. At the same time, an overview of previous criminological theories suggests that punishment could present different effects on different individuals. The truth is that the causal effect of punishment on students’ behaviour is still a long way from being fully understood. If school exclusion is simply a marker of an underlying antisocial syndrome, it could be beneficial to invest in prevention programmes targeting, for example, deviant behaviours (e.g., violence, drug use, crime, abuse, and neglect) and personality features (e.g., aggressiveness, lack of empathy, lack of remorse) associated with antisocial syndrome (ref. Farrington, 2003). However, if school exclusion is proved to be the cause of detrimental outcomes later in life, it will be worth investing in more programmes focusing specifically on the reduction of school exclusion.
• Mediating mechanisms that explain reductions/increases in exclusion rates. Future evaluations of school‐based interventions aimed at reducing exclusion need to explore the presumed causal mechanisms that lead to that reduction. More theoretically informed trials that examine causal mechanisms are important to design better interventions. In fact, this information would be crucial in planning future prevention programmes since causal mechanisms can shed light on what works for whom and under what conditions.
• Recognise the key role of statistical power. Future research could benefit from prospective power analysis. That means that, during the early stages of the evaluation’s design, there must be consideration given to the sample size required to detect an effect. This early effort can protect studies from “underpowered or wastefully overpowered” samples which can affect outcomes by Type I and Type II errors (ref. Ellis, 2010). Future meta‐analysis should also probably give more attention to this issue and, as suggested by ref. Farrington et al., (2016), it seems advantageous to set a minimum sample size for inclusion in reviews.
• Tackle the challenges of running randomised controlled trials in school settings. Research conducted within schools often struggles to isolate experimental and control groups, with the subsequent risk of contamination. As stated earlier in this report, the use of cluster randomised controlled trials can present an opportunity for tackling this when conducting research in educational settings.
• Attempt blind assessment in school‐based randomised controlled trials. The characteristics of schools make it hard to blind all those involved in trials. Teachers, students and school counsellors are likely to be involved in the experiment, making it difficult to control the social desirability effect of those participating in the study (ref. Hutchison & Styles, 2010). This challenge needs to be addressed by future studies by at least blinding those who collect data. It also seems necessary, at the level of meta‐analytical studies, that the tools used to measure quality bias in school contexts be adapted. In our experience, some of the available “risk of bias tools” seemed more suited for medical trials than for school‐based experiments.
• Risk of quality bias. Meta‐analysis integrates the quantitative evidence from different but related studies to summarise a whole body of knowledge on a research question. As in the present review, meta‐analysis can answer the question about the effectiveness of a given intervention. By the fact that meta‐analysis combines results from different primary research reports, quality bias involved in primary research can jeopardise the validity of the “meta” results. In brief, quality bias refers to systematic error, “meaning that multiple replications of the same study will produce different effect estimates because of sampling variation even if they would give the right answer on average” (ref. Higgins & Green, 2011, p.188).
• As stated by ref. Higgins & Green (2011, p. 189) “more rigorous studies are more likely to yield results that are closer to the truth”. Differences in the quality of performed studies can result in false positive conclusions when less rigorous studies are biased toward overestimating an intervention’s effect. They can also arrive to false negative conclusions in those cases where less rigorous studies are inclined towards underestimating an effect.
• To evaluate risk of quality bias, the present study involves an evaluation of each of the included reports. Three categories were used to judge a study report; i) low risk; ii) unclear risk and iii) high risk of bias. When it came to randomisation, we found a small percentage of high‐risk cases. However, it is important to note that a large proportion of studies (59%) were categorised as “unclear risk”. This is because many reports presented succinct descriptions of the randomisation process without detailing the methods of sequence generation for instance. When we tested the quality of allocation procedures, once again, we found a high percentage of studies (54%) reporting minimal details of allocation concealment. Similar findings were observed for baseline equivalence (22% of assessed studies reported limited information), attrition (30% unclear risk) and blindness of the outcome assessment (54%). These findings do not necessarily mean that studies present quality bias, they just inform us that the authors omitted data which is relevant to interpret the results. Future research would benefit from following CONSORT standards. Registration of trials, even if they are PhD theses, would represent a huge benefit not only for the scientific community, but also for those interested in evidence‐based decision making.
• Cross‐cultural research. Based on our findings, the evidence so far is largely coming from the United States. More research needs to be done in other countries where school exclusion is an issue. We know that evidence suggesting effective approaches in some countries/cultures will not necessarily have the same effectiveness when translated to different populations. Those making decisions about how to reduce exclusion in their own country need to have access to detailed information addressing their particular needs.
• Innovative strategies. More research needs to be conducted on innovative strategies, for example those involving empathy‐based philosophies. This was the basis for the intervention tested by ref. Okonofua et al. (2016, p.5521) included in this review. The intervention is focused on encouraging teachers to adopt an empathetic attitude towards discipline; it is low cost and demonstrates long‐term effects (rates from 9.6% to 4.8%). As stated by the authors, “teachers’ mind‐sets about discipline directly affect the quality of teacher‐student relationships and student exclusions and, moreover, can be changed through scalable intervention”.
• Restorative justice programmes. In addition to the above, the debate about strategies for reducing exclusion in schools have recently been enriched by some research suggesting that restorative justice programmes could have a promising impact (Anyon, Gregory, Stone, Farrar, & Downing, 2016; ref. Fronius et al., 2016). Scholars have highlighted the advantages of restorative justice over exclusion by showing that these programmes focus on building peaceful and empathetic relationships, ask for the involvement of all parties in achieving conflict resolution, look at the harm done to those affected by misbehaviour, and promote reintegration rather than exclusion (Drewery, 2004; ref. Hostetler, 2014; Varnham, 2005). However, to date, there is no clear evidence about its effectiveness (ref. Fronius et al., 2016). Empirical research needs to be conducted testing the impact of restorative justice in school settings. Two challenges need to be addressed here. First, restorative programmes need to be implemented following the guidelines and characteristics of the restorative processes. This is not a simple task for schools used to dealing with conflict in a punitive way. Secondly, randomised controlled trials or well‐controlled studies need to test the impact of restorative justice practices in dealing with conflict inside schools, such as bullying or other forms of violence (ref. Sherman & Strang, 2007). A potential impact in the reduction of conflict could lead to a reduction in the use of exclusion or other exclusionary strategies.

8. Information about this review

8.1 REVIEW AUTHORS

Lead review author:

The lead author is the person who develops and co‐ordinates the review team, discusses and assigns roles for individual members of the review team, liaises with the editorial base and takes responsibility for the on‐going updates of the review.

8.2 ROLES AND RESPONSIBILITIES

Please give brief description of content and methodological expertise within the review team. The recommended optimal review team composition includes at least one person on the review team who has content expertise, at least one person who has methodological expertise and at least one person who has statistical expertise. It is also recommended to have one person with information retrieval expertise.

Who is responsible for the below areas? Please list their names:

• Information retrieval and coding: Mr Aiden Cope and Dr Sara Valdebenito M.
• Risk of Bias assessment: Dr Alex Sutherland and Dr Sara Valdebenito M.
• Advise in statistical methods and contents: Professor Manuel Eisner, Professor David P. Farrington, Dr Alex Sutherland and Dr Maria M. Ttofi
• Statistical analysis and report writing: Dr Sara Valdebenito M.

8.3 ACKNOWLEDGEMENT

We are very grateful of Aiden Cope for his assistance with information retrieval and double coding.

8.4 SOURCES OF SUPPORT

Professor Manuel Eisner and Dr Sara Valdebenito have been awarded a grant by the Nuffield Foundation for conducting the proposed systematic review. Terms and conditions agreed with the sponsor involve the submission of results during 2016.

8.5 DECLARATIONS OF INTEREST

None of the researchers involved in the team present financial interest in this review. None of them have been involved in the development of interventions or systematic reviews on the scope of the present one. Three authors (Dr Sara Valdebenito M, Professor Manuel Eisner and Dr Alex Sutherland) were involved in the London Education and Inclusion Project cluster‐randomised controlled trial (ISRCTN 23244695). The study was designed as an independent evaluation and the authors have no financial or other links to the evaluated programme.

8.6 PLANS FOR UPDATING THE REVIEW

Dr Sara Valdebenito will be responsible for updating the present review every three years.

8.7 AUTHOR DECLARATION

Authors’ responsibilities

By completing this form, you accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every five years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group. If an update is not submitted according to agreed plans, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to propose the update to alternative authors.

Publication in the Campbell Library

The Campbell Collaboration places no restrictions on publication of the findings of a Campbell systematic review in a more abbreviated form as a journal article either before or after the publication of the monograph version in Campbell Systematic Reviews. Some journals, however, have restrictions that preclude publication of findings that have been, or will be, reported elsewhere, and authors considering publication in such a journal should be aware of possible conflict with publication of the monograph version in Campbell Systematic Reviews. Publication in a journal after publication or in press status in Campbell Systematic Reviews should acknowledge the Campbell version and include a citation to it. Note that systematic reviews published in Campbell Systematic Reviews and co‐registered with the Cochrane Collaboration may have additional requirements or restrictions for co‐publication. Review authors accept responsibility for meeting any co‐publication requirements.

I understand the commitment required to update a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:

Form completed by:

Dr Sara Valdebenito

Date:

25 September 2017

9. Tables

9.1 CHARACTERISTICS OF INCLUDED STUDIES

The following tables offer a succinct characterisation of each included study. We selected 11 characteristics from our coding which will allow the reader an overview of the study.

• Methodological design, clarifying the type of randomised controlled trial (e.g., cluster randomised, matched pairs randomisation).
• Characteristics of the participants, involving the size of the sample during randomisation, school grade of participants, percentage of males and predominant ethnicity.
• Location of the study, detailing the city and country where the study was implemented.
• Brief description of the intervention.
• Programme deliverers, those in charge of implementing the programme.
• Evaluator role. When the evaluator is only in charge of conducting the trial, we define that as “an independent evaluator.” Ifin addition the researcher delivers or designs the programme, we report that additional role.
• Outcomes measured in the study, not only the outcomes we are interested in.
• Length of the intervention.
• Assessment. This data refers to the measures carried out during the evaluation; for instance, baseline and post treatment, or baseline, post treatment and 12 month follow up.
• Attrition. This data provides an estimate of the percentage of reported attrition.
• Conflict of Interest Statement (COI). Offers information on the absence/presence of a formal and explicit statement declaring conflict of personal or institutional interest.
• Potential Conflict of Financial Interest (CoFI)

1) ref. Allen, Philliber, Herrling, & Kuperminc (1997)

2) ref. Arter (2005)

3) ref. Barnes, Bauza, & Treiber (2003)

4) ref. Berlanga (2004)

5) ref. Bradshaw, Waasdorp, & Leaf (2012)

6) ref. Bragdon (2010)

7) ref. Brett (1993)

8) ref. Burcham (2002)

9) ref. Collier (2002)

10) ref. Cook et al. (2014)

11) ref. Cornell, Allen, & Fan (2012)

12) ref. Crowder (2001)

13) ref. Dynarski et al. (2003)

14) ref. Edmunds et al. (2012)

15) ref. Farrell, Meyer, & White (2001)

16) ref. Feindler, Marriott, & Iwata (1984)

17) ref. Harding (2011)

18) ref. Hawkins, Doueck, & Lishner (1988)

19) ref. Hirsch, Hedges, Stawicki, & Mekinda (2011)

20) ref. Hostetler & Fisher (1997)

21) ref. Ialongo, Poduska, Werthamer, & Kellam (2001)

22) ref. Johnson (1983)

23) ref. Lewis et al. (2013)

24) ref. Mack (2001)

25) ref. Obsuth et al. (2016)

26) ref. Okonofua, Pauneskua, & Walton (2016)

27) ref. Panayiotopoulos & Kerfoot (2004)

28) ref. Peck (2006)

29) ref. Reese, Murphy, & Filipczak (1981)

30) ref. Russell (2007)

31) ref. Shetgiri, Kataoka, Lin, & Flores (2011)

32) ref. Smith (2004)

33) ref. Snyder et al. (2010)

34) ref. Sprague, Biglan, Rusby, Gau, & Vincent (2016)

35) ref. Tilghman, (1988)

36) ref. Ward & Gersten, (2013)

37) ref. Wyman et al. (2010)

9.2 RISK OF BIAS IN INCLUDED STUDIES BASED ON EPOC RISK OF BIAS TOOL (PER STUDY)

9.6 EXTRACTED DATA FOR EFFECT SIZE CALCULATIONS: PRIMARYOUTCOME

Figures

10.1 PERCENTAGE OF REPORTS WITHIN YEAR PERIOD

11. Data and analyses

11.1 SUB‐GROUP ANALYSES

Forest plot sub‐group analysis: Predominant gender in school

Forest plot sub‐group analysis: Grade at school

Forest plot sub‐group analysis: Type of intervention

Forest plot sub‐group analysis: Theoretical bases of interventions

Forest plot sub‐group analysis: Training before intervention

Forest plot sub‐group analysis: Monitoring the implementation of the intervention

Forest plot sub‐group analysis: Reasons for conducting the research

Forest plot sub‐group analysis: Evaluator role

Forest plot sub‐group analysis: Type of exclusion

12. Data collection instruments

12.1 ELIGIBILITY CHECK LIST

12.2 DATA COLLECTION INSTRUMENT

DATA‐CODING INSTRUMENT

School‐based interventions for reducing disciplinary school exclusion. A systematic review

[Variable names in brackets]

Contents

Section A. Codification

Section B. Bibliographical information

Section C. Ethics

Section D. Research design

Section E. Sample

Section F. Primary outcome coding

Section G. Secondary outcomes coding

Section H. Base‐line measurements

Section I. Programme delivered

Section J. Follow‐up measurement

Section K. Effect sizes

Section A. Codification

Instruction: use one data‐coding instrument for each manuscript. When more than one manuscript reports the same research project, select one of them as the principal (e.g., the older) and give it an ID number. The following manuscripts should use the same ID but it must be registered in the Crossref field.

[STUDYID] Study ID number:

[CROSSREF1] Cross reference document identifier:

[CROSSREF2] Cross reference document identifier:

[CROSSREF3] Cross reference document identifier:

[DATESCR] Date of screening:

[CODER] Coder Initials:

Section B. Bibliographical information

Before completing this section, please be sure that the manuscript is correctly uploaded in the reference manager programme.

[AUTHOR] Name of the main author(s):

[AFFIL] Main author affiliation:

[DATEPUB] Year of publication:

[DATEFIEDW] Year of fieldwork (usually reported in a range):

[COISTATEMENT] Has the paper included a conflict of interest statement?

• ⑥ 1. Yes
• ⑥ 0. No

[LANGPUB] Language of the publication:

• ⑥ 1. English
• ⑥ 2. German
• ⑥ 3. Italian
• ⑥ 4. Spanish
• ⑥ 5. Portuguese
• ⑥ 999. Other:________

[COUNTPUB] Country of publication:

• ⑥ 1. UK
• ⑥ 2. USA
• ⑥ 3. Canada
• ⑥ 4. Australia
• ⑥ 999. Other:_________
• ⑥ 99. Unknown

[TYPUB] Type of publication:

• ⑥ 1. Journal
• ⑥ 2. Book/book chapter
• ⑥ 3. Masters thesis
• ⑥ 4. PhD/doctoral thesis
• ⑥ 5. Technical/governmental report
• ⑥ 6. Conference proceedings
• ⑥ 999. Other:________

[AUTDIS] Main author discipline:

• ⑥ 1. Education
• ⑥ 2. Social Work
• ⑥ 3. Psychology
• ⑥ 4. Criminal Justice
• ⑥ 5. Sociology
• ⑥ 6. Psychiatry/Medicine
• ⑥ 999. Other:________
• ⑥ 99. Unknown

[LOCAT] How was the study/report located?

• ⑥ 1. Electronic database
• ⑥ 2. Web search
• ⑥ 3. Reference in a book/paper. Please specify:
• ⑥ 4. Hand search in specialised journal
• ⑥ 5. Peer/expert suggestion
• ⑥ 999. Other. Specify:____________

Section C. Ethics

[CONSENT] Did the study declare the use of “consent agreement forms”?

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 999. Other:________
• ⑥ 99. Unknown

[SIGNCONS] Who signed the consent?

• ⑥ 1. Students
• ⑥ 2. Parents
• ⑥ 3. Teachers
• ⑥ 4. Schools
• ⑥ 5. Parents and student
• ⑥ 999. Other. Specify:______________
• ⑥ 99. Unknown

Section D. Design

The present systematic review includes randomised control trials as well as quasi‐experimental reports (before/after measure plus a control or comparison group). If the control/comparison group is randomly allocated, non‐randomly allocated or matched and no intervention expected to produce impact is provided to it, you will be able to code that group as CONTROL. Subsequently, the TREATMENT group could be understood as the group that receives the intervention, no matter if that condition has been randomly allocated or not.

Please select always the data that is related with the sample effectively analysed.

[DESTYPE] What kind of design is this paper based on?

• ⑥ 1. Randomised controlled trial (true experiment)
• ⑥ 2. Before‐and‐after with control/comparison group/s
• ⑥ 3. Instrumental variable
• ⑥ 4. Propensity score matching
• ⑥ 5. Interrupted time series
• ⑥ 6. Pre/post measures with unmatched control/comparison group
• ⑥ 7. Inverse probability weighting
• ⑥ 999. Other. Specify:____________

[RANDUNIT] Units of randomization

• ⑥ 1. Individuals
• ⑥ 2. Clusters/groups (classroom, schools)
• ⑥ 999. Other. Specify:____________
• ⑥ 99. Unknown

[ANALUNIT] Unit of analysis

• ⑥ 1. Students
• ⑥ 2. Clusters/groups (classroom, schools)
• ⑥ 999. Other. Specify:____________
• ⑥ 99. Unknown

[COMPVAR] Variables measured to create comparability? (e.g., variables used to match the control and treatment groups)

__________________________________________________________________________________________

[MAINSTAT] What is the main statistical analysis used to produce the final results?

• ⑥ 1. Multilevel modelling
• ⑥ 2. Differences of means
• ⑥ 3. MANOVA
• ⑥ 4. Chi‐squared
• ⑥ 5. Propensity Score Matching
• ⑥ 999. Other. Specify:____________

Section E. Sample

[SAMPSELECT] How was the sample selected?

• ⑥ 1. Randomly
• ⑥ 2. Assessment
• ⑥ 3. Self‐selection
• ⑥ 999. Other. Specify:___________

[INSAMP] Initial sample size (i.e., individuals/schools):

[NUMBFOLL] Nº of follow‐up:

[FOLLSAMP1] Follow‐up 1 sample size:

[FOLLSAMP2] Follow‐up 2 sample size:

[FOLLSAMP3] Follow‐up 3 sample size:

[NSCHOOL] Initial number of schools:

[NSFOLL1] Follow‐up 1 sample size:

[NSFOLL2] Follow‐up 2 sample size:

[NSFOLL3] Follow‐up 3 sample size:

[NCLASS] Initial number of classes:

[NCFOLL1] Follow‐up 1 sample size:

[NCFOLL2] Follow‐up 2 sample size:

[NCFOLL3] Follow‐up 3 sample size:

Please code here the information on attrition described in the manuscript:

[MEANAGE] Mean age and standard deviation of overall sample at beginning of intervention:

[GENDER] Gender

• ⑥ % of males
• ⑥ % of females
• ⑥ 99. Unknown

[LOCAT] Location of programme

• ⑥ Urban area
• ⑥ Suburban area
• ⑥ Rural area
• ⑥ Mixture of areas
• ⑥ 99. Not enough information to determine

[GRADEX] Grade level of students

• ⑥ % of students in Elementary school or equivalent
• ⑥ % of students in Secondary school or equivalent
• ⑥ % of students in High school or equivalent
• ⑥ 4. Other:
• ⑥ 99. Unknown

[ETHNI] Predominant ethnicity¹³

• ⑥ 1. % of Caucasian:
• ⑥ 2. % of Black:
• ⑥ 3. % of Hispanic:
• ⑥ 4. % of Asian:
• ⑥ 5. % of other mixed background:
• ⑥ 99. Unknown

[COUNTRY] Please state the name of the country where schools and sample of students were located when tested.

______________________ (99 if unknown)

[LUNEX] Socio‐economic status

% of students receiving free/reduced school lunch:

99. Unknown

[SENEX] Special Educational Needs

% of students declaring SEN:

99. Unknown

Section F. Primary Outcome (School Exclusion)

[EXCLUSION] Is the manuscript reporting outcomes for school exclusion?

• ⑥ 1. Yes
• ⑥ 0. No

[TYPEXC] Type of exclusion measured

• ⑥ 1. In‐school exclusion
• ⑥ 2. Out‐of‐school exclusion
• ⑥ 99. Unknown

[CHEKTIP] Duration of school exclusion measured

• ⑥ 1. Days of Fixed‐term exclusion
• (Expressed in number or days, frequencies, percentages)
• ⑥ 2. Days of Permanent exclusion
• (Expressed in number or days, frequencies, percentages)
• ⑥ 99. Unknown

[ICCEXCLU] If the statistical analysis include cluster in MLM, please register the ICC for Exclusion:

Section G. Secondary outcomes

[BEHAVMES] Did the study include measures on behaviour domains?

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 99. Unknown

What types of the following behaviours are measured?

• [PROSO] Pro‐social behaviour (e.g., helping, empathy). Specify:____________

[MEPROSO] Measure(s) used to test the behaviour (name):

[ALPHAPROSO]

• ⑥ Reliability test. Specify alpha value:_____________
• ⑥ Non reported

[PAGEPROSO] Number of the page from where you extract statistical data:

[ICCPROSO] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes:

• [INTERNAL] Internalising problem behaviour
• (e.g., anxiety, depression, attention‐deficit and hyperactivity disorder (ADHD), attention deficit, hyperactivity). Specify:____________

[MINTERNAL] Measure (s) used to test the behaviour (name):

[ALPHAINTERNAL]

• ⑥ Reliability test. Specify alpha value:_____________
• ⑥ Non reported

[PAGEINTERNAL] Number of the page from where you extract statistical data:

[ICCINTERNAL] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes:

• [NAEXTERNAL] Non‐aggressive externalising problem behaviour
• (e.g., stealing, lying, graffiti, illegal drugs). Specify: _____________

[MNAEXTERNAL] Measure used to test the behaviour (name):

[ALPHANAEXTER]

• ⑥ Reliability test. Specify alpha value:_____________
• ⑥ Non reported

[PAGENAXTERN] Number of the page from where you extract statistical data:

[ICCNAEXT] If the statistical analysis include cluster in MLM, please register the ICC for behavioural outcomes:

• [AAGRESEXT] Aggressive externalising problem behaviour
• (e.g., Opposition/defiance, physical aggression, indirect aggression, instrumental aggressions/dominance, reactive aggression, school bullying). Specify:_____________

[MAGRESSEXT] Measure used to test the behaviour (name):

[ALPHAAEXT]

• ⑥ Reliability test. Specify alpha value:_____________
• ⑥ Non reported

[AGRESPAGE] Number of the page from where you extract statistical data:

[ICCAGREEX] If the statistical analysis include cluster in MLM, please register the ICC for Behavioural outcomes:

Section H. Base‐line measurements

[DATABAS] Date of baseline assessment:

What measures were used?

[SRMES] Self‐report

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 99. Unknown

[TRMES] Teachers’ report

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 99. Unknown

[SCHRMES] School records

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 99. Unknown

[PAREP] Parents

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 99. Unknown

[OMES] Other:_________________

[EXCBL] Frequency of exclusion at baseline (register any measure given by the study)

Section I. Programme delivered

This section aims to codify data on the delivery process. Be aware that sometimes final reports do not describe all the data related to delivery. In those cases it would be helpful to search for registered protocols or earlier publications reporting more data on this.

[PRONAME] Name of the programme:

[PROCURRI] Was the programme curricular?

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 99. Unknown
• ⑥ 999. Other. Specify:____________

[PROEND] The programme was conducted for:

• ⑥ 1. Research ends
• ⑥ 2. Demonstration ends
• ⑥ 3. Routine
• ⑥ 99. Unknown
• ⑥ 999. Other. Specify:_____________

[PROSIT] Primary programme site:

• ⑥ 1. Public school
• ⑥ 2. Private school
• ⑥ 3. Other, (specify):_________
• ⑥ 99. Unknown

[PROSCH] Was at least one of the components of the intervention was settled at school?

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 99. Unknown

[PRODEL] Who delivered the programme?

• ⑥ 1. External facilitators
• ⑥ 2. School facilitators
• ⑥ 3. Both
• ⑥ 99. Unknown

[PDBACK] Deliverer’s background 1

• ⑥ 1. Social worker
• ⑥ 2. Psychologist
• ⑥ 3. Teacher
• ⑥ 4. Police officers
• ⑥ 5. Peers
• ⑥ 999. Other. Specify:_____________
• ⑥ 99. Unknown

[PDBACK] Deliverer’s background 2

• ⑥ 1. Social worker
• ⑥ 2. Psychologist
• ⑥ 3. Teacher
• ⑥ 4. Police officers
• ⑥ 5. Peers
• ⑥ 999. Other. Specify:_____________
• ⑥ 99. Unknown

[TRAINBEF] Did the deliverer receive training BEFORE implementing the programme?

• ⑥ 1. Yes.
• ⑥ 0. No.
• ⑥ 99. Unknown

[THOURS] How long was the training in hours?:____________

[TRAINDUR] Did the deliverer receive training DURING the implementation?

• ⑥ 1. Yes.
• ⑥ 0. No.
• ⑥ 99. Unknown

[THOURS2] How long was the training in hours?:

What type of intervention was delivered? If the manuscript indicates a mixture of interventions you can select more than one using TYPEPRO 1, 2 and 3.

Theoretical background of the intervention. If the manuscript indicates a mixture of theories, you can select more than one using THEORY 1, 2 and 3.

[PROCONT] What happened to the control group?

• ⑥ 1. No intervention
• ⑥ 2. Wait‐list control
• ⑥ 3. Minimal contact
• ⑥ 4. Treatment as usual
• ⑥ 5. Alternative treatment
• ⑥ 5. Placebo
• ⑥ 999. Other. Specify:____________

[PROFORM] Delivery format:

• ⑥ 1. Manualised programme
• ⑥ 2. Unstructured programme
• ⑥ 3. Mixed
• ⑥ 99. Unknown
• ⑥ 999. Other. Specify:____________

What was the programme dosage?

[PRODOSW] AVERAGE Duration in weeks:

[PRODOSH] AVERAGE Hours per week:

[PROFREQ] What was the frequency of the programme counted?

• ⑥ 1. Less than a week
• ⑥ 2. Once a week
• ⑥ 3. Twice a week
• ⑥ 4. 3‐4 times a week
• ⑥ 5. Daily
• ⑥ 99. Unknown

[EVROLE] What was the “evaluator” role?

• ⑥ 1. Deliver the programme
• ⑥ 2. Designed the programme
• ⑥ 3. Both design and delivery
• ⑥ 4. Independent evaluator
• ⑥ 99. Unknown

[MONITOR] Was the programme implementation monitored?

• ⑥ 1. Yes
• ⑥ 0. No
• ⑥ 99. Unknown. Not enough information

[IMPROB] Does the report provide information about implementation problems?

• ⑥ 1. Yes, there were clear problems which are reported
• ⑥ 0. No, non‐reported problems, reasonably well implemented
• ⑥ 2. Possible problems based on the description of the intervention
• ⑥ 99. Unknown. Not enough information

[PROCOST] Is the cost of the intervention mentioned?

• ⑥ 1. Yes
• ⑥ 0. No

[AMOUNT] Cost:

[UNITCURR] Currency:

Section J. Follow‐up measurement

[DATEFALL] Date of follow up:

Multiple follow‐ups

[MONTHFO1] Nº of months from baseline to 1^st follow‐up:

[MONTHFO2] Nº of months from baseline to 2^nd follow‐up:

[MONTHFO3] Nº of months from baseline to 3^rd follow‐up:

[MONTHFO4] Nº of months from baseline to 4^th follow‐up:

What measures were used?

[POSTSR] Children/adolescent self‐report

• ⑥ 1. Yes
• ⑥ 0. No

[POSTTR] Teachers’ report

• ⑥ 1. Yes
• ⑥ 0. No

[POSTSR] School records

• ⑥ 1. Yes
• ⑥ 0. No

[POSTPR] Parents report

• ⑥ 1. Yes
• ⑥ 0. No

[POSTO] Other:__________

[FREQEXFOLL] Frequency of exclusion at follow‐up (register any measure given by the study)

Section K. Effect sizes of intervention on school exclusion

• Effect size: outcomes expressed in continuous data.

[CSSEX] Sample size for the ES (Treatment group)

[CSSCON] Sample size for the ES (Control group)

[MEANEX] Mean (Treatment group)

[MEANCON] Mean (Control group)

[MEANADJ] Are the Means adjusted?

• ⑥ 1. Yes.
• ⑥ 0. No

[ADJBY] Adjusted by (describe):______________

[SDEX] Standard deviation (Treatment group)

[SDCON] Standard deviation (Control group)

[SEEX] Standard error (Treatment group)

[SECON] Standard error (Control group)

[CORREX] Correlation coefficient + p value (Treatment group)

[CORRCON] Correlation coefficient + p value (Control group)

[SMDTREAT] Standardised mean difference + confidence intervals

• Effect size: outcomes expressed in dichotomous data.

[DSSTRE] Sample size for the ES (Treatment group)

[DSSCONT] Sample size for the ES (Control group)

[NUMTRE] Treatment group; number of successful cases:

[NUMCON] Control group; number of successful cases:

[PROPTRE] Treatment group; proportion of successful cases:

[PROPCON] Control group; proportion of successful cases:

[ORTRE] Treatment group; odds ratios:

• Confidence Intervals:
• p‐value:

[ORCON] Control group; odds ratios:

• Confidence Intervals:
• p‐value:

[ORADJ] Are the odds ratios adjusted?

• ⑥ 1. Yes.
• ⑥ 0. No

Adjusted by (explain):_________

[CHISC] X² value with df:

[PAGEEFFECT] Number of the page from where you extract statistical data:

• Effect sizes at follow‐up

[ESFOLLOW1] Calculated effect at follows up 1:______

[ESFOLL1] Number of months after intervention for follow‐up 1:______

[ESFOLLOW2] Calculated effect at follows up 2:______

[ESFOLL2] Number of months after intervention for follow‐up 2:______

[ESFOLLOW3] Calculated effect at follows up 3:______

[ESFOLL3] Number of months after intervention for follow‐up 3:______

[ESFOLLOW4] Calculated effect at follows up 4:______

[ESFOLL4] Number of months after intervention for follow‐up 4:______

12.3 EPOC ‘RISK OF BIAS’ TOOL

12.4 TRICHOTOMOUS SCALE FOR POTENTIAL FINANCIAL CONFLICT OF INTEREST

13. Electronic searches

Australian Education Index

Date: 05/10/ 2015

Output of the searches: 148

Saved hits: 1

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

British Education Index

Date: 05/10/ 2015

Output of the searches: 202

Saved hits: 5

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school AND stand down

BMJ Controlled Trials

Date: 06/10/ 2015

Output of the searches: 550

Saved hits: 0

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

CBCA Education (Canada)

Date: 14/10/2015

Output of the searches: 5652

Saved hits: 58

Final searches string:

• school exclusion
• school suspension
• school expelled
• school expulsion
• school stand down

ClinicalTrials.gov

Date: 07/10/ 2015

Output of the searches: 285

Saved hits: 3

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

Criminal Justice Abstract

Date: 06/10/ 2015

Output of the searches: 369

Saved hits: 19

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

Cochrane Central Register of Controlled Trials (CENTRAL)

Date: 16/10/2015

Output of the searches: 154

Saved hits: 12

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

Educational Resources Information Center ‐ ERIC

Date: 14/10/2015

Output of the searches: 1491

Saved hits: 48

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

Ethos Beta

Date: 08/10/ 2015

Output of the searches: 482

Saved hits: 5

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

EMBASE

Date: From 08/10/ 2015 to 12/10/2015

Output of the searches: 1569

Saved hits: 12

Final searches string:

• “school exclusion” AND
• “school exclusion” AND evaluation
• “school exclusion” AND random*
• “school exclusion” AND intervention,
• “school exclusion” AND effective*,
• “school exclusion” AND efficacy,
• “school exclusion” AND quasi,
• “school exclusion” AND impact,
• “school exclusion” AND RCT,
• “school exclusion” AND school management
• “school exclusion” AND classroom management
• “school exclusion” AND school support project*
• “school exclusion” AND skills training
• “school exclusion” AND disciplinary methods
• “school exclusion” AND token economy
• “school exclusion” AND program*
• “school exclusion” AND *intervention*
• “school exclusion” AND strateg*
• “school exclusion” AND schoolchildren
• “school exclusion” AND *children*
• “school exclusion” AND school‐age*
• “school exclusion” AND adolescent*
• “school exclusion” AND pupil*
• “school exclusion” AND student*
• “school suspension” AND experiment*
• “school suspension” AND evaluation
• “school suspension” AND random*
• “school suspension” AND intervention,
• “school suspension” AND effective*,
• “school suspension” AND efficacy,
• “school suspension” AND quasi,
• “school suspension” AND impact,
• “school suspension” AND RCT,
• “school suspension” AND school management
• “school suspension” AND classroom management
• “school suspension” AND school support project*
• “school suspension” AND skills training
• “school suspension” AND disciplinary methods
• “school suspension” AND token economy
• “school suspension” AND program*
• “school suspension” AND *intervention*
• “school suspension” AND strateg*
• “school suspension” AND schoolchildren
• “school suspension” AND *children*
• “school suspension” AND school‐age*
• “school suspension” AND adolescent*
• “school suspension” AND pupil*
• “school suspension” AND student*
• “school expulsion” AND experiment*
• “school expulsion” AND evaluation
• “school expulsion” AND random*
• “school expulsion” AND intervention,
• “school expulsion” AND effective*,
• “school expulsion” AND efficacy,
• “school expulsion” AND quasi,
• “school expulsion” AND impact,
• “school expulsion” AND RCT,
• “school expulsion” AND school management
• “school expulsion” AND classroom management
• “school expulsion” AND school support project*
• “school expulsion” AND skills training
• “school expulsion” AND disciplinary methods
• “school expulsion” AND token economy
• “school expulsion” AND program*
• “school expulsion” AND *intervention*
• “school expulsion” AND strateg*
• “school expulsion” AND schoolchildren
• “school expulsion” AND *children*
• “school expulsion” AND school‐age*
• “school expulsion” AND adolescent*
• “school expulsion” AND pupil*
• “school expulsion” AND student*

Google Scholar

Date: From 27/10/ 2015 to 04/11/2015

Output of the searches: 13525

Saved hits: 165

Final searches string:

• “school exclusion” AND experiment OR evaluation experiment OR evaluation OR quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random*
• “school exclusion” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy OR program* OR *intervention* OR strateg*
• “school suspen*” AND experiment* OR evaluation OR quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random*
• “school suspen*” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy OR program* OR *intervention* OR strateg*
• “school expulsion” AND experiment OR evaluation experiment OR evaluation OR quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random*
• “school expulsion” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy OR program* OR *intervention* OR strateg*
• “stand‐down” AND experiment* OR evaluation OR quasi OR effective* OR RCT OR impact OR efficacy OR intervention OR random*
• “stand‐down” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy OR program* OR *intervention* OR strateg*

[In order to manage the searches we break down the searches by year. For instance 1980‐1990, 1991‐2005 and 2006 to date]

Google

Date: From 27/10/ 2015 to 04/11/2015

Output of the searches: 4092

Saved hits: 22

Final searches string:

• “school exclusion” AND experiment* OR evaluation OR quasi OR RCT OR impact OR efficacy OR intervention
• “school exclusion” AND random*
• “school exclusion” AND effective*
• “school exclusion” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy
• “school exclusion” AND *intervention*
• “school exclusion” AND strateg*
• “school exclusion” AND program*
• “school suspension” AND experiment* OR evaluation OR quasi OR RCT OR impact OR efficacy OR intervention
• “school suspension” AND random*
• “school suspension” AND effective*
• “school suspension” AND school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy
• “school suspension” AND *intervention*
• “school suspension” AND strateg*
• “school suspension” AND program*
• school expulsion AND…intervention* OR strategy* OR program*
• school expulsion AND…random* OR effective*
• school expulsion AND…school management OR classroom management OR school support project* OR skills training OR disciplinary methods OR token economy
• school expulsion AND…random* OR effective
• school expulsion AND…experiment* OR evaluation OR quasi OR RCT OR impact OR efficacy OR intervention

Institute of Education Sciences – What Works Clearinghouse

Date: 30/09/ 2015

Output of the searches: 2

Saved hits: 0

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

ISI Web of Knowledge

Date: from 07/09/ 2015 to 17/09/2015

Output of the searches: 4391

Saved hits: 270

Final searches string:

• TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)
• AND
• TOPIC: (school) AND TOPIC: (*exclusion)
• TOPIC: (school) AND TOPIC: (*suspension*)
• AND
• TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)
• TOPIC: (school) AND TOPIC: (suspended)
• AND
• TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)
• Refined by: RESEARCH AREAS: (PSYCHOLOGY OR FAMILY STUDIES OR EDUCATION EDUCATIONAL RESEARCH OR BEHAVIORAL SCIENCES OR PSYCHIATRY OR CRIMINOLOGY PENOLOGY OR SOCIOLOGY OR ETHNIC STUDIES OR SOCIAL WORK OR URBAN STUDIES OR SOCIAL SCIENCES OTHER TOPICS OR SOCIAL ISSUES)
• Timespan=1980‐2015
• TOPIC: (school) AND TOPIC: (expelled)
• AND
• TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)
• TOPIC: (school) AND TOPIC: (expulsion)
• AND
• TOPIC: (experiment*) OR TOPIC: (evaluation) OR TOPIC: (random*) OR TOPIC: (intervention) OR TOPIC: (effective*) OR TOPIC: (efficacy) ORTOPIC: (quasi) OR TOPIC: (impact) OR TOPIC: (RCT)
• TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)
• AND
• TOPIC: (school) AND TOPIC: (*exclusion)
• TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)
• AND
• TOPIC: (school) AND TOPIC: (*exclusion)
• TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)
• AND
• TOPIC: (school) AND TOPIC: (*suspension*)
• TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)
• AND
• TOPIC: (school) AND TOPIC: (suspended)
• TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)
• AND
• TOPIC: (school) AND TOPIC: (expelled)
• TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)
• AND
• TOPIC: (school) AND TOPIC: (expulsion)
• TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school‐age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student)
• AND
• TOPIC: (school) AND TOPIC: (*exclusion)
• TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school‐age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student)
• AND
• TOPIC: (school) AND TOPIC: (*suspension*)
• TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school‐age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student)
• AND
• TOPIC: (school) AND TOPIC: (suspended)
• TOPIC: (schoolchildren) OR TOPIC: (*children*) OR TOPIC: (school‐age*) OR TOPIC: (adolescent*) OR TOPIC: (pupil*) OR TOPIC: (student)
• AND
• TOPIC: (school) AND TOPIC: (expelled)
• TOPIC: (school management) OR TOPIC: (classroom management) OR TOPIC: (school support project*) OR TOPIC: (skills training) OR TOPIC:(disciplinary methods) OR TOPIC: (token economy) OR TOPIC: (program*) OR TOPIC: (*intervention*) OR TOPIC: (strateg*)
• AND
• TOPIC: (school) AND TOPIC: (expulsion)

All searches in ISI Web of Knowledge were Refined by: RESEARCH AREAS: (PSYCHOLOGY OR FAMILY STUDIES OR EDUCATION EDUCATIONAL RESEARCH OR BEHAVIORAL SCIENCES OR PSYCHIATRY OR CRIMINOLOGY PENOLOGY OR SOCIOLOGY OR ETHNIC STUDIES OR SOCIAL WORK OR URBAN STUDIES OR SOCIAL SCIENCES OTHER TOPICS OR SOCIAL ISSUES)

Timespan=1980‐2015

MEDLINE

Date: 05/10/ 2015

Output of the searches: 142

Saved hits: 0

Final searches string:

• “school exclusion” AND experiment*
• “school exclusion” AND evaluation
• “school exclusion” AND random*
• “school exclusion” AND intervention,
• “school exclusion” AND effective*,
• “school exclusion” AND efficacy,
• “school exclusion” AND quasi,
• “school exclusion” AND impact,
• “school exclusion” AND RCT,
• “school exclusion” AND school management
• “school exclusion” AND classroom management
• “school exclusion” AND school support project*
• “school exclusion” AND skills training
• “school exclusion” AND disciplinary methods
• “school exclusion” AND token economy
• “school exclusion” AND program*
• “school exclusion” AND *intervention*
• “school exclusion” AND strateg*
• “school exclusion” AND schoolchildren
• “school exclusion” AND *children*
• “school exclusion” AND school‐age*
• “school exclusion” AND adolescent*
• “school exclusion” AND pupil*
• “school exclusion” AND student*
• “school suspension” AND experiment*
• “school suspension” AND evaluation
• “school suspension” AND random*
• “school suspension” AND intervention,
• “school suspension” AND effective*,
• “school suspension” AND efficacy,
• “school suspension” AND quasi,
• “school suspension” AND impact,
• “school suspension” AND RCT,
• “school suspension” AND school management
• “school suspension” AND classroom management
• “school suspension” AND school support project*
• “school suspension” AND skills training
• “school suspension” AND disciplinary methods
• “school suspension” AND token economy
• “school suspension” AND program*
• “school suspension” AND *intervention*
• “school suspension” AND strateg*
• “school suspension” AND schoolchildren
• “school suspension” AND *children*
• “school suspension” AND school‐age*
• “school suspension” AND adolescent*
• “school suspension” AND pupil*
• “school suspension” AND student*
• “school expulsion” AND experiment*
• “school expulsion” AND evaluation
• “school expulsion” AND random*
• “school expulsion” AND intervention,
• “school expulsion” AND effective*,
• “school expulsion” AND efficacy,
• “school expulsion” AND quasi,
• “school expulsion” AND impact,
• “school expulsion” AND RCT,
• “school expulsion” AND school management
• “school expulsion” AND classroom management
• “school expulsion” AND school support project*
• “school expulsion” AND skills training
• “school expulsion” AND disciplinary methods
• “school expulsion” AND token economy
• “school expulsion” AND program*
• “school expulsion” AND *intervention*
• “school expulsion” AND strateg*
• “school expulsion” AND schoolchildren
• “school expulsion” AND *children*
• “school expulsion” AND school‐age*
• “school expulsion” AND adolescent*
• “school expulsion” AND pupil*
• “school expulsion” AND student*

The National dropout prevention centre network

Date: 16/10/2015

Output of the searches: 26

Saved hits: 0

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

The Netherlands Trial Register ‐ NTR

Date: 07/10/ 2015

Output of the searches: 0

Saved hits: 0

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

Open Grey

Date: 07/10/ 2015

Output of the searches: 169

Saved hits: 3

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

ProQuest Dissertations & Theses A&I: Social Sciences

Including: Dissertation Abstracts & Index to Theses Database

Date: 19/10/2015

Output of the searches: 5280

Saved hits: 344

Final searches string:

• “school exclusion” AND experiment
• “school exclusion” AND evaluation
• “school exclusion” AND random*
• “school exclusion” AND intervention
• “school exclusion” AND effective*
• “school exclusion” AND efficacy
• “school exclusion” AND quasi
• “school exclusion” AND impact
• “school exclusion” AND RCT
• “school exclusion” AND school management
• “school exclusion” AND classroom management
• “school exclusion” AND school support project*
• “school exclusion” AND skills training
• “school exclusion” AND disciplinary methods
• “school exclusion” AND token economy
• “school exclusion” AND program*
• “school exclusion” AND strateg*
• “school suspension” AND experiment
• “school suspension” AND evaluation
• “school suspension” AND random*
• “school suspension” AND intervention
• “school suspension” AND effective*
• “school suspension” AND efficacy
• “school suspension” AND quasi
• “school suspension” AND impact
• “school suspension” AND RCT
• “school suspension” AND school management
• “school suspension” AND classroom management
• “school suspension” AND school support project*
• “school suspension” AND skills training
• “school suspension” AND disciplinary methods
• “school suspension” AND token economy
• “school suspension” AND program*
• “school suspension” AND strateg*
• “school stand‐down” AND experiment
• “school stand‐down” AND evaluation
• “school stand‐down” AND random*
• “school stand‐down” AND intervention
• “school stand‐down” AND effective*
• “school stand‐down” AND efficacy
• “school stand‐down” AND quasi
• “school stand‐down” AND impact
• “school stand‐down” AND RCT
• “school stand‐down” AND school management
• “school stand‐down” AND classroom management
• “school stand‐down” AND school support project*
• “school stand‐down” AND skills training
• “school stand‐down” AND disciplinary methods
• “school stand‐down” AND token economy
• “school stand‐down” AND program*
• “school stand‐down” AND strateg*
• “school expelled” AND experiment
• “school expelled” AND evaluation
• “school expelled” AND random*
• “school expelled” AND intervention
• “school expelled” AND effective*
• “school expelled” AND efficacy
• “school expelled” AND quasi
• “school expelled” AND impact
• “school expelled” AND RCT
• “school expelled” AND school management
• “school expelled” AND classroom management
• “school expelled” AND school support project*
• “school expelled” AND skills training
• “school expelled” AND disciplinary methods
• “school expelled” AND token economy
• “school expelled” AND program*
• “school expelled” AND strateg*
• “school expulsion” AND experiment
• “school expulsion” AND evaluation
• “school expulsion” AND random*
• “school expulsion” AND intervention
• “school expulsion” AND effective*
• “school expulsion” AND efficacy
• “school expulsion” AND quasi
• “school expulsion” AND impact
• “school expulsion” AND RCT
• “school expulsion” AND school management
• “school expulsion” AND classroom management
• “school expulsion” AND school support project*
• “school expulsion” AND skills training
• “school expulsion” AND disciplinary methods
• “school expulsion” AND token economy
• “school expulsion” AND program*
• “school expulsion” AND strateg*
• “school suspended” AND experiment
• “school suspended” AND evaluation
• “school suspended” AND random*
• “school suspended” AND intervention
• “school suspended” AND effective*
• “school suspended” AND efficacy
• “school suspended” AND quasi
• “school suspended” AND impact
• “school suspended” AND RCT
• “school suspended” AND school management
• “school suspended” AND classroom management
• “school suspended” AND school support project*
• “school suspended” AND skills training
• “school suspended” AND disciplinary methods
• “school suspended” AND token economy
• “school suspended” AND program*
• “school suspended” AND strateg*

PsychINFO

Date: From 12/10/ 2015 to 14/10/2015

Output of the searches: 1538

Saved hits: 86

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

Sociological Abstract (ProQuest)

Date: From 21/09/ 2015 to 23/09/2015

Output of the searches: 2440

Saved hits: 355

Final searches string:

• “school exclusion” AND experiment*
• “school exclusion” AND evaluation
• “school exclusion” AND random*
• “school exclusion” AND intervention,
• “school exclusion” AND effective*,
• “school exclusion” AND efficacy,
• “school exclusion” AND quasi,
• “school exclusion” AND impact,
• “school exclusion” AND RCT,
• “school exclusion” AND school management
• “school exclusion” AND classroom management
• “school exclusion” AND school support project*
• “school exclusion” AND skills training
• “school exclusion” AND disciplinary methods
• “school exclusion” AND token economy
• “school exclusion” AND program*
• “school exclusion” AND *intervention*
• “school exclusion” AND strateg*
• “school exclusion” AND schoolchildren
• “school exclusion” AND *children*
• “school exclusion” AND school‐age*
• “school exclusion” AND adolescent*
• “school exclusion” AND pupil*
• “school exclusion” AND student*
• “school suspension” AND experiment*
• “school suspension” AND evaluation
• “school suspension” AND random*
• “school suspension” AND intervention,
• “school suspension” AND effective*,
• “school suspension” AND efficacy,
• “school suspension” AND quasi,
• “school suspension” AND impact,
• “school suspension” AND RCT,
• “school suspension” AND school management
• “school suspension” AND classroom management
• “school suspension” AND school support project*
• “school suspension” AND skills training
• “school suspension” AND disciplinary methods
• “school suspension” AND token economy
• “school suspension” AND program*
• “school suspension” AND *intervention*
• “school suspension” AND strateg*
• “school suspension” AND schoolchildren
• “school suspension” AND *children*
• “school suspension” AND school‐age*
• “school suspension” AND adolescent*
• “school suspension” AND pupil*
• “school suspension” AND student*
• “school expulsion” AND experiment*
• “school expulsion” AND evaluation
• “school expulsion” AND random*
• “school expulsion” AND intervention,
• “school expulsion” AND effective*,
• “school expulsion” AND efficacy,
• “school expulsion” AND quasi,
• “school expulsion” AND impact,
• “school expulsion” AND RCT,
• “school expulsion” AND school management
• “school expulsion” AND classroom management
• “school expulsion” AND school support project*
• “school expulsion” AND skills training
• “school expulsion” AND disciplinary methods
• “school expulsion” AND token economy
• “school expulsion” AND program*
• “school expulsion” AND *intervention*
• “school expulsion” AND strateg*
• “school expulsion” AND schoolchildren
• “school expulsion” AND *children*
• “school expulsion” AND school‐age*
• “school expulsion” AND adolescent*
• “school expulsion” AND pupil*
• “school expulsion” AND student*

SciElo

Date: 06/10/ 2015

Output of the searches: 32

Saved hits: 0

Final searches string:

• “school exclusion evaluation”
• “school exclusion experiment”
• “school suspension evaluation”
• “school suspension experiment”
• “school expulsion evaluation”
• “school expulsion experiment”
• suspension escolar
• suspension escolar evaluación
• suspension escolar experiment
• expulsion escolar
• expulsion escolar evaluación
• expulsion escolar experiment
• expulsion escuela

Sciencegov

Date: From 05/10/ 2015 to 06/10/2015

Output of the searches: 319

Saved hits: 61

Final searches string:

• “school suspension” AND evaluation
• “school suspension” AND experiment*
• “school exclusion” AND evaluation
• “school exclusion” AND experiment*
• “school expulsion” AND evaluation
• “school expulsion” AND experiment*

The Campbell Collaboration Social, Psychological, Educational and Criminological Trials Register (C2‐SPECTR)

Date: 16/10/2015

Output of the searches: 1

Saved hits: 1

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

Trials Journal

Date: 06/10/ 2015

Output of the searches: 56

Saved hits: 1

Final searches string:

• school exclusion
• school suspension
• school suspended
• school expelled
• school expulsion
• school stand down

WHO‐ International Clinical Trials Registry Platform (ICTRP)

Date: 30/09/2015

Output of the searches: 3

Saved hits: 3

Final searches string:

• “school exclusion” (other outcome terms searched with no results)

References

1. Preventing teen pregnancy and academic failure: Experimental evaluation of a developmentally based approach.. Child Development,, 1997
2. The effects of the positive alternative learning supports program on the attendance, academic performance, and behavior of students with chronic behavior problems. (Doctoral Dissertation).. 2005
3. Impact of stress reduction on negative school behavior in adolescents.. Health and quality of life outcomes,, 2003. [PubMed]
4. The efficacy of an intensive guidance intervention program to increase academic success of at‐risk middle school students. (Doctoral Dissertation).. 2004
5. Effects of School‐Wide Positive Behavioral Interventions and Supports on child behavior problems.. Pediatrics,, 2012. [PubMed]
6. The effects of a career awareness project on the academic performance of at‐risk eighth grade students. (Doctoral Dissertation).. 2010
7. An examination of the impact of a mentoring program on inner city, seventh‐grade, African American students’ attitudes toward learning and other selected variables. (Doctoral Dissertation).. 1993
8. Impact of school‐based social problem‐solving training on middle school students with disruptive behavior. (Doctoral Dissertation).. 2002
9. The effect of pro‐social skills training on the problem behavior of selected African American children in the District of Columbia public schools: Recommendations for change. (Doctoral Dissertation).. 2002
10. Cook, P. J. , Dodge, K. , Farkas, G. , Fryer Jr, R. G. , Guryan, J. , Ludwig, J. , … Steinberg, L. (2014). The (surprising) efficacy of academic and behavioral intervention with disadvantaged youth from a randomized experiment in Chicago. Institute for Policy Research Northwestern University. WP‐14‐03. doi:10.3386/w19862
11. A randomized controlled study of the Virginia Student Threat Assessment Guidelines in kindergarten through grade 12.. School Psychology Review,, 2012
12. The effect of the gang resistance education and training program on character education and the development of leadership skills in children at an inner‐city middle school. (Doctoral Dissertation).. 2001
13. Dynarski, M. , Moore, M. , Mullens, J. , Gleason, P. , James‐Burdumy, S. , Rosenberg, L. , … Levy, D. (2003). When schools stay open late: The national evaluation of the 21st‐Century Community Learning Centers program. First year findings. Retrieved from Mathematic Policy Research, Inc. Decision Information Resources, Inc.: U.S. Department of Education
14. Expanding the start of the college pipeline: Ninth‐grade findings from an experimental study of the impact of the early college high school model.. Journal of Research on Educational Effectiveness,, 2012
15. Evaluation of Responding in Peaceful and Positive Ways (RIPP): A school‐based prevention program for reducing violence among urban adolescents.. Journal of Clinical Child Psychology,, 2001. [PubMed]
16. Group anger control training for junior high school delinquents.. Cognitive Therapy and Research,, 1984
17. An evaluation of a targeted group intervention delivered to year 8 pupils and broadly based on cognitive‐behavioural approaches. (Doctoral Dissertation).. 2011
18. Changing teaching practices in mainstream classrooms to improve bonding and behavior of low achievers.. American Educational Research Journal,, 1988. [DOI]
19. After‐school programs for high school students.. An evaluation of After School Matters. Retrieved from, 2011
20. Project CARE substance abuse prevention program for high‐risk youth: A longitudinal evaluation of program effectiveness.. Journal of Community Psychology,, 1997
21. The distal impact of two first‐grade preventive interventions on conduct problems and disorder in early adolescence.. Journal of Emotional and Behavioral Disorders,, 2001
22. The effects of affective education on the discipline of selected seventh and eighth grade students in the district of Columbia public schools. (Doctoral Dissertation).. 1983
23. Problem behavior and urban, low‐income youth a randomized controlled trial of Positive Action in Chicago.. American Journal of Preventive Medicine,, 2013. [PubMed]
24. The effects of a behavioral intervention on discipline referrals and school suspension. (Doctoral Dissertation).. 2001
25. London Education and Inclusion Project (leip): Results from a cluster‐randomized controlled trial of an intervention to reduce school exclusion and antisocial behavior.. Journal of Youth & Adolescence. doi:10.1007/s10964‐016‐0468‐4, 2016
26. Brief intervention to encourage empathic discipline cuts suspension rates in half among adolescents.. PNAS,, 2016. [PubMed]
27. A home and school support project for children excluded from primary and first year secondary school.. Child and Adolescent Mental Health,, 2004. [PubMed]
28. The effects of group involvement in a prevention program on at‐risk youth’s grades, attendance, substance use, and contacts with juvenile probation: An archival study of the STOP program. (Doctoral Dissertation).. 2006
29. Assessment of multiple behavioral procedures on academic and social classroom behavior.. Psychology in the Schools,, 1981
30. Influencing adaptive functioning in school‐age children: implementation and program evaluation of the Coping Power Program. (Doctoral Dissertation).. 2007
31. A randomized, controlled trial of a school‐based intervention to reduce violence and substance use in predominantly Latino high school students.. Journal of the National Medical Association,, 2011. [PubMed]
32. A mixed paradigm: Study of a transformational learning program for at‐risk high school students. (Doctoral Dissertation).. 2004
33. Impact of a social‐emotional and character development program on school‐level indicators of academic achievement, absenteeism, and disciplinary outcomes: A matched‐pair, cluster‐randomized, controlled trial.. Journal of Research on Educational Effectiveness,, 2010. [PubMed]
34. Sprague, J. R. , Biglan, A. , Rusby, J. , Gau, J. M. , & Vincent, C. G. (2016). Implementing School‐Wide PBIS in middle schools: Results of a randomized trial (unpublished paper). The University of Oregon.
35. The effect of peer counseling on lowering the rate of middle school suspensions. (Doctoral Dissertation).. 1988
36. A randomized evaluation of the safe and civil schools model for positive behavioral interventions and supports at elementary schools in a large urban school district.. School Psychology Review,, 2013
37. Intervention to strengthen emotional self‐regulation in children with emerging mental health problems: Proximal impact on school behavior.. Journal of Abnormal Child Psychology,, 2010. [PubMed]
38. Ballard, K. L. (2009). School‐related and social‐emotional outcomes of providing mental health services in schools. (Doctoral Dissertation). The University of Wisconsin ‐ Madison, US. Retrieved from EBSCOhost psyh database.
39. The influence of the Alabama Reading Initiative on the culture of three inner‐city elementary schools in the state of Alabama.(Doctoral Dissertation).. 2002
40. An evaluation of the right choices program to determine effectiveness in delivering constructive interventions and providing an early support program in order to modify behavior of first‐time student offenders who commit drug and violent acts. (Doctoral Dissertation).. 2010
41. Effectiveness of 1‐2‐3 Magic for Teachers to improve classroom management and discipline. (Doctoral Dissertation).. 2013
42. Positive behavior intervention and support: An evaluation of its impact in elementary schools. (Doctoral Dissertation).. 2014
43. The impact of a transition program on ninth grade students’ performance. (Doctoral Dissertation).. 2008
44. The use of school‐wide positive behavior support at a rural high school to decrease disruptive behavior for both typical students and students identified with special needs. (Doctoral Dissertation).. 2007
45. The effects of the positive behavior intervention and support program on the out‐of‐school suspension rates and english achievement of African american males in grade nine. (Doctoral Dissertation).. 2013
46. Evaluation of a brief group parent training intervention in the context of an after‐school program for middle‐school students. (Doctoral Dissertation).. 2007
47. Stuarts draft high school: Quantitative study evaluating the ninth‐grade transition program. (Doctoral Dissertation).. 2010
48. A Cognitively Based Anger Control Training Program with Aggressive Adolescents in School Settings. (Doctoral Dissertation).. 1995
49. The effects of the teacher‐advisor system on the students of omaha technical high school, Omaha, Nebraska, in the areas of attitude, attendance, behavior, and achievement. (Doctoral Dissertation).. 1985
50. Positive behavior intervention and support: An alternative approach. (Doctoral Dissertation).. 2012
51. Assessing the effectiveness of a behavior intervention model on fifth grade in‐school suspension rates. (Doctoral Dissertation).. 2012
52. The effects of implementing restorative justice in an in‐school suspension program. (Doctoral Dissertation).. 2011
53. The effect of PBIS on the school environment and test scores. (Doctoral Dissertation).. 2013
54. A case study of reducing a junior high school disciplinary infractions and increasing a positive school climate: After‐school detentions vs. evidence‐based computer program. (Doctoral Dissertation).. 2009
55. Reducing Suspensions of Minority Males through a Group Guidance/Mentoring Intervention Program. (Master Thesis).. 1995
56. An evaluation of the short‐term influence of Lock Haven University’s Upward Bound program. (Doctoral Dissertation).. 2001
57. Chatzinikolaou, R. (2015). Strength‐based interventions in secondary school: How can they be most helpful for pupils at risk and not‐at‐risk of exclusion?(Doctoral Dissertation). University of East London. Retrieved from http://roar.uel.ac.uk/4432/
58. The effectiveness of supervised discipline centers as an alternative to suspensions in two public secondary schools in Prince George’s County, Maryland. (Doctoral Dissertation).. 1980
59. Program evaluation: A study to evaluate the effectiveness of an alternative Education program in a rural county of a mid‐Atlantic U.S. State. (Doctoral Dissertation).. 2008
60. The effects of group counseling compared with individual counseling in the reduction of at‐risk behaviors using grade point average, discipline referrals, and attendance among sixth grade black males. (Doctoral Dissertation).. 2005
61. An evaluation of the effectiveness of the Check‐in/Check‐out intervention for students engaging in relational aggression and their classmates. (Doctoral Dissertation).. 2014
62. Orientation to continuation high school: A controlled experiment. (Doctoral Dissertation).. 1990
63. The effectiveness of block scheduling. (Master Thesis),. 2000
64. The effects of online education on the at‐risk student in an alternative school setting. (Doctoral Dissertation).. 2012
65. Program evaluation of universal Positive Behavioral Interventions and Supports in Kentucky. (Doctoral Dissertation).. 2011
66. A comparison based on student outcomes of an individualized student improvement program with an in‐school suspension program. (Doctoral Dissertation).. 1998
67. Effects of quality and quantity of the mentoring relationship on outcomes for at‐risk early adolescent girls. (Doctoral Dissertation).. 2009
68. A program evaluation of the peer mediation programs in the junior high schools of a large urban school system. (Doctoral Dissertation).. 1995
69. Effects of in‐school suspension with conflict resolution training on suspensions, referrals, and normative beliefs about aggression. (Doctoral Dissertation).. 2006
70. The relationship of disability status on attendance, behavior, and achievement indicators for students receiving intensive school‐based mental health counseling. (Doctoral Dissertation).. 2015
71. School uniforms in urban public high schools. (Doctoral Dissertation).. 2005
72. Evaluation of the union county alternative to suspension program. (Doctoral Dissertation).. 2010
73. School‐Wide Positive Behavior Interventions and Supports: A case study measuring implementation fidelity and implementation impact. (Doctoral Dissertation).. 2015
74. Implementing successful middle school and elementary school tier one positive behavior intervention support strategies in a high school. (Doctoral Dissertation).. 2011
75. The effects of block scheduling in a high school. (Doctoral Dissertation).. 1996
76. Behavioral treatment of alienated adolescents. (Doctoral Dissertation).. 1980
77. Effects of a middle school counseling program on attendance, academic achievement, and suspensions with a study of attitudes of students, principals, counselors and teachers toward the program. (Doctoral Dissertation).. 1980
78. The effects of an urban middle school dropout prevention program on the academic achievement, attendance, attitudes, and conflict resolution skills of at‐risk student. (Doctoral Dissertation).. 1994
79. Alternative schooling: The influence of a positive peer culture model in a large urban public school setting. (Doctoral Dissertation).. 2004
80. Bully prevention, peer mediation and conflict resolution: Impact of prevention programs on reducing school suspensions. (Doctoral Dissertation).. 2012
81. An analysis of the impact of emotional literacy instruction on at‐risk students. (Doctoral Dissertation).. 2015
82. A comparison research study on the use of school uniforms and graduation, attendance, and suspension rates in East Tennessee. (Doctoral Dissertation).. 2011
83. Increasing academic, behavioral and psychological engagement among at risk junior high students. (Doctoral Dissertation).. 2006
84. System‐wide implementation of an assertive discipline‐based behavior management plan: A program evaluation. (Doctoral Dissertation).. 1988
85. Character education programs and student suspension rates from school: Do character education programs decrease student suspensions from regular instructional public elementary schools in Texas?. (Doctoral Dissertation).. 2005
86. The disproportionate use of discipline: An investigation of the potential impact of school‐wide positive behavioral interventions and supports. (Doctoral Dissertation).. 2013
87. The effects of a research based high school dropout prevention program on multiple student outcomes. (Doctoral Dissertation).. 1996
88. An evaluation of the Technical Preparation Program in the Tennessee public school system. (Doctoral Dissertation).. 2003
89. Disciplinary referrals in response to School‐Wide Positive Behavior Plan in a rural middle school setting. (Doctoral Dissertation).. 2011
90. An evaluation of the effects of the academics and behavior Check‐in/Check‐out intervention. (Doctoral Dissertation).. 2013
91. Positive Behavior Supports: A study of implementation at the intermediate school level. (Doctoral Dissertation).. 2011
92. A mixed methods examination of resilience and social connectedness among youth in an alternative to suspension program. (Doctoral Dissertation).. 2012
93. An analysis of data and student opinion regarding two alternatives for reducing aberrant student behavior at Niles East High School. (Doctoral Dissertation).. 1980
94. Evaluating a school‐based day treatment program for students with challenging behaviors. (Doctoral Dissertation).. 2014
95. An examination of the effectiveness of the louisiana gear up program in promoting self‐efficacy, improving academic achievement and increasing teachers’ aspirations for their students. (Doctoral Dissertation).. 2014
96. The effects of positive behavioral interventions and supports on students behavior in grade five. (Doctoral Dissertation).. 2011
97. Hochman, J. S. (1986). The effectiveness of in‐school suspension as a disciplinary technique in an urban high school with and without intervention strategies. (Dissertation). Virginia Polytechnic Institute & State University, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=1987‐53042‐001&site=ehost‐live&scope=site Available from EBSCOhost psyh database.
98. Project C.A.R.E. substance abuse prevention program: An evaluation of program effectiveness. (Doctoral Dissertation).. 1995
99. Effect of group counseling on at‐risk African‐American female students. (Doctoral Dissertation).. 2008
100. A Comparison study of the San Francisco community board whole schools conflict resolution project. (Master Thesis),. 1999
101. After‐school intervention programs’ impact upon select variables among 10th grade students in the Education Longitudinal Study (ELS) 2002 dataset. (Doctoral Dissertation).. 2006
102. The effectiveness of a proactive school‐wide approach to discipline at the middle school level. (Doctoral Dissertation).. 2008
103. The impact of developmental counseling in relation to selected variables associated with disruptive behavior of public secondary school students. (Doctoral Dissertation).. 1980
104. Analyses of the impact of school uniforms on violence in North Carolina public high schools. (Doctoral Dissertation).. 2010
105. Measuring the efficacy of a ninth grade academy on students with disabilities. (Doctoral Dissertation).. 2008
106. An evaluation of an alternative discipline method: The saturday suspension program. (Doctoral Dissertation).. 1984
107. Assessing an induction program designed to improve achievement of ninth grade students receiving federal lunch benefits. (Doctoral Dissertation).. 2013
108. Social and emotional learning in the schools: The effects of Why Try as an alternative to suspension program. (Doctoral Dissertation).. 2012
109. Examination of the fidelity of School‐Wide Positive Behavior Support implementation and its relationship to academic and behavioral outcomes in Florida. (Doctoral Dissertation).. 2009
110. Impact of school‐wide PBS on indicators of social development and academic performance in an inner‐city middle school. (Doctoral Dissertation).. 2007
111. The effect of an in‐class behavioral intervention plus differentiated instruction program on the achievement and behavior outcomes of verbally disruptive 8th‐grade students with and without co‐occurring reading delimitations. (Doctoral Dissertation).. 2009
112. The effects of the responsible discipline and IMPACT transition models on the behaviors and attitudes of ninth‐grade students in a large urban high school. (Doctoral Dissertation).. 2003
113. Effectiveness of animal‐assisted therapy interventions in altering childhood aggressive behaviors. (Doctoral Dissertation).. 2009
114. Diminishing the discipline gap: Restorative justice as a promising alternative in one urban school. (Doctoral Dissertation).. 2015
115. The impact of a literacy intervention program on student achievement and behavior at two urban high schools. (Doctoral Dissertation).. 2012
116. Block scheduling: A comparison of block scheduling and traditional scheduling and its relationship to student achievement, student satisfaction and teacher methodology in a junior‐senior high school in New Jersey. (Doctoral Dissertation).. 1999
117. A longitudinal study of two at‐risk student programs in Henry County. (Doctoral Dissertation).. 1997
118. An evaluation of the assertive discipline program at Charles D. Owen high school. (Doctoral Dissertation).. 1989
119. An evaluation of the High Point, North Carolina Cities in Schools Program. (Doctoral Dissertation).. 1991
120. The impact of an alternative education intervention (Student transition and recovery) on middle schools’ attendance, academic performance, and discipline. (Doctoral Dissertation).. 2011
121. Girl power: The impact of an academic and social support program on the educational outcomes of urban seventh and eighth grade Latinas. (Doctoral Dissertation).. 2007
122. The effectiveness of a school‐based drug and alcohol treatment program. (Doctoral Dissertation).. 2004
123. The effect of a master schedule on student discipline and classroom attendance. (Doctoral Dissertation).. 2006
124. An analysis of the relationship between the Fight Free Program and school climate. (Doctoral Dissertation).. 2002
125. Examining the effectiveness of a school‐based mental health center’s services. (Doctoral Dissertation).. 2007
126. The effect of conflict management training programs in preventing discipline problems of students in grades nine through twelve in a suburban school district. (Doctoral Dissertation).. 1996
127. An exploration of the effects of a behavior modification program and a school/business partnership upon adolescents’ self‐esteem and school performance. (Doctoral Dissertation).. 1989
128. An exploratory study on the effect of restorative justice on school success and disciplinary incidents. (Master Thesis),. 2008
129. The impact of the ‘Social And Emotional Aspects Of Learning’ framework on primary aged pupils’ learning, attendance and behaviour. (Doctoral Dissertation).. 2012
130. A study of school culture, leadership, teacher quality and student outcomes via a performance framework in elementary schools participating in a school reform initiative. (Doctoral Dissertation).. 2009
131. Survey of students’ perceptions of assertive discipline and restorative discipline in American Christian Academy, Ibadan, Nigeria. (Master Thesis),. 2015
132. An examination of the leadership practices that support and sustain school based mental health programs. (Doctoral Dissertation).. 2013
133. The effects of character education in South Carolina’s secondary schools. (Doctoral Dissertation).. 2012
134. The impact of a school‐based enterprise program on the achievement and behavior of special education students attending high schools in South Carolina. (Doctoral Dissertation).. 2011
135. Exploring the value of engagement mentoring as a preventative strategy with at‐risk youth. (Doctoral Dissertation).. 2010
136. The impact of a School‐Wide Token economy on behavior, attendance, and academics at Morgan High School. (Master Thesis),. 2007
137. The effect of implementing School‐Wide Positive Behavior Interventions and Supports on student misbehavior in a large urban high school. (Doctoral Dissertation).. 2012
138. Pratt, L. (2009). Supporting pupils at risk of exclusion: an evaluation of an intensive, out‐of‐school, emotional literacy programme for key stage 3 pupils. Institute of Education, University of London.
139. The effectiveness of different disciplinary programs in two New Jersey high schools. (Doctoral Dissertation).. 2007
140. An examination of treatment integrity practices and behavioral outcomes when utilizing the Second Step curriculum. (Doctoral Dissertation).. 2004
141. A longitudinal study to determine the long‐term effects of prevention and intervention substance abuse strategies on at‐risk fourth grade students and their families. (Doctoral Dissertation).. 1996
142. The efficacy of a violence prevention intervention with elementary school children. (Doctoral Dissertation).. 2004
143. Program evaluation: A study to evaluate the effectiveness of academic and behavioral interventions in an alternative to out of school suspension program. (Doctoral Dissertation).. 2011
144. Investigation of three specific intervention strategies to reduce the attrition rate of the potential high school dropout. (Doctoral Dissertation).. 1985
145. The academic effectiveness of school‐site drug intervention programs. (Master Thesis),. 1998
146. An applied experimental evaluation of conflict resolution curriculum and social skills development. (Doctoral Dissertation).. 1997
147. The relationship of school uniforms to students’ achievement, attendance, discipline referrals and perceptions: An analysis of one urban school district. (Doctoral Dissertation).. 2003
148. School violence: Evaluation of an elementary school peer mediation program. (Doctoral Dissertation).. 2005
149. Schmidt, P. A. M. (1995). Helping teen mothers make positive choices thereby reducing suspensions through a curriculum for conflict resolution/peer mediation. (Dissertation). Nova Southeastern University. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED386642&site=ehost‐live&scope=site Available from EBSCOhost eric database.
150. State and local level implementation of Schoolwide Positive Behavior Support: An examination of the Texas Behavior Support initiative (TBSI). (Doctoral Dissertation).. 2007
151. A study to determine what impact, if any, the implementation of a mandatory uniform dress policy has had on improving student behavior among ninth‐grade students in three california high schools. (Doctoral Dissertation).. 2000
152. The use of brief family counseling with middle school students experiencing discipline problems. (Doctoral Dissertation).. 1996
153. Impacts of prosocial intervention on delinquent and defiant behaviors in youth grades 7‐12: A program evaluation. (Doctoral Dissertation).. 2015
154. The impact of Schoolwide Positive Behavior Support on the discipline gap in the middle school setting. (Doctoral Dissertation).. 2010
155. The effects of the web transition program on middle school student gpa, attendance, and suspensions. (Doctoral Dissertation).. 2011
156. Positive behavior support in schools universal intervention: Suspension rates and school climate in implementing middle schools. (Doctoral Dissertation).. 2009
157. Decreasing suspensions in grades nine through twelve through the implementation of a peace curriculum. (Doctoral Dissertation).. 1994
158. An analysis of requiring school uniforms and its impact on student behavior: Implications for school reform. (Doctoral Dissertation).. 1999
159. An investigation of the effectiveness of bibliotherapy on middle grade students who repeatedly display inappropriate behavior in the school setting. (Doctoral Dissertation).. 1986
160. Freshman transition and its effectiveness on student success as measured by improved attendance, improved grades, decreased discipline referrals, and decreased dropout rate. (Doctoral Dissertation).. 2010
161. A quantitative study comparing traditional high schools and high schools implementing freshman academies in the state of Tennessee. (Doctoral Dissertation).. 2008
162. An evaluation of the effectiveness of PASS, a junior high school dropout prevention program. (Doctoral Dissertation).. 1990
163. A mentor program for improving the academic attainment of black adolescent males. (Doctoral Dissertation).. 1994
164. The effects of Check‐in Check‐out on the social and academic planning and outcomes of African‐American males in an urban secondary setting. (Doctoral Dissertation).. 2012
165. State‐level implementation of schoolwide positive behavior support: An evaluation of the Arizona Behavioral Initiative. (Doctoral Dissertation).. 2003
166. Examining the effectiveness of a ninth‐grade transition program. (Doctoral Dissertation).. 2009
167. The implementation of uniform dress policies: Student performance outcomes and perceptions of student impact at five public elementary schools in Hampton roads, Virginia. (Doctoral Dissertation).. 2001
168. The role of interscholastic sport in public high schools: A zero‐sum game or a bridge to success. (Doctoral Dissertation).. 2012
169. The positive alternative credit experience (PACE) program a quantitative comparative study. (Doctoral Dissertation).. 2011
170. The effects of school uniforms on attendance and discipline among schools with and without mandatory school uniform policies. (Doctoral Dissertation).. 2003
171. Effects of Positive Behavior Support Programs on Student Behaviors. (Doctoral Dissertation).. 2009
172. A mixed‐method study utilizing a token economy to shape behavior and increase academic success in urban students. (Doctoral Dissertation).. 2008
173. The impact of behavior intervention programs on student suspensions. (Doctoral Dissertation).. 2015
174. Reducing school stand‐downs: A case study of one New Zealand secondary school. (Master Thesis),. 2013
175. A comparative evaluation to determine the effectiveness of the behaviour support classrooms and other positive behaviour management interventions in designated disadvantaged schools. (Doctoral Dissertation).. 2010
176. An evaluation of a high school alternative to suspension program for students with gang‐related disciplinary violations. (Doctoral Dissertation).. 2004
177. The influence of participation in School‐Wide Positive Behavior Support on elementary student behavior. (Doctoral Dissertation).. 2015
178. Conflict resolution programs and the Houston independent school district: Peer mediation vs. non‐peer mediation approaches for resolving student conflict in elementary and middle schools. (Master Thesis),. 1998
179. Wolgemuth, J. , Cobb, R. , & Dugan, J. (2007). The effects of self‐management interventions on academic outcomes for youth with disabilities. Ft. Collins, CO: Colorado State University, School of Education.
180. Woodfork, J. D. (1987). The effects of an assertive discipline approach on disciplnary problems in selected high schools. (Education Especialist Degree), Atlanta University.
181. The impact of a ninth and tenth grade academy program on student attendance, discipline, and achievement. (Doctoral Dissertation).. 2006
182. The impact of problem‐based learning on ninth‐grade african american male disengagement in high school. (Doctoral Dissertation).. 2012
183. Group counseling compared with individual counseling in the reduction of at‐risk behaviors in black male students. (Doctoral Dissertation).. 2001
184. An elementary school mediation program: Its effect on student mediators and school violence. (Doctoral Dissertation).. 1997
185. Sociocultural correlates of disciplinary exclusion among students with emotional, behavioral, and learning disabilities in the SEELS national dataset.. Journal of Emotional and Behavioral Disorders,, 2007
186. A critical review of strategies to reduce school violence.. Children & Schools,, 1995
187. Programmatic prevention of adolescent problem behaviors: The role of autonomy, relatedness, and volunteer service in the Teen Outreach Program.. American Journal of Community Psychology,, 1994. [PubMed]
188. Who benefits most from a broadly targeted prevention program? Differential efficacy across populations in the Teen Outreach Program.. Journal of Community Psychology,, 2001. [PubMed]
189. School‐based social work interventions: A cross‐national systematic review.. Social Work,, 2013. [PubMed]
190. Evaluation of an alternative discipline program.. The High School Journal,, 1998
191. The persistent effect of race and the promise of alternatives to suspension in school discipline outcomes.. Children and Youth Services Review,, 2014
192. The positive alternative learning supports program: collaborating to improve student success.. Teaching Exceptional Children,, 2007
193. Suspensions and detentions in an urban, low‐income school: Punishment or reward?. Journal of Abnormal Child Psychology,, 2002. [PubMed]
194. Does Head Start yield long‐term benefits?. The Journal of Human Resources,, 2001
195. Spiralling up and spiralling down: Implications of a long‐term study of temperament and conduct disorder for social work with children.. Child & Family Social Work,, 2000
196. The reduction of problem behaviours and school exclusion in at‐risk youth: An experimental study of school social work with cost‐benefit analyses.. Child & Family Social Work,, 1998
197. School‐related and social‐emotional outcomes of providing mental health services in schools.. Community Mental Health Journal,, 2014. [PubMed]
198. The association between school exclusion, delinquency and subtypes of cyber‐ and F2F‐victimizations: Identifying and predicting risk profiles and subtypes using latent class analysis.. Child Abuse & Neglect,, 2015. [DOI | PubMed]
199. Bartik, T. J. , & Lachowska, M. (2012). The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes. Upjohn Institute Working Paper No. 12‐186. W. E. Upjohn Institute for Employment Research.
200. The Kalamazoo Promise Scholarship.. Education Next,, 2014
201. The effectiveness of the Olweus Bullying Prevention Program in Public Middle Schools: A Controlled Trial.. Journal of Adolescent Health,, 2007
202. Use of a social and character development program to prevent substance use, violent behaviors, and sexual activity among elementary‐school students in Hawaii.. Research and Practice,, 2009
203. The effectiveness of peer mediation in a low‐SES rural elementary school.. Psychology in the Schools,, 2000
204. Appreciative evaluation of restorative approaches in schools.. Pastoral Care in Education,, 2015
205. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): A pilot randomised controlled trial.. Health Technology Assessment,, 2015
206. Exclusion of primary school students: Archival analysis of school records over a 30‐year period.. Reimagining Practice: Researching Change,, 2003
207. The Youth Crime Reduction Video Project: An evaluation of a pilot intervention targeting young people at risk of crime and school exclusion.. The Howard Journal,, 2006
208. Implementation and evaluation of a comprehensive, school‐wide bullying prevention program in an urban/suburban middle school.. Journal of School Health,, 2011
209. The impact of the home instructional program for preschool youngsters (HIPPY) on school performance in 3rd and 6th grades.. Early Education and Development,, 2002
210. Examining the effects of schoolwide positive behavioral interventions and supports on student outcomes. Results from a randomized controlled effectiveness trial in elementary schools.. Journal of Positive Behavior Interventions,, 2010
211. School‐police partnership effectiveness in urban schools ‐ An analysis of New York City’s Impact Schools Initiative.. Education and Urban Society,, 2007
212. Conflict resolution training as an alternative to suspension for violent behavior.. Journal of Educational Research,, 2002. [DOI]
213. Making the Smart Choice: A systemic response to school‐based violence.. Journal of Family Therapy,, 2006
214. The impact of school mental health services on out‐of‐school suspension rates.. Journal of Youth and Adolescence,, 2005
215. Implementation of a classroom management program with urban elementary schools in low‐income neighborhoods: Does program fidelity affect student behavior and academic outcomes?. Journal of Education for Students Placed at Risk (JESPAR),, 2011. [DOI]
216. ‘Over to You’: Group work to help pupils avoid school exclusion.. Educational Psychology in Practice,, 2006. [DOI]
217. Peer mediation and middle school students.. Perspectives in Peer Programs,, 2011
218. Effects of innovative programming on youngsters at risk.. Journal of At‐Risk Issues,, 1997
219. HIPP: A comprehensive school‐based substance abuse program with cooperative community involvement.. Journal of Primary Prevention,, 1990
220. Reducing juvenile recidivism: Evaluating the wraparound services model.. Research on Social Work Practice,, 2003
221. Promoting prevention: Evaluating a multi‐agency initiative of Youth Consultation and Crime Prevention in Swansea.. Children & Society,, 2004
222. Delinquency prevention through vocational entrepreneurship: The new Smyrna Beach Employability skill training model for youthful offenders.. Preventing School Failure: Alternative Education for Children and Youth,, 1996
223. Positive youth development in the United States: Research findings on evaluations of Positive Youth Development programs.. The Annals of the American Academy,, 2004
224. Integrating home, school, and community resources: Evaluation of a district‐wide prevention program.. Reclaiming Children & Youth,, 2004
225. Targeting exclusion, disaffection and truancy in secondary schools: An evaluation of an alternative curriculum for older pupils.. Emotional and Behavioural Difficulties,, 2004
226. In‐school alternatives to suspension: A description of Ten School District programs.. The Urban Review,, 1982
227. Early intervention in emotional and behavioural difficulties: The role of Nurture Groups.. European Journal of Special Needs Education,, 1999
228. A retrospective study of school safety conditions in high schools using the Virginia Threat Assessment Guidelines versus alternative approaches.. School Psychology Quarterly,, 2009
229. Reductions in long‐term suspensions following adoption of the Virginia Student Threat Assessment Guidelines.. NASSP Bulletin,, 2011
230. Guidelines for student threat assessment: Field‐test findings.. School Psychology Review,, 2004
231. Modeling the bullying prevention program design recommendations of students from grades five to eight: A discrete choice conjoint experiment.. Aggressive Behavior,, 2011. [PubMed]
232. Implementation of an empirically based drug and violence prevention and intervention program in public school settings.. Journal of Clinical Child & Adolescent Psychology,, 2001
233. Outcomes of a School‐Wide Positive Behavioral Support Program.. Professional School Counseling,, 2010
234. Three‐year longitudinal study of school behavior and academic outcomes: Results from a comprehensive expanded school mental health program.. Advances in School Mental Health Promotion,, 2014
235. Discipline with dignity: Oakland classrooms try healing Instead of punishment.. Reclaiming Children and Youth,, 2014
236. A Study on the effectiveness of life space crisis Intervention for students identified with emotional disturbances.. Reclaiming Children and Youth,, 2003
237. The multifaceted conflict cycle. A New York City perspective.. Reclaiming Children and Youth,, 2015
238. The Helping Families Programme: A new parenting intervention for children with severe and persistent conduct problems.. Child and Adolescent Mental Health,, 2011. [PubMed]
239. A strengths‐based approach to promoting prosocial behavior among African American and Latino students.. Professional School Counseling,, 2008
240. The evolution of juvenile justice: Community‐based partnerships through balanced and restorative justice.. Corrections Today,, 2005
241. An alternative to suspension for middle school youths with behavior problems: Findings from a “school survival” group.. Research on Social Work Practice,, 1998
242. School‐based social‐cognitive skills training for middle school students with school behavior problems.. Children and Youth Services Review,, 1993
243. Aay it straight: Adolescent substance abuse prevention training.. Academic Psychology Bulletin,, 1985
244. Brief SAY IT STRAIGHT training and follow‐up in adolescent substance abuse prevention.. Journal of Primary Prevention,, 1986
245. Communication‐skills and self‐esteem in prevention of destructive behaviors.. Adolescence,, 1989. [PubMed]
246. The role of early, multilevel youth development programs in preventing health risk behavior in adolescents and young adults.. Archives of Pediatrics & Adolescent Medicine,, 2002. [PubMed]
247. Discipline provisions of IDEA: Misguided policy or tacit reform initiatives?. Behavioral Disorders,, 2002
248. The birth to school study: Evidence on the effectiveness of PEEP, an early intervention for children at risk of educational under‐achievement.. Oxford Review of Education,, 2007
249. Unravelling emotional, behavioural and educational outcomes in a longitudinal study of children in foster‐care.. British Journal of Social Work,, 2008. [DOI]
250. In search of differentiating features to enhance the performance and expectations of students at risk of educational exclusion.. Revista De Educacion,, 2011
251. Long‐term effects of the Positive Action® program.. American Journal of Health Behavior,, 2003. [PubMed]
252. Interventions addressing the social determinants of teenage pregnancy.. Health Education,, 2007. [DOI]
253. Classroom management—a pathway to student achievement: A study of fourteen inner‐city elementary schools.. The Elementary School Journal,, 2009
254. Self‐coded indirect memory associations in a brief school‐based intervention for substance use suspensions.. Psychology of Addictive Behaviors,, 2009. [PubMed]
255. An alternative approach to school suspension: The Dothan Model CHECK UL.. Phi Delta Kappan,, 1980
256. Academic achievement and School‐Wide Positive Behavior Supports.. Journal of Disability Policy Studies,, 2015
257. Conflict resolution training, teacher effectiveness, and student suspension: The impact of a health and safety initiative in the New Orleans public schools.. Journal of Negro Education,, 1996
258. Dressed for success? The effect of school uniforms on student achievement and behavior.. Journal of Urban Economics,, 2012
259. Keeping kids in school: Restorative justice, punitive discipline, and the school to prison pipeline.. Journal of Law & Education,, 2012
260. An empirical test of school‐based environmental and individual interventions to reduce the risk of delinquent behavior.. Criminology,, 1986
261. Managing adolescent behavior. A multiyear, multischool study.. American Educational Research Journal,, 1993
262. Implementation and evaluation of a cognitive‐behavioral intervention to prevent problem behavior in a disorganized school.. Prevention Science,, 2002. [PubMed]
263. Prevention work with children disaffected from school: Findings from the evaluation of two innovative community‐based projects.. Health Education,, 2005
264. Welcome to the House System.. Educational Leadership,, 2006
265. Group therapy for adolescents with repeated self harm: Randomised controlled trial with economic evaluation.. British Medical Journal,, 2011
266. Examining disproportionality in school discipline for Aboriginal students in schools implementing PBIS.. Canadian Journal of School Psychology,, 2014
267. Screening, assessment, treatment, and outcome evaluation of behavioral difficulties in an RTI model.. Exceptionality,, 2013
268. Effectiveness of a social skills training program using self/other perspective‐taking: A nine‐month follow‐up.. American Journal of Orthopsychiatry,, 2000
269. The effect of conflict resolution training on Turkish elementary school students: A quasi‐experimental investigation.. International Journal for the Advancement of Counselling,, 2004
270. Promoting prevention: Targeting family‐based risk and protective factors for drug use and youth offending in Swansea.. British Journal of Social Work,, 2005
271. Evaluation of behavioural management in schools: A review of the behaviour improvement programme and the role of behaviour and education support teams.. Child and Adolescent Mental Health,, 2007. [PubMed]
272. Exclusion from school: What can help prevent it?. Educational Review,, 2001
273. The coalition for nonviolence in schools. A case study in Milwaukee, Wisconsin, USA.. Peace Research,, 1998
274. Multi‐disciplinary approaches to pupil behaviour in school. The role of evaluation in service delivery.. Educational Psychology in Practice,, 2010
275. Evaluation of comprehensive violence prevention education: Effects on student behavior.. Journal of Adolescent Health,, 1996
276. Preventing adolescent health‐risk behaviors by strengthening protection during childhood.. Archives of Pediatrics & Adolescent Medicine,, 1999. [PubMed]
277. Family group conferences.. Are they an effective and viable way of working with attendance and behaviour problems in schools? British Educational Research Journal,, 2009
278. ‘Safer Cities’ and exclusion from school.. Journal of Youth Studies,, 1998
279. Behaviour support in secondary schools: what works for schools?. Emotional and Behavioural Difficulties,, 2003
280. Principal attitudes regarding zero tolerance and racial disparities in school suspensions.. Psychology in the Schools,, 2015
281. Does school suspension affect subsequent youth non‐violent antisocial behaviour? A longitudinal study of students in Victoria, Australia and Washington State, United States.. Australian Journal of Psychology,, 2013. [PubMed]
282. Rates of student‐reported antisocial behavior, school suspensions, and arrests in Victoria, Australia and Washington state, United States*.. The Journal of School Health,, 2007. [PubMed]
283. Student and school factors associated with school suspension: A multilevel analysis of students in Victoria, Australia and Washington State, United States.. Children and Youth Services Review,, 2014. [PubMed]
284. Are rates of school suspension higher in socially disadvantaged neighbourhoods? An Australian study.. Health Promotion Journal of Australia,, 2010. [PubMed]
285. Using mixed methods to explore resilience, social connectedness, and re‐suspension among youth in a community‐based alternative‐to‐suspension Program.. International Journal of Child, Youth and Family Studies,, 2014
286. Predictors and responses to the growth in physical violence during adolescence: A comparison of students in Washington State and Victoria, Australia.. American Journal of Orthopsychiatry,, 2012
287. Middle school black and white student assignment to disciplinary consequences. A clear lack of equity.. Education and Urban Society,, 2014
288. Supporting inclusion of at risk students in secondary school through positive behaviour support.. International Journal of Inclusive Education,, 2013
289. In‐school suspension and group counseling: Helping the at‐risk student.. NASSP Bulletin,, 1987
290. Targeting non‐cognitive skills to improve cognitive outcomes: Evidence from a remedial education intervention.. 2009
291. Principles for defining adverse events in behavioral intervention research: Lessons from a family‐focused adolescent drug abuse trial.. Clinical Trials,, 2010. [PubMed]
292. Attributional bias and reactive aggression.. American Journal of Preventive Medicine,, 1996. [PubMed]
293. Personalized behavior modification: An in‐school suspension program that teaches students how to change.. The School Counselor,, 1988
294. An evaluation of a short cognitive‐behavioural anger management intervention for pupils at risk of exclusion.. Emotional & Behavioural Difficulties,, 2006
295. After‐school program effects on behavior: Results from the 21st century community learning centers program national evaluation.. Economic Inquiry,, 2008
296. Localizing restorative justice: An in‐depth look at a Denver public school program.. Restorative Justice: From Theory to Practice,, 2008
297. Conflict resolution and peer mediation programs in elementary and secondary schools: A review of the research.. Review of Educational Research,, 1996. [DOI]
298. Evaluating national initiatives: The case of ‘On Track’.. Children & Society,, 2001
299. School‐wide positive behavioral interventions and supports in a residential school for students with emotional and behavioral disorders: First years of implementation and maintenance follow‐up focus groups.. Residential Treatment for Children & Youth,, 2014
300. Reducing exclusions whilst developing effective intervention and inclusion for pupils with behaviour difficulties.. Emotional and Behavioural Difficulties,, 2004
301. Long‐term effects of Head Start on academic and school outcomes of children in persistent poverty: Girls vs. boys.. Children and Youth Services Review,, 2010
302. School‐based mental health program evaluation: Children’s school outcomes and acute mental health service use.. Journal of School Health,, 2013
303. Assessing effectiveness of students taking a right stand (STARS) Nashville Student Assistance Programs.. Journal of Child & Adolescent Substance Abuse,, 2015
304. A response to intervention model to promote school attendance and decrease school absenteeism.. Child & Youth Care Forum,, 2014
305. The effects of positive verbal reinforcement on the time spent outside the classroom for students with emotional and behavioral disorders in a residential setting.. Behavioral Disorders,, 2008
306. Making schools safe: A system‐wide school intervention to increase student prosocial behaviors and enhance school climate.. Journal of Applied School Psychology,, 2006
307. Impact of a comprehensive whole child intervention and prevention program among youths at risk of gang involvement and other forms of delinquency.. Children & Schools,, 2009
308. Effects of after‐school programs with at‐risk youth on attendance and externalizing behaviors: A systematic review and meta‐analysis.. Journal of Youth & Adolescence,, 2015. [PubMed]
309. Effectiveness of the parent connectors program: Results from a randomized controlled trial.. School Mental Health,, 2013
310. The use of evidence‐based instructional strategies in special education settings in secondary schools: Development, implementation and outcomes.. Teaching and Teacher Education,, 2009
311. The relationship of school‐wide positive behavior support to academic achievement in an urban middle school.. Psychology in the Schools,, 2006
312. Promoting the communication skills of primary school children excluded from school or at risk of exclusion: An intervention study.. Child Language Teaching and Therapy,, 2003. [DOI]
313. Determining the efficacy of a relational aggression intervention for urban African American girls. Retrieved from. 2015
314. A randomized study of neighborhood effects on low‐income children’s educational outcomes.. Developmental Psychology,, 2004. [PubMed]
315. Positive alternatives to school exclusion: An empirical investigation.. Emotional and Behavioural Difficulties,, 1997
316. Whole‐school positive behaviour support: Effects on student discipline problems and academic performance.. Educational Psychology,, 2005
317. Lost lessons?: The experience of a time‐limited Home‐School support project.. Children & Society,, 1999
318. School‐wide discipline and the intransigency of exclusion.. Children and Youth Services Review,, 2012
319. Evaluating three school‐based integrated health centres established by a partnership in cornwall to inform future provision and practice.. International Journal of Educational Management,, 2013
320. Early interventions: Preventing school exclusions in the primary setting.. Westminster studies in education,, 2003
321. Severe sanctions, easy choice? Investigating the role of school sanctions in preventing adolescent violent offending.. Criminology,, 2012. [DOI]
322. Psychology and interventions for pupils at risk of underachievement and school exclusion: The year 10 effect.. International Journal of Psychology,, 2008
323. Increasing prosocial behavior and academic achievement among adolescent African American males.. Adolescence,, 2007. [PubMed]
324. Preliminary study of a brief intervention program for adolescents who initiate alcohol and other drugs consumption.. Salud Mental,, 2008
325. School violence interventions in the safe schools/healthy students initiative: Evaluation of two early intervention programs.. Journal of School Violence,, 2007
326. TOPS: An alternative approach for the at‐risk middle school student.. Journal of At‐Risk Issues,, 1999
327. First‐year effectiveness on school functioning of a self‐contained ED middle school.. Behavioral Disorders,, 2009
328. Investigation of the construct of trait emotional intelligence in children.. European Child & Adolescent Psychiatry,, 2008. [PubMed]
329. Exploring alternatives to out‐of‐school suspension in Kentucky: A quasi‐experimental study examining the effectiveness of community service work.. PRISM: A Journal of Regional Engagement,, 2015
330. School‐based clinic use and school performance.. Journal of Adolescent Health,, 1993
331. The effect of schoolwide positive behavioral supports on children in impoverished rural community schools.. Preventing School Failure,, 2012
332. Using a changing‐criterion design to evaluate the effects of Check‐in/Check‐out with goal modification.. Journal of Positive Behavior Interventions,, 2015
333. Effectiveness of A‐CRA/ACC in treating adolescents with cannabis‐use disorders.. Community Mental Health Journal,, 2014. [PubMed]
334. Efficacy of LAIV‐T on absentee rates in a school‐based health center sample.. Journal of Adolescent Health,, 2009
335. The Implementation of Positive Behavioral Support in an elementary school: Processes, procedures, and outcomes.. The Alberta Journal of Educational Research,, 2008
336. The effects of the Good Behavior Game with general‐education high school students.. School Psychology Review,, 2015. [DOI]
337. Policy in practice: Enabling and inhibiting factors for the success of suspension centres.. Australian Journal of Teacher Education,, 2014. [DOI]
338. The effects of volunteering on the young volunteer.. The Journal of Primary Prevention,, 1996. [PubMed]
339. Evaluation of a program for developing socio‐emotional competencies in preschool children.. Procedia ‐ Social and Behavioral Sciences,, 2011
340. An early college initiative in an urban, high‐poverty high school: First‐year effects on student achievement and non‐academic indicators.. Journal of Education for Students Placed at Risk (JESPAR),, 2014. [DOI]
341. Peer contexts: Do old for grade and retained peers influence student behavior in middle school?. Teachers College Record,, 2014
342. Positive Behavioral Interventions and Supports in New Hampshire Effects of Large‐Scale Implementation of Schoolwide Positive Behavior Support on Student Discipline and Academic Achievement.. Journal of Positive Behavior Interventions,, 2008
343. Creating home‐school partnerships by engaging families in schoolwide positive behavior supports.. Teaching Exceptional Children,, 2008
344. School‐based secondary prevention programmes for preventing violence.. Cochrane Database Syst Rev (, 2006
345. Positive behavioral intervention in children who were wards of the court attending a mainstream school.. Psychological Reports,, 2007. [PubMed]
346. The child outcomes of a behavior model.. Exceptional Children,, 2009
347. Impact of School‐Wide Positive Behavior Supports on student behavior in the middle grades.. RMLE Online: Research in Middle Level Education,, 2014
348. Relationship between school suspension and student outcomes: A meta‐analysis.. School Psychology Review,, 2015
349. London Education and Inclusion Project (LEIP): A cluster‐randomised controlled trial protocol of an intervention to reduce antisocial behaviour and improve educational/occupational attainment for pupils at risk of school exclusion.. BMC Psychology,, 2014
350. Long‐term effects of nurse home visitation on children’s criminal and antisocial behavior. 15‐year follow‐up of a randomized controlled trial.. Journal of the American Medical Association,, 1998. [PubMed]
351. Bullying in school: Evaluation and dissemination of the Olweus Bullying Prevention program.. American Journal of Orthopsychiatry,, 2010
352. Evaluating parenting classes held at a secondary school.. Research in Post‐Compulsory Education,, 2007
353. Evaluation of the safe school ambassadors program: A student‐led approach to reducing mistreatment and bullying in schools.. Clearing House: A Journal of Educational Strategies, Issues and Ideas,, 2011
354. A follow‐up of a home and school support project for children with emotional and behavioural difficulties.. Emotional and Behavioural Difficulties,, 2004
355. Early intervention and prevention for children excluded from primary schools.. International Journal of Inclusive Education,, 2007
356. The relationships between after‐school programs, academic outcomes, and behavioral developmental outcomes of Latino children from immigrant families: Findings from the 2005 National Household Education Surveys Program.. Children and Youth Services Review,, 2015
357. Persistent young offenders: An evaluation of two projects.. British Journal of Social Work,, 2008. [DOI]
358. A multilevel exploration of the influence of teacher efficacy and burnout on response to student problem behavior and school‐based service use.. School Psychology Quarterly,, 2010
359. Implementation of a comprehensive schoolwide behavioral intervention: The ABC program.. Journal of Attention Disorders,, 2005. [PubMed]
360. An alternative program to improve student behavior: The focus program.. Thresholds in Education, November,, 1996
361. A journey of self‐discovery: An intervention involving massage, yoga and relaxation for children with emotional and behavioural difficulties attending primary schools.. European Journal of Special Needs Education,, 2008
362. A consumer view study of young people’s views on their educational social worker: Engagement as a measure of an effective relationship.. British Journal of Social Work,, 1998
363. A three‐year comparative longitudinal study of a school‐based social work family service to reduce truancy, delinquency and school exclusions.. Journal of Social Welfare & Family Law,, 2001
364. Does social work make a difference?. A controlled study of former ‘looked‐after‐children’ and ‘excluded‐from‐school’ adolescents now men aged 16‐24 subsequent offences, being victims of crime and suicide.. Journal of Social Work,, 2009
365. Improving student bus‐riding behavior through a whole‐school intervention.. Journal of Applied Behavior Analysis,, 2003. [PubMed]
366. Can a self‐regulation skills and cultural arts program promote positive outcomes in mental health symptoms and academic achievement for at‐risk youth?. Journal of Social Service Research,, 2011
367. School‐based, short‐term group treatment for behaviorally disturbed young adolescent males. A pilot intervention.. Journal of School Health,, 1987
368. The effect of a professional development classroom management model on at‐risk elementary students’ misbehaviors.. Education,, 2012
369. A strategic approach to tackling school absenteeism and truancy: The PSCC scheme.. Educational Studies,, 2003
370. Strategic approaches to tackling school absenteeism and truancy: The traffic lights (TL) scheme.. Educational Review,, 2003
371. Are zero tolerance policies effective in the schools? An evidentiary review and recommendations.. American Psychologist,, 2008
372. A school‐based violence prevention model for at‐risk eighth grade youth.. Psychology in the Schools,, 2003
373. Encouraging engagement: an emerging role for school based family workers in english secondary schools.. International Journal on School Disaffection,, 2008
374. Does school suspension increase long‐term substance use?: Quantifying the impact of school suspension on young adult substance use 12 years after school suspension using the Rubin causal inference model.. Journal of Adolescent Health,, 2012
375. Bully prevention in positive behavior support.. Journal of Applied Behavior Analysis,, 2009. [PubMed]
376. L’exclusion scolaire au secondaire: évaluation d’un programme alternatif à la suspension scolaire. Suspension from secondary school: Evaluation of an alternative program.. Revue Canadienne de Psycho‐Education,, 1996
377. Maintaining or changing a drinking behavior? Goka’s short‐term outcomes.. Journal of Business Research,, 2015. [DOI]
378. A behavioral‐program for disruptive junior‐high school students: Results and follow‐up.. Journal of Abnormal Child Psychology,, 1981. [PubMed]
379. Overview of meta‐analyses of the prevention of mental health, substance use and conduct problems.. Annual Review of Clinical Psychology,, 2014
380. Reducing levels of elementary school violence with peer mediation.. Professional School Counseling,, 2007. [DOI]
381. Puppetry as a group counseling technique with middle school students.. Elementary School Guidance and Counseling,, 1985
382. A solution to exclusion: A total involvement model to support pupils and reduce exclusions.. Emotional & Behavioural Difficulties,, 2002
383. Youth initiated mentoring: Investigating a new approach to working with vulnerable adolescents.. American Journal of Community Psychology,, 2013. [PubMed]
384. Using staff and student time engaged in disciplinary procedures to evaluate the impact of school‐wide PBS.. Journal of Positive Behavior Interventions,, 2004
385. Evaluation of the Peacemakers Program: School‐based violence prevention for students in grades four through eight.. Psychology in the Schools,, 2002
386. Early intervention in behaviour: A study of the FAST‐Track programme.. Emotional and Behavioural Difficulties,, 2003
387. Improving student behavior and school discipline with family and community involvement.. Education and Urban Society,, 2002
388. Illinois statewide Positive Behavioral Interventions and Supports: Evolution and impact on student outcomes across years.. Journal of Positive Behavior Interventions,, 2012
389. Relationships between implementing character education, student behavior, and student achievement.. Journal of Advanced Academics,, 2006
390. Teaching the social curriculum: School discipline as instruction.. Preventing School Failure,, 2003
391. “Discipline is always teaching”: Effective alternatives to zero tolerance in Indiana’s schools.. Education Policy Briefs.,, 2004
392. The safe and responsive schools project: A school reform model for implementing best practices in violence prevention.. 2005
393. Race is not neutral: A national investigation of African American and Latino disproportionality in school discipline.. School Psychology Review,, 2011
394. Decreasing in‐school suspensions with function‐based interventions.. Kappa Delta Pi Record,, 2012
395. Bullies, victims and bystanders: A method of in‐school intervention and possible parental contributions.. Child Psychiatry & Human Development,, 1999. [PubMed]
396. Schoolwide social‐behavioral climate, student problem behavior, and related administrative decisions.. Journal of Positive Behavior Interventions,, 2010. [DOI]
397. Reducing boys’ aggression: Learning to fulfill basic needs constructively.. Journal of Applied Developmental Psychology,, 2000
398. Reducing violence in the high school: The power of peers.. Journal of At‐Risk Issues,, 2000
399. A preliminary investigation of the relationship between in‐school suspension and the disruptive classroom behavior of students with behavioral disorders.. Behavioral Disorders,, 1997
400. Effective services to support children in special circumstances.. Child Care, Health and Development,, 2004
401. A pilot study of a brief motivational intervention to enhance parental engagement: A brief report.. Journal of Child and Family Studies,, 2010
402. Determining the efficacy of a community‐based early intervention program adapted for middle school.. J Child Adolesc Behav,, 2015
403. Educational outcomes in a system of care for children with emotional disturbance.. American Journal of Orthopsychiatry,, 2012
404. Beyond behavior: Multilevel analysis of the influence of sociodemographics and school characteristics on students’ risk of suspension.. School Psychology Review,, 2013
405. School suspensions: An in‐house prevention model.. Children today,, 1980
406. A meta‐analysis of the effects of dropout prevention programs on school absenteeism.. Prevention Science,, 2013. [PubMed]
407. School social workers’ perceived efficacy at tasks related to curbing suspension and undesirable behaviors.. Children & Schools,, 2011
408. A study of zero tolerance policies in schools: A multi‐integrated systems approach to improve outcomes for adolescents.. Journal of Child and Adolescent Psychiatric Nursing,, 2011. [PubMed]
409. School resource officers and the criminalization of student behavior.. Journal of Criminal Justice,, 2009
410. Multilevel evaluation of factors predicting school exclusion among middle and high school students.. Children and Youth Services Review,, 2010
411. Short‐term outcomes for youth receiving runaway and homeless shelter services.. Research on Social Work Practice,, 2002
412. Peer mediation: A peaceful solution.. The School Counselor,, 1996
413. The research and development in organisations approach and the evaluation of a mainstream behaviour support initiative.. Educational Psychology in Practice,, 2003
414. Evaluation of a mental health outreach service for homeless families.. Archives of Disease in Childhood,, 2002. [PubMed]
415. Social control in a school setting: evaluating a school‐based boot camp.. Crime & Delinquency,, 2001
416. Towards inclusive schools. Sustaining normal in‐school careers: an alternative to pupil exclusions.. Emotional and Behavioural Difficulties,, 2003
417. Effects of participation in a martial arts‐based antibullying program in elementary schools.. Psychology in the Schools,, 2008
418. Comparison of suspensions of EMR and non retarded students.. American Journal of Mental Deficiency,, 1983. [PubMed]
419. Interscholastic sports participation and school based delinquency: Does participation in sport foster a positive high school environment?. Sociological Spectrum,, 2012
420. The relationship between implementation of School‐Wide Positive Behavior Support (SWPBS) and disciplinary exclusion of students from various ethnic backgrounds with and without disabilities.. Journal of Emotional and Behavioral Disorders,, 2011
421. A short‐term, quasi‐experimental evaluation of D.A.R.E’s revised elementary school curriculum.. Journal of Drug Education,, 2010. [PubMed]
422. Reducing school exclusions: An evaluation of a multi‐site development project.. Oxford Review of Education,, 2003
423. Story links: Working with parents of pupils at risk of exclusion.. Support for Learning,, 2014
424. Long‐term outcomes of incredible years parenting program: Predictors of adolescent adjustment.. Child and Adolescent Mental Health,, 2011. [PubMed]
425. Aggression management training for youth in behaviour schools: A quasi‐experimental study.. Youth Studies Australia,, 2009
426. Health outcomes of youth development programme in england: Prospective matched comparison study.. British Medical Journal,, 2009
427. Evaluation of a program for reduction of childhood aggression.. Psychological Reports,, 2008. [PubMed]
428. An analysis of the psychosocial backgrounds of youths (13‐18) who‐pose‐sexual‐ risk to children.. Journal of Social Work,, 2016
429. In‐school suspension programs: The King William county model.. Educational Leadership,, 1980
430. Getting the law involved a quasi‐experiment in early intervention involving collaboration between schools and the district attorney’s office.. Criminology & Public Policy,, 2012
431. Secondary schools demonstration project: program effects of school‐based interventions on antisocial behaviour.. Exceptionality Education Canada,, 2005
432. The effects of social skills instruction on the social behaviors of students at risk for emotional or behavioral disorders.. Behavioral Disorders,, 2002
433. Social responsibility therapy for harmful, abusive behavior.. Journal of Contemporary Psychotherapy,, 2010
434. An effective approach to violence prevention: Traditional martial arts in middle school.. Adolescence,, 2001. [PubMed]
435. Ackerman, C. M. , Cooksy, L. J. , Murphy, A. , Rubright, J. , Bear, G. , & Fifield, S. (2010). Positive behavior support in Delaware schools: Developing perspectives on implementation and outcomes. (Technical Report No. T2010.3). Newark, DE. University of Delaware: Education Research & Development Center.
436. Allen, J. P. , & Philliber, S. (1991). Evaluating why and how the teen outreach program works: years 3‐5 of the teen outreach national replication (1986/87‐1988/89). New York: Association of Junior Leagues.
437. Ed White middle school restorative discipline evaluation: Implementation and impact, 2013/2014.. Sixth & seventh grade. Retrieved from, 2014
438. Baker, M. L. (2009). DPS restorative justice project: Year Three. Retrieved from Outcomes, Inc: Denver.
439. Burke, A. (2015). Suspension, expulsion, and achievement of english learner students in six Oregon districts. (REL 2015‐094). Regional Educational Laboratory Northwest. US: Department of Education.
440. Clarke, A. , Wydall, S. , & Williams, K. (2008). An evaluation of the introduction of a youth inclusion programme in rural Wales (Final Report). Wales: Department of Law and Criminology Aberystwyth University.
441. Commission of the European Communities, B. B. D.‐G. f. E. a. C . (2001). Second chance schools: The results of a European pilot project (Report). Brussels: Directorate‐General for Education & Culture.
442. Exclusion from school and alternatives.. Report on phase 3: The case study., 1996
443. Davis, A. J. , & Pfeifer, H. (2014). How has the Baltimore county public school system addressed disproportionate minority suspensions?. Schaefer Center for Public Policy University of Baltimore: College of Public Affairs.
444. Dynarski, M. , James‐Burdumy, S. , Moore, M. , Rosenberg, L. , Deke, J. , Mansfield, W. , & Warner, E. (2004). When schools stay open late: the national evaluation of the 21st century community learning centers program. New findings. Retrieved from U.S. Department of Education, National Center for Education Evaluation and Regional Assistance. Washington, DC: U.S. Government Printing Office, 2004.
445. Project success: A school‐based alternative to expulsion.. Lancaster County Schol District. Retrieved from, 1994
446. Elliot, W. L. (1991). RE‐FOCUS program. redefine efforts: Focus on change under supervision. (a support program for students and teachers). Kansas City, MO. Hickman Mills Consolidated School District. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED337830&site=ehost‐live&scope=site
447. Eric Clearinghouse on Educational Management . (1986). Preventing dropouts. The best of ERIC on educational management (No.83). Office of Educational Research and Improvement. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+experiment&pg=2&id=ED268666;http://files.eric.ed.gov/fulltext/ED268666.pdf
448. Flay, B. , Acock, A. , Vuchinich, S. , & Beets, M. (2006). Progress report of the randomized triall of Positive Action in Hawaii’s end of third year of intervention (spring 2005). Oregon: Oregon State University.
449. Foley, E. M. , Klinge, A. , & Reisner, E. R. (2007). Evaluation of new century high schools: Profile of an initiative to create and sustain small, successful high schools (final report). Policy Studies Associates, Inc. Retrieved from http://eric.ed.gov/?q=school+AND+suspension+AND+evaluation&pg=4&id=ED498781;http://files.eric.ed.gov/fulltext/ED498781.pdf
450. Fronius, T. , Persson, H. , Guckenburg, S. , Hurley, N. , & Petrosino, A. (2016). Restorative justice in us schools: A research review. Retrieved from West Ed. Justice & Prevention Research Center.: http://jprc.wested.org/wp‐content/uploads/2016/02/RJ_Literature‐Review_20160217.pdf
451. A snapshot of ost programs in philadelphia: An evaluation of eleven 21st century community learning center grantees.. Prepared by Research For Action. Retrieved from, 2014
452. Positive Alternatives to School Suspension (P.A.S.S.), 1985‐86.. Portland Public Schools: Evaluation Report. Retrieved from, 1986
453. Research and evaluation of the behaviour improvement programme. Retrieved from. 2005
454. Harlow, K. , & Baenen, N. (2002). NovaNET, 2000‐2001: Analyses of Student Outcomes Relative to a Comparison Group. Eye on Evaluation. E&R Report. Retrieved from Wake County Public School System, Raleigh, NC: http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED466488&site=ehost‐live&scope=site
455. A snapshot of OST programming in Philadelphia: An evaluation of six 21st Century Community Learning Center grantees.. Research For Action. Retrieved from, 2013
456. Heller, S. , Pollack, H. A. , Ander, R. , & Ludwig, J. (2013). Preventing youth violence and dropout: A randomized field experiment. Massachusetts: National Bureau of Economic Research.
457. Is school‐wide positive behavior support an evidence‐based practice. Retrieved from. 2007
458. Houston Independent School District . (1997). Project SAFE alternative school, 1996‐97. Research report on an educational program. Texas:Department of Research and Accountability. Retrieved from http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=5&id=ED428166;http://files.eric.ed.gov/fulltext/ED428166.pdf
459. James‐Burdumy, S. , Dynarski, M. , Moore, M. , Deke, J. , & Mansfield, W. (2005). When schools stay open late: The national evaluation of the 21st Century Community Learning Centers program. Final report. Retrieved from U.S. Department of Education
460. Kendall, S. , Kinder, K. , Halsey, K. , Fletcher‐Morgan, C. , White, R. , & Brown, C. (2003). An evaluation of alternative education initiatives (Research Report RR403). National Foundation for Educational Research. Retrieved from http://www.opengrey.eu/item/display/10068/421869
461. Laliberté, D. , Rosario, G. , Léonard, L. , Smith‐Moncrieffe, D. , & Warner, A. (2014). Results of crime prevention programs for 12 to 17 year olds (Research Report: 2015‐R006). Ottawa, Ontario Canada: National Crime Prevention Centre (NCPC) Public Safety Canada Retrieved from https://www.publicsafety.gc.ca/cnt/rsrcs/pblctns/rslts‐crm‐prvntn‐12‐17/index‐en.aspx
462. Leonard, C. M. (1984). Evaluation Report on “PASS.” Positive Alternatives to Student Suspensions. Portland Public Schools: Research & Evaluation Department. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED256772&site=ehost‐live&scope=site
463. Linker, J. , & Marion, B. (1995). A study of the effects of options, an alternative educational program, on the personal success of at‐risk students: A five year study. Alleghany County School District NC. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED398338&site=ehost‐live&scope=site
464. McAleavy, G. , & O’Hagan, C. (n/d) An evaluation of the impact of the xL intervention in schools in northern ireland. Northern Ireland: The Prince’s Trust. Retrieved from https://www.researchgate.net/profile/Celia_OHagan/publication/237343072_An_Evaluation_of_The_Impact_of_the_xl_Intervention_in_Schools_in_Northern_Ireland/links/0046352f7d3fb49433000000.pdf
465. Mearns, C. (1999). Brief family counseling intervention: Cibola cluster, 1997‐98, reserach, development and accountability. Albuquerque Public Schools. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED448374&site=ehost‐live&scope=site
466. Munn, P. , Cullen, M. , Johnstone, M. , & Lloyd, G. (1997). Exclusions and in‐school alternatives. The Scottish Office Education and Industry Department: Research and Intelligence Unit. Retrieved from http://files.eric.ed.gov/fulltext/ED414616.pdf
467. Norton, M. , & Piccinino, K. (2014). Project LIFT, year two report. Philadelphia: Research for Action. Retrieved from http://8rri53pm0cs22jk3vvqna1ub‐wpengine.netdna‐ssl.com/wp‐content/uploads/2015/10/RFA_Project_LIFT_Year_Two_Report_October_2014.pdf
468. Opuni, K. A. , Tullis, R. , Sanchez, K. , & Gonzalez, J. (1991). Student assignment centers: An in‐school suspension program, 1990‐91. Houston TX: Department of Researh and Education. Retrieved from http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=6&id=ED339137;http://files.eric.ed.gov/fulltext/ED339137.pdf
469. Peterson, P. E. , Myers, D. , & Howell, W. G. (1998). An evaluation of the new york city school choice scholarships program: The first year. New York: Mathematica Policy Research, Washington, DC. Retrieved from http://hks.harvard.edu/pepg/PDF/Papers/ny1ex.pdf
470. Pritchard, C. (2001). A family‐teacher‐social work alliance to reduce truancy and delinquency: The Dorset healthy alliance project (RDS occasional paper no. 78). England:Home Office, Communication Development Unit, Research, Development and Statistics Directorate. Retrieved from http://webarchive.nationalarchives.gov.uk/20110218135832/http:/rds.homeoffice.gov.uk/rds/pdfs/occ78‐dha.pdf
471. Puma, M. , Bell, S. , Shapiro, G. , Broene, P. , Cook, R. , Friedman, J. , & Heid, C. (2001). Building futures: the head start impact study. Research design plan. Washington DC: Administration for Children, Youth, and Families Retrieved from http://eric.ed.gov/?q=school+AND+exclusion+AND+experiment&id=ED478789;http://files.eric.ed.gov/fulltext/ED478789.pdf
472. Rhea, A. (2010). An evaluation of the Wake county public school system alternative educational options (E&R Report No. 10.15). Wake County, Public School System. Retrieved from http://www.wcpss.net/evaluation‐research
473. Program evaluation. Positive Outreach Program (POP).. Jefferson County Public Schools: Shaping the Future. Retrieved from, 2005
474. A report of the team‐teaching, team‐governance program at rockland junior high.. Distributed by ERIC Clearinghouse. Retrieved from, 1972
475. Schoeneberger, J. (2008). The effect of Positive Behavioral Intervention Support (PBIS). Charlotte‐ Mecklenburg Schools: Center for Research & Evaluation. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.563.5527&rep=rep1&type=pdf
476. Smith, L. , Jackson, L. , & Comber, N. (2013). Therapeutic early interventions to prevent school exclusion: Evaluation of three contemporaneous projects. Office for Public Management ‐ OPM. Retrieved from http://www.opm.co.uk/wp‐content/uploads/2014/01/Therapeutic‐early‐interventions‐to‐prevent‐school‐exclusion‐and‐truancy‐Final‐Report.pdf
477. St James‐Roberts, I. , & Singh, C. S. (2001). Can mentors help primary school children with behaviour problems? (Study 233). England: Home Office. Retrieved from: https://core.ac.uk/download/files/161/4157929.pdf
478. Tarling, R. , Davison, T. , & Clarke, A. (2004). Mentoring projects. The national evaluation of the Youth Justice Board’s Mentoring Projects. London: Youth Justice Board. Retrieved from http://yjbpublications.justice.gov.uk/Resources/Downloads/mentoringfull.pdf
479. U.S. Department of Education, I. o. E. S., What Works Clearinghouse . (2014). The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes. Princeton: What Works Clearinghouse Single Study Review. Retrieved from http://search.proquest.com/docview/1651831519?accountid=9851
480. Wasilewski, Y. , Gifford, B. , Bonneau, K. , Duke University, C. F. C. , & Family, P. (2008). Evaluation of the School‐Wide Positive Behavioral Support Program in eight North Carolina elementary schools. Duke University: Center for Child and Family Policy. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=ED503191&site=ehost‐live&scope=site
481. Check & Connect.. 2011
482. WWC review of the report “The short‐term effects of the Kalamazoo Promise Scholarship on student outcomes”.. What Works Clearinghouse Single Study Review. Retrieved from, 2014
483. National evaluation of the restorative justice in schools programme. Retrieved from. 2004
484. Armstrong, K. , Boroughs, M. , Massey, O. T. , Uzzell, D. , Sansosti, F. , & Perry, A. (2002). Symposium‐The safe schools/healthy students initiative: Methodologies and results in program‐based evaluation. Paper presented at the A System of Care for Children’s Mental Health: Expanding the Research Base, Tampa, Florida.
485. Bickmore, K. (2001). Good training is not enough: Research on peer mediation program implementation. Paper presented at the Association for Conflict Resolution Annual Conference, Toronto, Canada.
486. Evaluation of Massachusetts Expanded Learning Time (ELT) initiative: Implementation and outcomes after four years.. 2011
487. Effects of an evidence‐based positive youth development program on academic outcomes among adolescents living in non‐metropolitan communities.. 2015
488. Collins, K. M. , & Onwuegbuzie, A. J. (2001). Effect of an After‐School Tutorial program on academic performance of middle school students at‐risk. Paper presented at the Anual Meting of the Mid‐South Educational Research Asociation, Little Rock, AR.
489. Eber, L. , Lewis‐Palmer, T. , & Pacchiano, D. (2002). School‐Wide Positive Behavior Systems: Improving school environments for all students including those with EBD. Paper presented at the 14th Annual Conference Proceedings. A Symposium of Care for Children’s Mental Health: Expanding the Research Base, University of South Florida, Tampa.
490. Eber, L. , Rolf, K. , & Sullivan, M. P. (1998). School‐based systems of care: Early intervention and day treatment examples from Illinois. Paper presented at the 1th Anual Research Conference. A System of Care for Children’s Mental Health: Expanding the Research Base, Tampa, FL.
491. Hammond, R. W. , Kadis, P. , & Betty, Y. (1990). Positive Adolescents Choices Training (PACT): Preliminary findings of the effects of a school‐based violence prevention program for African American Adolescents. Paper presented at the 98th Annual Convention of the American Psychological Asociation, Boston, MA. http://search.proquest.com/docview/62955491?accountid=9851
492. Head, G. , Cogan, N. , & Kane, J. (2001). Evaluating behaviour support in mainstream secondary schools in one education authority. Paper presented at the British Educational Research Association Annual Conference, University of Leeds. http://www.leeds.ac.uk/educol/documents/00001916.htm
493. Heger, H. K. (1992). Retaining Hispanic youth in school: An evaluation of a counseling‐based alternative school program. Paper presented at the Anual Conference of the Rocky Mountain Educational Research Asociation, Stilwater, OK. http://eric.ed.gov/?q=school+AND+expulsion+AND+evaluation&pg=4&id=ED355302;http://files.eric.ed.gov/fulltext/ED355302.pdf
494. Howell, W. G. , & Peterson, P. E. (2000). School choice in Dayton, Ohio: An evaluation after one year. Paper presented at the Conference on Vouchers, Charters and Public Education (Harvard University, Cambridge, MA).
495. Kane, J. , Cogan, N. , & Head, G. (2002). Lessons to learn? Models of behaviour support in secondary schools in one education authority in Scotland. Paper presented at the European Conference on Educational Research, University of Lisbon. http://www.leeds.ac.uk/educol/documents/00002306.htm
496. O’Sullivan, R. G. (1988). Evaluation of a decision‐making program to improve school success with at‐risk middle school students. Paper presented at the The Anual Meting of the American Educational Research Asociation (New Orleans). http://files.eric.ed.gov/fulltext/ED296054.pdf
497. Smith, J. D. , Pare, D. , & Gravelle, F. (2002). Alternatives to school suspension: An intervention for at‐risk students. Paper presented at the Anual Meting of the American Educational Research Asociation, New Orleans, LA.
498. Winborn, J. D. (1992). A study of the effectiveness of a saturday school in reducing suspension, expulsion, and corporal punishment. Paper presented at the The Anual Meting of the Mid‐South Educational Research Asociation, Knoxvile, TN. http://search.proquest.com/docview/62851284?accountid=9851
499. Billig, S. H. (2009). Does quality really matter? Testing the new k‐12 service‐learning standards for quality practice. In B. E. B. Moely , Holland, B. A. S. H. (Ed.), Creating our identities in service‐learning and community engagement (pp. 131–157). Greenwich, CT: Information Age.
500. McIntosh, K. , Fisher, E. S. , Kennedy, K. S. , Craft, C. B. , & Morrison, G. M. (2011). Using office discipline referrals and school exclusion data to assess school discipline. In S. R. N. Jimerson , Amanda B. ; Mayer, Matthew J. ; Furlong, Michael J. (Ed.), Handbook of school violence and school safety: International research and practice, 2nd Edition (pp. 305–315). New York: Taylor & Francis.
501. Sexton, T. L. , Fisher, A. R. , Graham, C. , & Elnahrawy, N. A. (2015). Evidence‐based practice in the prevention and treatment of adolescent behavior problems. In T. P. Gullotta , Plant, Robert W. , Evans, Melanie (Ed.), Handbook of adolescent behavioral problems (pp. 85–104). United States: Springer.
502. Trotter, J. C. , & Jones, L. T. (1998). Create peace now. The peace project: An in‐school suspension program for middle and high school students with violent behaviors. Atlanta: Wholistic Stress Control Institute.
503. Vincent, C. G. , Sprague, J. R. , Pavel, M. , Tobin, T. , & Gau, J. M. (2014). Effectiveness of Schoolwide Positive Behavior Interventions and Supports in educing racially inequitable disciplinary exclusion. In D. J. Losen (Ed.), Closing the School Discipline Gap: Equitable Remedies for Excessive Exclusion (pp. 207). New York & London: Teachers College Columbia University.
504. A study to determine the effectiveness of a positive approach to discipline system for classroom management. Retrieved from. 1981
505. The implementation of a positive discipline program to increase the social skills of middle grade students. (Master Thesis).. 1996
506. Foster, E. L. (2011). Implementation of a social skills curriculum to reduce behavioral problems of african american boys in elementary classroom settings. United States of America: Xlibris Corporation.
507. Addressing the needs of at‐risk youth: An evaluation of the San Bernardino County, California School Probation Officer Program.. 2005
508. Gallegos, A. E. (1998). School expulsions, suspensions, and dropouts: Understanding the issues. Hot topics series. Bloomington: Phi Delta Kappa International.
509. The effectiveness of problem solving training on the performance of high frequency discipline students’ classroom and school. (Doctoral Dissertation).. 1983
510. Gender and race effects of a Restorative Justice intervention on school success.. 2009
511. The effects of interdisciplinary teaming and parent contacts on the academic achievement, motivation, attendance, and suspension rate of students identified to be at‐risk of repeating the ninth‐grade. (Doctoral Dissertation).. 1993
512. Acosta, J. (2015). RCT of the restorative practices intervention (RPI). Clinical trials.gov, RAND. Retrieved from https://clinicaltrials.gov/ct2/show/NCT02155296?term=RCT+of+the+Restorative+Practices+Intervention+%28RPI%29&rank=1
513. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): Study protocol for a cluster randomised controlled trial.. Trials,, 2014. [PubMed]
514. Replication of the Teen Outreach program in Kansas City. Retrieved from. 2015
515. A cluster randomized trial to evaluate external support for the implementation of Positive Behavioral Interventions and Supports by school personnel (Study protocol).. Implementation Science,, 2014. [PubMed]
516. The child behavior profile: I. Boys aged 6‐11.. Journal of Consulting and Clinical Psychology,, 1978. [PubMed]
517. The child behavior profile: II Boys aged 12‐16 and girls 6‐11 and 12‐16.. Journal of Consulting and Clinical Psychology,, 1979. [DOI | PubMed]
518. Acosta, J. (2015). Reducing problem behaviors through PYD: An RCT of restorative school practices. Clinical Trials.gov, RAND. Retrieved from https://clinicaltrials.gov/ct2/show/NCT02155296?term=RCT+of+the+Restorative+Practices+Intervention+%28RPI%29&rank=1
519. Use of the estimated intraclass correlation for correcting differences in effect size by level.. Behavior Research Methods,, 2012. [DOI | PubMed]
520. APA ZTTF . (2008). Are zero tolerance policies effective in the schools? An evidentiary review and recommendations. The American Psychologist (Vol. 63). Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/19086747
521. Achievement and enrollment status of suspended students: Outcomes in a large, multicultural school district.. Education and Urban Society,, 2006. [DOI]
522. Wirksame gewaltprävention.. Eine übersicht zum internationalen wissensstand. Retrieved from, 2014
523. Both suspension and alternatives work, depending on one’s aim.. Journal of School Violence,, 2012. [DOI]
524. Child social skills training in developmental crime prevention: effects on antisocial behavior and social competence.. Psicothema,, 2006. [PubMed]
525. Borenstein, M. , Hedges, L. , Higgins, J. P. T. , & Rothstein, H. R. (2009). Introduction to meta‐analysis. Chichester, UK: John Wiley & Sons, Ltd.
526. Patterns and predictors of disciplinary exclusion over time: An analysis of the SEELS national dataset.. Journal of Emotional and Behavioral Disorders,, 2013. [DOI]
527. Brookes, M. , Goodall, E. , & Heady, L. (2007). Misspent youth. The cost of truancy and exclusion. New Philanthropy Capital, (June). Retrieved from http://socialwelfare.bl.uk/subject‐areas/services‐activity/education‐skills/newphilanthropycapital/misspent07.aspx
528. Lost and turned out: Academic, social, and emotional experiences of students excluded from school.. Urban Education,, 2007. [DOI]
529. Evaluating peer mediation outcomes in educational settings: A meta‐analytic review.. Conflict Resolution Quarterly,, 2003. [DOI]
530. Juveniles in custody: A unique insight into the perceptions of young people held in prison service custody in England and Wales.. Her Majesty’s Inspectorate of Prisons,, 2004
531. Methodological standards for the conduct of new Cochrane Intervention Reviews. Version2.3.. 2013
532. Alternatives to suspensions: Rationale and recommendations.. Journal of School Violence,, 2012. [DOI]
533. A Coefficient for Agreement for Nominal Scales.. Educational and Psychological Measurement,, 1960. [DOI]
534. School suspension: A study with secondary school students.. Journal of School Psychology,, 1998
535. Potential for significant reductions in dropout rates: Analysis of an entire 3rd grade state cohort.. Economics of Education Review,, 2012. [DOI]
536. DfE . (2012). A profile of pupil exclusions in England. Research Report DFE‐RR190. London: Department of Education. Retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/183498/DFE‐RR190.pdf
537. DfE . (2013). Statistical first release: Permanent and fixed period exclusion from schools and exclusion appeals in England, 2011/12 (SFR 29/2013, 12) London: Department of Education. Retrieved from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/224893/SFR29‐2013.pdf
538. The impact of enhancing students’ social and emotional learning: A meta‐analysis of school‐based universal interventions.. Child Development,, 2011. [DOI | PubMed]
539. Understanding how programs work to prevent overt aggressive behaviors: A meta‐analysis of mediators of elementary school‐based programs.. Journal of School Violence,, 2011. [DOI]
540. Does Mentoring Matter? A Multidisciplinary Meta‐Analysis Comparing Mentored and Non‐Mentored Individuals.. Journal of Vocational Behavior,, 2008. [DOI | PubMed]
541. No effects in independent prevention trials: Can we reject the cynical view?. Journal of Experimental Criminology,, 2009. [DOI]
542. Eisner, M. , & Humphreys, D. K. (2012). Measuring conflict of interest in prevention and intervention research. A feasibility study. In T. Bliesener , A. Beelmann , & M. Stemmler (Eds.), Antisocial behavior and crime: Contributions of developmental and evaluation research to prevention and intervention. Göttingen, London: Hogrefe Publishing.
543. Disclosure of financial conflicts of interests in interventions to improve child psychosocial health: A cross‐sectional study.. PLoS ONE,, 2015. [PubMed]
544. Ellis, P. D. (2010). The essential guide to effect sizes. Statistical power, meta‐analysis, and the interpretation of research results. Cambridge, UK: Cambridge University Press.
545. Fabelo, T. , Thompson, M. D. , Plotkin, M. , Carmichael, D. , Marchbanks III, M. P. , & Booth, E. A. (2011). Breaking schools’ rules: A statewide study of how school discipline relates to students’ success and juvenile justice involvement. New York: Council of State Governments Justice Center.
546. Early predictors of adolescent aggression and adult violence.. Violence and Victims, (, 1989. [PubMed]
547. Farrington, D. P. (1997). Human development and criminal careers. In M. Maguire , R. Morgan , & R. Reiner (Eds.) The Oxford handbook of criminology (Second edition). Oxford University Press.
548. Farrington, D. P. (2003). Advancing knowledge about the early prevention of adult antisocial behaviour. In D. P. Farrington & J. W. Coid (Eds.), Early Prevention of Adult Antisocial Behavior (pp. 1–20). Cambridge, UK: Cambridge University Press.
549. Farrington, D. P. , Ttofi, M. M. , & Lösel, F. (2016). Developmental and social prevention. In D. Weisburd , D. P. Farrington , & C. Gill (Eds.), What works in crime prevention and rehabilitation. Lessons from systematic reviews (pp. 15–75). New York: Springer Series on Evidence‐Based Crime Policy.
550. School‐based programs to reduce bullying and victimization.. Campbell Systematic Reviews,, 2009. [DOI]
551. Call to action: A critical need for designing alternatives to suspension and expulsion.. Journal of School Violence,, 2012. [DOI]
552. Freiberg, H. J. , & Lapointe, J. M. (2006). Research‐based programs for preventing and solving discipline problems. In C. M. Evertson & C. S. Weinstein (Eds.), Handbook of classroom management research, practice, and contemporary issues (pp. 735–786). London: Routledge. 10.4324/9780203874783.ch28
553. Fronius, T. , Persson, H. , Guckenburg, S. , Hurley, N. , & Petrosino, A. (2016). Restorative justice in us schools: A research review. Retrieved from West Ed. Justice & Prevention Research Center.: http://jprc.wested.org/wp‐content/uploads/2016/02/RJ_Literature‐Review_20160217.pdf
554. Preventing bias in cluster randomised trials.. PLoS Medicine,, 2009. [DOI]
555. Characteristics of effective school‐based substance abuse prevention.. Prevention Science,, 2003. [PubMed]
556. Gottfredson, D. C. , Cook, P. J. , & Na, C. (2012). Schools and prevention. In B. C. Welsh & D. P. Farrington (Eds.), The Oxford handbook of crime prevention (pp. 269–287). Oxford, United Kingdom: Oxford University Press.
557. Gottfredson, D. , Wilson, D. B. , & Najaka, S. (2002). School‐based crime prevention. In L. W. Sherman , D. P. Farrington , B. C. Welsh , & D. MacKenzie (Eds.), Evidence‐Based Crime Prevention (pp. 56–164). London: Routledge.
558. The discipline gap and African Americans: Defiance or cooperation in the high school classroom.. Journal of School Psychology,, 2008. [DOI | PubMed]
559. The achievement gap and the discipline gap: Two sides of the same coin?. Educational Researcher,, 2010. [DOI]
560. Effect sizes in cluster‐randomized designs.. Journal of Educational and Behavioral Statistics,, 2007. [DOI]
561. The effect of school suspensions and arrests on subsequent adolescent antisocial behavior in Australia and the United States.. The Journal of Adolescent Health,, 2006. [DOI | PubMed]
562. Higgins, J. P. T. , & Green, S. (Eds). (2011). Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, V 5.1.0.
563. Measuring inconsistency in meta‐analyses.. BMJ,, 2003. [DOI | PubMed]
564. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials.. BMJ,, 2011. [DOI]
565. Preparing for prison?: The criminalization of school discipline in the USA.. Theoretical Criminology,, 2008. [DOI]
566. Is children’s free school meal “eligibility” a good proxy for family income??. British Educational Research Journal,, 2010. [DOI]
567. Hutchison, D. , & Styles, B. (2010). A guide to running randomised controlled trials for educational researchers. Slough, UK: NFER.
568. Irvin, J. (2002). In search of wholeness. African American teachers and their culturally specific classroom practices. New York: Palgrave Macmillan. 10.1057/9780230107182
569. Building better schools not prisons: A review of the literature surrounding school suspension and expulsion programs and the implications of such programs on the lives of racial and ethnic minority students. (Master Thesis).. 2012
570. Classroom goal structure and student disruptive behaviour.. The British Journal of Educational Psychology,, 2002. [PubMed]
571. Krohn, M. D. , Lopes, G. , & Ward, J. T. (2014). Effects of official intervention on later offending in the Rochester youth development study. In D. P. Farrington & J. Murray (Eds.), Labelling theory. Empirical tests. Advances in criminological theory. Volume 18 (pp. 179–207). New Brunswick: Transaction Publishers.
572. Excluding students from classroom: Teacher techniques that promote student responsibility.. Teaching and Teacher Education,, 2012. [DOI]
573. Lipsey, M. W. , & Wilson, D. B. (2001). Practical Meta‐Analysis. Thousand Oaks, California: Sage Publications.
574. Liu, J. C. (2013). The suspension spike: Changing the discipline culture in NYC’s middle schools. New York City Comptroller’s Office, 1–65.
575. Lösel, F. , & Beelmann, A. (2006). Child social skills training. In B. C. Welsh & D. P. Farrington (Eds.), Preventing crime. What works for children, offenders, victims and places. (pp. 32–54). Dordrecht, The Netherlands: Springer.
576. Best behaviour: School discipline, intervention and exclusion.. Policy Exchange,, 2011
577. Research into factors contributing to discipline use and disproportionality in major urban schools.. Current Issues in Education,, 2010
578. Understanding school responses to students’ challenging behaviour: A review of literature.. Improving Schools,, 2011. [DOI]
579. Preferred reporting items for systematic reviews and meta‐analyses: The PRISMA statement.. PLoS Med,, 2009. [DOI | PubMed]
580. Why are “bad boys” always black? Causes of disproportionality in school discipline and recommendation for changes. The Clearing House.. Classroom Management for Middle and Secondary Schools,, 2005
581. Exclusion from school: A view from Scotland of policy and practice.. Research Papers in Education,, 2001. [DOI]
582. School‐based secondary prevention programmes for preventing violence.. Cochrane Database Syst Rev, (, 2006. [DOI]
583. An exploration of discipline and suspension data.. The Journal of Negro Education,, 2004
584. Influences on authoritarian and educational/therapeutic approaches to school violence prevention.. Journal of School Violence,, 2007
585. Noltemeyer, A. L. , & Fenning, P. (2013). Systemic school discipline: Issues of equity from a social justice perspective. In D. Shriberg , S. Y. Song , A. H. Miranda , & K. M. Radliff (Eds.), School psychology and social justice: Conceptual foundations and tools for practice. (pp. 91–117). New York, NY: Routledge.
586. Changes in exclusionary discipline rates and disciplinary disproportionality over time.. International Journal of Special Education,, 2010
587. A non‐bipartite propensity score analysis of the effects of teacher ‐ student relationships on adolescent problem and prosocial behavior.. Journal of Youth & Adolescence,, 2016. [DOI]
588. OECD . (2009). Creating effective teaching and learning environments: First results from TALIS. OECD Publishing, 305.
589. OECD . (2010). PISA 2009 results: What makes a school successful? Resources, policies and practices. OECD Publishing, Vol. 4, 306.
590. OECD . (2013). PISA 2012 results: What makes schools successful? Resources, policies and practices (Volume IV). PISA, OECD Publishing. 10.1787/9789264201156-en
591. Teacher Classroom Management Practices: Effects on Disruptive or Aggressive Student Behavior.. Campbell Systematic Reviews, (, 2011. [DOI]
592. How teachers use power in the classroom to avoid or support exclusionary school discipline practices.. Urban Education,, 2013. [DOI]
593. The impact of program developers as evaluators on criminal recidivism: Results from meta‐analyses of experimental and quasi‐experimental research.. Journal of Experimental Criminology,, 2005
594. Pigott, T. (n.d.). Computing effects sizes for clustered randomized trials. The Campbell Collaboration. Retrieved from http://www.campbellcollaboration.org/artman2/uploads/1/2_Pigott_Cluster_Randomized_Trials.pdf
595. Predictors of suspension and negative school outcomes: A longitudinal investigation. New directions for youth development.. Special issue: Deconstructing the school‐to‐prison pipeline,, 2003. [DOI]
596. Unplanned outcomes: Suspensions and expulsions in Indiana.. Education Policy Briefs,, 2004
597. Empirically derived subtypes of child academic and behavior problems: Co‐occurrence and distal outcomes.. Journal of Abnormal Child Psychology,, 2008. [DOI | PubMed]
598. Cochrane: The unfinished symphony of research synthesis.. BioMed Central,, 2016. [DOI]
599. Understanding the antecedents of the “School‐to‐Jail” link: The relationship between race and school discipline.. Journal of Criminal Law and Criminology,, 2011
600. The high cost of harsh discipline and its disparate impact.. The Centre for Civil Rights Remedies, June. Retrieved from, 2016
601. Schindler, H. S. , & Yoshikawa, H. (2012). Preventing crime through intervention in the preschool years. In B. C. Welsh & D. P. Farrington (Eds.), The Oxford Handbook of Crime Prevention (pp. 71–88). Oxford, United Kingdom: Open University Press.
602. Allocation concealment in randomised trials: Defending against deciphering.. The Lancet,, 2002
603. Youth initiated mentoring: Investigating a new approach to working with vulnerable adolescents.. American Journal of Community Psychology,, 2013. [DOI | PubMed]
604. Shadish, W. , Cook, T. , & Campbell, D. (2002). Experimental and Quasi‐Experimental Designs for Generalized Causal Inference. Boston New York: Houghton Mifflin Company.
605. Evaluation of the Peacemakers Program: School‐based violence prevention for students in grades four through eight.. Psychology in the Schools,, 2002. [DOI]
606. Rationale for designing school contexts in support of proactive discipline.. Journal of School Violence,, 2012. [DOI]
607. Defiance, deterrence, and irrelevance: A theory of the criminal sanction.. Journal of Research in Crime and Delinquency,, 1993. [DOI]
608. Sherman, L. W. , Farrington, D. , Welsh, B. C. , & Mackenzie, D. L. (2002). Evidence‐based crime prevention. Revised edition. New York NY: Routledge.
609. Restorative justice: The evidence. Retrieved from. 2007
610. Clinical/narrative review. A primer on effectiveness and efficacy trials.. Clinical and Translational Gastroenterology,, 2014. [DOI]
611. Assessment of contamination and misclassification biases in a randomized controlled trial of a social network peer education intervention to reduce HIV risk behaviors among drug users and risk partners in Philadelphia, PA and Chiang Mai, Thailand.. AIDS Behaviour,, 2015. [DOI]
612. The dark side of zero tolerance: Can punishment lead to safe schools?. The Phi Delta Kappan,, 1999
613. Skiba, R. , Trachok, M. , Chung, C. , Baker, T. , & Hughes, R. (2012). Parsing disciplinary disproportionality: Contributions of behavior, student, and school characteristics to suspension and expulsion. Paper Presented at the Annual Meeting of the American Educational Research Association, Vancouver, British Columbia, Canada. 1–32.
614. The failure of zero tolerance.. Reclaiming Children and Youth,, 2014
615. Skiba, R. J. , Chung, C.‐G. , Trachok, M. , Baker, T. , Sheya, A. , & Hughes, R. (2015). Where should we intervene? In D. J. Losen (Ed.), Closing the school discipline gap: Equitable remedies for excessive exclusion (pp. 132–146). New York, NY: Teachers College, Columbia University.
616. The colour of discipline: Sources of racial and gender disproportionality in school punishment.. The Urban Review,, 2002
617. A meta‐analysis of School‐Wide Positive Behavior Support: An exploratory study using single‐case synthesis.. Psychology in the Schools,, 2012. [DOI]
618. Increasing participation: Understanding young people who do not participate in education or training at 16 and 17.. National Foundation, DCSF‐RR072,, 2009
619. Parental, Familial, and Community Support Interventions to Improve Children’s Literacy in Developing Countries?:. A Systematic Review, (July),, 2013
620. Disciplinary exclusion: The impact of intervention and influence of school ethos.. 2011
621. Sprick, R. S. , Sprick, M. , & Garrison, M. (1992). Foundations: Establishing positive discipline policies. Longmont, CO: Sopris West.
622. A Cochrane Risk Of Bias Assessment Tool: for Non‐ Randomized Studies of Interventions (ACROBAT‐NRSI), Version 1.0.0. Retrieved from. 2014
623. Exclusionary discipline of students with disabilities: Student and school characteristics predicting suspension.. Remedial and Special Education,, 2014. [DOI]
624. Sutherland, A. , Ilie, S. , & Vignoles, A. (2015). Factors associated with achievement: key stage 2. RAND Europe & University of Cambrdge.
625. Torche, F. , & Mizala, A. (2012). School discipline in Chile. In R. Arum & M. Velez (Eds.), Improving learning environments. School discipline and student achievement in comparative perspective (pp. 74–103). Stanford, California: Stanford University Press.
626. Contamination in trials: Is cluster randomisation the answer?. BMJ,, 2001. [PubMed]
627. School climate and discipline: Know the data. Retrieved from. 2012
628. Webb, R. , & Vulliamy, G. (2004). A multi‐agency approach to reducing disaffection and exclusions from school. Nottingham: DfES Publications.
629. Are mainstream programs for juvenile delinquency less effective with minority youth than majority youth? A meta‐analysis of outcomes research.. Research on Social Work Practice,, 2003. [DOI]
630. Dropout prevention and intervention programs: Effects on school completion and dropout among school‐aged children and youth.. Campbell Systematic Reviews, (, 2011. [DOI]
631. Suspension and expulsion of black students from the public schools: Academic capital punishment and the constitution.. Law and Contemporary Problems,, 1975