Hostname: page-component-546b4f848f-zwmfq Total loading time: 0 Render date: 2023-06-02T00:07:38.545Z Has data issue: false Feature Flags: { "useRatesEcommerce": true } hasContentIssue false

How do collective student behavior and other classroom contextual factors relate to teachers’ implementation of an evidence-based intervention? A multilevel structural equation model

Published online by Cambridge University Press:  23 August 2019

Rashelle J. Musci*
Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA
Elise T. Pas
Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA
Amie F. Bettencourt
Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA School of Medicine, Johns Hopkins University, Baltimore, MD, USA
Katherine E. Masyn
School of Public Health, Georgia State University, Atlanta, GA, USA
Nicholas S. Ialongo
Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA
Catherine P. Bradshaw
Bloomberg School of Public Health, Johns Hopkins University, Baltimore, MD, USA Curry School of Education and Human Development, Unviersity of Virginia, Charlottesville, VA, USA
Author for Corresponence: Rashelle J. Musci, Bloomberg School of Public Health, Johns Hopkins University, 624 N. Boadway, Baltimore, MD 21205; E-mail:


Building on prior work regarding the potential for peer contagion or deviance training in group delivered interventions (Dishion & Dodge, 2005, 2006; Dodge, Dishion, & Lansford, 2006), we leveraged data from a randomized trial, testing the integration of two preventive interventions (Promoting Alternative THinking Strategies and PAX Good Behavior Game), to explore the extent to which classroom contextual factors served as either a barrier to or a motivator for teachers to implement the evidence-based PAX Good Behavior Game with high frequency or dosage. We included students’ baseline levels of behavior, measured with regard to both positive (i.e., engagement and social emotional skills) and negative (i.e., hyperactive and aggressive-disruptive) behaviors. Data were collected from 204 teachers in 18 urban elementary schools. A series of multilevel structural equation models were fit to the data. The analyses indicated that classrooms with higher classroom levels of aggressive behavior, on average, at baseline had teachers with lower implementation dosage (i.e., played fewer games) across the school year. In addition, teachers who reported higher baseline levels of emotional exhaustion, regardless of student behavior, also reported lower implementation dosage. Taken together, the results indicated that negative, but not positive, contextual factors at baseline were related to lower implementation dosage; this, in turn, suggests that negative contextual factors may serve as a barrier, rather than a motivator, of teachers’ implementation dosage of classroom-based preventive interventions.

Special Issue Articles
Copyright © Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


Barrish, H., Saunders, M., & Wolf, M. (1969). Good Behavior Game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavior Analysis, 2, 119124.CrossRefGoogle Scholar
Becker, K. D., Bradshaw, C. P., Domitrovich, C., & Ialongo, N. S. (2013). Coaching teachers to impove implementation of the good behavior game. Administration and Policy in Mental Health and Mental Health Services Research, 40, 482493.CrossRefGoogle Scholar
Becker, K. D., Darney, D., Domitrovich, C., Keperling, J. P., & Ialongo, N. S. (2013). Supporting universal prevention programs: A two-phased coaching model. Clinical Child and Family Psychology Review, 16, 213228.CrossRefGoogle ScholarPubMed
Bradshaw, C. P., Koth, C. W., Thornton, L. A., & Leaf, P. J. (2009). Altering school climate through school-wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10, 100115. doi:10.1007/s11121-008-0114-9CrossRefGoogle ScholarPubMed
Bradshaw, C. P., & Kush, J. (in press). TOCA-C: An efficient approach to measuring children's social, emotional, and behavioral functioning by teachers. Children & Schools.Google Scholar
Bradshaw, C. P., Mitchell, M. M., O'Brennan, L. M., & Leaf, P. J. (2010). Multilevel exploration of factors contributing to the overrepresentation of black students in office disciplinary referrals. Journal of Educational Psychology, 102, 508.CrossRefGoogle Scholar
Chiapa, A., Parra Morris, G., Véronneau, M. H., & Dishion, T. J. (2016). Translational research on parenting of adolescents: Linking theory to valid observation measures for family centered prevention and treatment. Translational Behavioral Medicine, 6, 90104.CrossRefGoogle ScholarPubMed
Conduct Problems Prevention Research Group. (1999). Initial impact of the Fast Track prevention trial for conduct problems: II. Classroom effects. Journal of Consulting and Clinical Psychology, 67, 648657.CrossRefGoogle Scholar
Dishion, T. J., & Dodge, K. A. (2005). Peer contagion in interventions for children and adolescents: Moving towards an understanding of the ecology and dynamics of change. Journal of Abnormal Child Psychology, 33, 395400.CrossRefGoogle Scholar
Dishion, T. J., & Dodge, K. A. (2006). Deviant peer contagion in interventions and programs: An ecological framework for understanding influence mechanisms. In Dodge, K. A., Dishion, T. J., & Lansford, J. E. (Eds.), Deviant peer influences in programs for youth: Problems and solutions (pp. 1443). New York: Guilford Press.Google Scholar
Dishion, T. J., Nelson, S. E., & Kavanagh, K. (2003). The Family Check-Up with high-risk young adolescents: Preventing early-onset substance use by parent monitoring. Behavior Therapy, 34, 553571. doi:10.1016/S0005-7894(03)80035-7CrossRefGoogle Scholar
Dishion, T. J., & Tipsord, J. M. (2011). Peer contagion in child and adolescent social and emotional development. Annual Review of Psychology, 62, 189214. doi:10.1146/annurev.psych.093008.100412CrossRefGoogle ScholarPubMed
Dodge, K. A., Dishion, T. J., & Lansford, J. E. (2006). Deviant peer influences in programs for youth: Problems and solutions. New York: Guilford Press.Google Scholar
Domitrovich, C., Bradshaw, C. P., Berg, J., Pas, E., Becker, K., Musci, R., … Ialongo, N. (2016). How do school-based prevention programs impact teachers? Findings from a randomized trial of an integrated classroom management and social-emotional program. Prevention Science, 17, 325337.CrossRefGoogle ScholarPubMed
Domitrovich, C. E., Bradshaw, C. P., Greenberg, M. T., Embry, D., Poduska, J. M., & Ialongo, N. S. (2010). Integrated models of school-based prevention: Logic and theory. Psychology in the Schools, 47, 7188. doi:10.1002/pits.20452Google ScholarPubMed
Domitrovich, C. E., Bradshaw, C. P., Poduska, J. M., Hoagwood, K. E., Buckley, J. A., Olin, S., … Ialongo, N. S. (2008). Maximizing the implementation quality of evidence-based preventive interventions in schools: A conceptual framework. Advances in School Mental Health Promotion, 1, 628. doi:10.1080/1754730X.2008.9715730CrossRefGoogle ScholarPubMed
Domitrovich, C. E., Gest, S. D., Gill, S., Jones, D. J., & DeRouise, R. S. (2009). Teacher factors related to the professional development process of the Head Start REDI intervention. Early Education and Development, 20, 402430.CrossRefGoogle Scholar
Domitrovich, C. E., Pas, E. T., Bradshaw, C. P., Becker, K., Keperling, J., Embry, D., & Ialongo, N. (2015). Individual and organizational school factors that influence implementation of the PAX Good Behavior Game intervention. Prevention Science, 16, 10641074. doi:10.1007/s11121-015-0557-8CrossRefGoogle ScholarPubMed
Dunn, E. C., Masyn, K. E., & Johnston, W. R. (2015). Modeling contextual effects using indlvidual-level data and without aggregation: An illustration of multilevel factor analysis (MLFA) with collective efficacy. Population Health Metrics, 13, 12.CrossRefGoogle ScholarPubMed
Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327350. doi:10.1007/s10464-008-9165-0CrossRefGoogle ScholarPubMed
Embry, D., Staatemeier, G., Richardson, C., Lauger, K., & Mitich, J. (2003). The PAX Good Behavior Game (1st ed.). Center City, MN: Hazelden.Google Scholar
Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009). Core implementation components. Research in Social Work Practice, 19, 531540.CrossRefGoogle Scholar
Greenberg, M. T., Kusché, C. A., & Conduct Problems Prevention Research Group. (2011). Grade level PATHS (Grades 3–5). South Deerfield, MA: Channing-Bete.Google Scholar
Han, S. S., & Weiss, B. (2005). Sustainability of teacher implementation of school-based mental health programs. Journal of Abnormal Child Psychology, 33, 665679. doi:10.1007/s10802-005-7646-2CrossRefGoogle ScholarPubMed
Hoy, W. K., & Feldman, J. (1987). Organizational health: The concept and its measure. Journal of Research and Development in Education, 20, 3038.Google Scholar
Hoy, W. K., & Tarter, C. J. (1997). The road to open and healthy schools: A handbook for change. Thousand Oaks, CA: Corwin Pess.Google Scholar
Huang, F. L., & Cornell, D. G. (2016). Using multilevel factor analysis with clustered data: Investigating the factor structure of the Positive Values Scale. Journal of Psychoeducational Assessment, 34, 314.CrossRefGoogle Scholar
Ialongo, N., Domitrovich, C., Embry, D., Greenberg, M., Lawson, A., Becker, K., & Bradshaw, C. (2019). A randomized controlled trial of the combination of two school-based universal preventive interventions. Developmental Psychology. Advance online publication. doi:10.1037/dev0000715CrossRefGoogle ScholarPubMed
Ialongo, N., Werthamer, L., Kellam, S., Brown, C., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and anti-social behavior. American Journal of Community Psychology, 27, 599641.CrossRefGoogle Scholar
Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 5563.CrossRefGoogle ScholarPubMed
Kim, E. S., Dedrick, R. F., Cao, C., & Ferron, J. M. (2016). Multilevel factor analysis: Reporting guidelines and a review of reporting practices. Multivariate Behavioral Research, 51, 881898.Google Scholar
Koth, C. W., Bradshaw, C. P., & Leaf, P. J. (2009). Teacher Observation of Classroom Adaptation—Checklist: Development and factor structure. Measurement and Evaluation in Counseling and Development, 42, 1530.CrossRefGoogle Scholar
Kusché, C. A., Greenberg, M. T., & Conduct Problems Prevention Research Group. (2011). Grade level PATHS (Grades 1–2). South Deerfield, MA: Channing-Bete.Google Scholar
Latimore, A. D., Burrell, L., Crowne, S., Ojo, K., Cluxton-Keller, F., Gustin, S., … Duggan, A. (2017). Exploring multilevel factors for family engagement in home visiting across two national models. Prevention Science, 18, 577589. doi:10.1007/s11121-017-0767-3CrossRefGoogle ScholarPubMed
Lochman, J. E., Dishion, T. J., Boxmeyer, C. L., Powell, N., & Qu, L. (2017). Variation in response to evidence-based group preventive intervention for disruptive behavior problems: A view from 938 Coping Power sessions. Journal of Abnormal Child Psychology, 45, 12711284. doi:10.1007/s10802-016-0252-7CrossRefGoogle ScholarPubMed
Lochman, J. E., Dishion, T. J., Powell, N., Boxmeyer, C. L., Qu, L., & Salle, M. (2015). Evidence-based preventive intervention for preadolescent aggressive children: One-year outcomes following randomization to group versus individual delivery. Journal of Consulting and Clinical Psychology, 83, 728735. doi:10.1037/ccp0000030CrossRefGoogle ScholarPubMed
Main, S., & Hammond, L. (2008). Best pactice or most practiced? Pre-service teachers' beliefs about effective behaviour management strategies and reported self-efficacy. American Journal of Teacher Education, 33, n4.Google Scholar
Malloy, M., Acock, A., DuBois, D. L., Vuchinich, S., Silverthorn, N., Ji, P., & Flay, B. R. (2015). Teachers' perceptions of school organizational climate as predictors of dosage and quality of implementation of a social-emotional and character development progam. Prevention Science, 31, 10861095.CrossRefGoogle Scholar
Maslach, C., Jackson, S. E., & Leiter, M. P. (1996). Maslach Burnout Inventory manual (3rd ed.). Palo Alto, CA: Consulting Psychologists Press.Google Scholar
Muthén, B., & Muthén, L. (1997–2019). Mplus user's guide. Los Angeles: Author.Google Scholar
Nilsen, P. (2015). Making sense of implementation science. Prevention Science, 10, 214225.Google Scholar
Pas, E., & Bradshaw, C. P. (2014). What affects teacher ratings of student behaviors? The potential influence of teachers’ perceptions of the school environment and experiences. Prevention Science, 15, 940950. doi:10.1007/s11121-013-0432-4CrossRefGoogle ScholarPubMed
Pas, E. T., Bradshaw, C. P., Becker, K., Domitrovich, C., Berg, J., Musci, R., & Ialongo, N. (2015). Trajectories for coaching dosage as a means for improving implementation of the Good Behavior Game. School Mental Health, 7, 6173. doi:10.1007/s12310-015-9145-0CrossRefGoogle Scholar
Pas, E. T., Waasdorp, T. E., & Bradshaw, C. P. (2015). Examining contextual influences on classroom-based implementation of positive behavior support strategies: Findings from a randomized controlled effectiveness trial. Prevention Science, 16, 10961106. doi:10.1007/s11121-014-0492-0CrossRefGoogle ScholarPubMed
Patterson, G. R., Reid, J., & Dishion, T. (1992). A social learning approach: IV. Antisocial boys. Eugene, OR: Castalia.Google Scholar
Payne, A. A., Gottfredson, D. C., & Gottfredson, G. D. (2006). School predictors of the intensity of implementation of school-based prevention programs: Results from a national study. Prevention Science, 7, 225237. doi:10.1007/s11121-006-0029-2CrossRefGoogle ScholarPubMed
Petras, H., Chilcoat, H. D., Leaf, P. J., Ialongo, N. S., & Kellam, S. G. (2004). Utility of TOCA-R scores during the elementary school years in identifying later violence among adolescent males. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 8896.CrossRefGoogle ScholarPubMed
Ringwalt, C. L., Ennett, S., Johnson, R., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., & Thorne, J. (2003). Factors associated with fidelity to substance use prevention curriculum guides in the nation's middle schools. Health Education & Behavior, 30, 375391. doi:10.1177/1090198103030003010CrossRefGoogle ScholarPubMed
Rohrbach, L. A., Graham, J. W., & Hansen, W. B. (1993). Diffusion of a school-based substance abuse prevention program: Predictors of program implementation. Preventive Medicine, 22, 237260.CrossRefGoogle ScholarPubMed
Smith, J. D., Berkel, C., Rudo-Stern, J., Montaño, Z., St George, S. M., Prado, G., … Dishion, T. J. (2018). The Family Check-Up 4 Health (FCU4Health): Applying implementation science frameworks to the process of adapting an evidence-based parenting program for prevention of pediatric obesity and excess weight gain in primary care. Frontiers in Public Health, 6, 293. doi:10.3389/fpubh.2018.00293CrossRefGoogle ScholarPubMed
Sutherland, K. S., Conroy, M. A., McLeod, B. D., Algina, J., & Kunemund, R. L. (2018). Factors associated with teacher delivery of a classroom-based Tier 2 prevention program. Prevention Science, 19, 186196. doi:10.1007/s11121-017-0832-yCrossRefGoogle ScholarPubMed
Sutherland, K. S., Conroy, M. A., McLeod, B. D., Algina, J., & Wu, E. (2018). Teacher competence of delivery of BEST in CLASS as a mediator of treatment effects. School Mental Health, 10, 214225. doi:10.1007/s12310-017-9224-5CrossRefGoogle Scholar
Wanless, S. B., Rimm-Kaufman, S. E., Abry, T., Larsen, R. A., & Patton, C. L. (2015). Engagement in training as a mechanism to understanding fidelity of implementation of the responsive classroom approach. Prevention Science, 16, 11071116. doi:10.1007/s11121-014-0519-6CrossRefGoogle ScholarPubMed
Wehby, J. H., Maggin, D. M., Moore Partin, T. C., & Robertson, R. (2012). The impact of working alliance, social validity, and teacher burnout on implementation fidelity of the Good Behavior Game. School Mental Health, 4, 2233. doi:10.1007/s12310-011-9067-4CrossRefGoogle Scholar
Werthamer-Larsson, L., Kellam, S., & Wheeler, L. (1991). Effect of first grade classroom environment on shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology, 19, 585602.CrossRefGoogle ScholarPubMed