Skip to main content Accessibility help
×
Home

Using automated computer vision and machine learning to code facial expressions of affect and arousal: Implications for emotion dysregulation research

  • Nathaniel Haines (a1), Ziv Bell (a1), Sheila Crowell (a2) (a3), Hunter Hahn (a1), Dana Kamara (a1), Heather McDonough-Caplan (a1), Tiffany Shader (a1) and Theodore P. Beauchaine (a1)...

Abstract

As early as infancy, caregivers’ facial expressions shape children's behaviors, help them regulate their emotions, and encourage or dissuade their interpersonal agency. In childhood and adolescence, proficiencies in producing and decoding facial expressions promote social competence, whereas deficiencies characterize several forms of psychopathology. To date, however, studying facial expressions has been hampered by the labor-intensive, time-consuming nature of human coding. We describe a partial solution: automated facial expression coding (AFEC), which combines computer vision and machine learning to code facial expressions in real time. Although AFEC cannot capture the full complexity of human emotion, it codes positive affect, negative affect, and arousal—core Research Domain Criteria constructs—as accurately as humans, and it characterizes emotion dysregulation with greater specificity than other objective measures such as autonomic responding. We provide an example in which we use AFEC to evaluate emotion dynamics in mother–daughter dyads engaged in conflict. Among other findings, AFEC (a) shows convergent validity with a validated human coding scheme, (b) distinguishes among risk groups, and (c) detects developmental increases in positive dyadic affect correspondence as teen daughters age. Although more research is needed to realize the full potential of AFEC, findings demonstrate its current utility in research on emotion dysregulation.

Copyright

Corresponding author

Author for Correspondence: Theodore P. Beauchaine, Department of Psychology, Ohio State University, 1835 Neil Avenue, Columbus, OH 43210; Email: beauchaine.1@osu.edu.

References

Hide All
Adams, A., & Robinson, P. (2015). Automated recognition of complex categorical emotions from facial expressions and head motions. Paper presented at the 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), Xi'an, China.
Baltrusaitis, T., Robinson, P., & Morency, L.-P. (2016). OpenFace: An open source facial behavior analysis toolkit. Paper presented at the 2016 IEEE Winter Conference on Applications of Computer Vision, Lake Placid, NY.
Barrett, L. F. (2009). The future of psychology: Connecting mind to brain. Perspectives on Psychological Science, 4, 326339. doi:10.1111/j.1745-6924.2009.01134.x
Barsade, S. G. (2002). The ripple effect: Emotional contagion and its influence on group behavior. Administrative Science Quarterly, 47, 644675. doi:10.2307/3094912
Bartlett, M. S., Hager, J. C., Ekman, P., & Sejnowski, T. J. (1999). Measuring facial expressions by computer image analysis. Psychophysiology, 36, 253263. doi:10.1017/S0048577299971664
Bartlett, M. S., Littlewort, G. C., Frank, M. G., & Lee, K. (2014). Automatic decoding of facial movements reveals deceptive pain expressions. Current Biology, 24, 738743. doi:10.1016/j.cub.2014.02.009
Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 148. doi:10.18637/jss.v067.i01
Beauchaine, T. P. (2015). Future directions in emotion dysregulation and youth psychopathology. Journal of Clinical Child and Adolescent Psychology, 44, 875896. doi:10.1080/15374416.2015.1038827
Beauchaine, T. P., & Haines, N. (in press). Functionalist and constructionist perspectives on emotion. In Beauchaine, T. P. & Crowell, S. E. (Eds.), The Oxford handbook of emotion dysregulation. New York: Oxford University Press.
Beauchaine, T. P., & Webb, S. J. (2017). Developmental processes and psychophysiology. In Cacioppo, J. T., Tassinary, L. G., & Berntson, G. G. (Eds.), Handbook of psychophysiology (4th ed., pp. 495510). New York: Cambridge University Press.
Beauchaine, T. P., & Zalewski, M. (2016). Physiological and developmental mechanisms of emotional lability in coercive relationships. In Dishion, T. J. & Snyder, J. J. (Eds.), The Oxford handbook of coercive relationship dynamics (pp. 3952). New York: OXford University Press.
Beckes, L., & Edwards, W. L. (2018). Emotions as regulators of social behavior. In Beauchaine, T. P. & Crowell, S. E. (Eds.), The Oxford handbook of emotion dysregulation. New York: Oxford University Press.
Blair, R. (1995). A cognitive developmental approach to morality: Investigating the psychopath. Cognition, 57, 129. doi:10.1016/0010-0277(95)00676-P
Blair, R. J. R., & Coles, M. (2000). Expression recognition and behavioural problems in early adolescence. Cognitive Development, 15, 421434. doi:10.1016/S0885-2014(01)00039-9
Bodner, N., Kuppens, P., Allen, N. B., Sheeber, L. B., & Ceulemans, E. (2018). Affective family interactions and their associations with adolescent depression: A dynamic network approach. Development and Psychopathology, 30, 14591473. doi:10.1017/S0954579417001699
Bonanno, G. A., & Burton, C. L. (2013). Regulatory flexibility: An individual differences perspective on coping and emotion regulation. Perspectives on Psychological Science, 8, 591612. doi:10.1177/1745691613504116
Bonanno, G. A., Colak, D. M., Keltner, D., Shiota, M. N., Papa, A., Noll, J., … Trickett, P. K. (2007). Context matters: The benefits and costs of expressing positive emotion among survivors of childhood sexual abuse. Emotion, 7, 824837. doi:10.1037/1528-3542.7.4.824
Bonanno, G. A., Papa, A., Lalande, K., Westphal, M., & Coifman, K. (2004). The importance of being flexible: The ability to both enhance and suppress emotional expression predicts long-term adjustment. Psychological Science, 15, 482487. doi:10.1111/j.0956-7976.2004.00705.x
Buck, R. (1975). Nonverbal communication of affect in children. Journal of Personality and Social Psychology, 31, 644653. doi:10.1037/h0077071
Buck, R. (1977). Nonverbal communication of affect in preschool children: Relationships with personality and skin conductance. Journal of Personality and Social Psychology, 35, 225236. doi:10.1037/0022-3514.35.4.225
Campos, J. J., Mumme, D. L., Kermoian, R., & Campos, R. G. (1994). A functionalist perspective on the nature of emotion. Monographs of the Society for Research in Child Development, 59, 284303. doi:10.1111=j.1540-5834.1994.tb01289.x
Camras, L. A., Grow, J. G., & Ribordy, S. C. (1983). Recognition of emotional expression by abused children. Journal of Clinical Child Psychology, 12, 325328. doi:10.1207/s15374424jccp1203_16
Camras, L. A., Ribordy, S., Hill, J., Martino, S., Spaccarelli, S., & Stefani, R. (1988). Recognition and posing of emotional expressions by abused children and their mothers. Developmental Psychology, 24, 776781. doi:10.1037//0012-1649.24.6.776
Carp, J. (2012). On the plurality of (methodological) worlds: Estimating the analytic flexibility of fMRI experiments. Frontiers in Neuroscience, 6, 149. doi:10.3389/fnins.2012.00149
Chen, S., Chen, T., & Bonanno, G. A. (2018). Expressive flexibility: Enhancement and suppression abilities differentially predict life satisfaction and psychopathology symptoms. Personality and Individual Differences, 126, 7884. doi:10.1016/j.paid.2018.01.010
Chevalier-Skolnikoff, S. (1973). Facial expression of emotion in nonhuman primates. In Ekman, P. (Ed.), Darwin and facial expression (pp. 1183). New York: Oxford University Press.
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155159. doi:10.1037/0033-2909.112.1.155
Cohn, J. F. (2010). Advances in behavioral science using automated facial image analysis and synthesis [social sciences]. IEEE Signal Processing Magazine, 27, 128133. doi:10.1109/MSP.2010.938102
Cohn, J. F., & De la Torre, F. (2014). Automated face analysis for affective. In Calvo, R. A., D'Mello, S., Gratch, J., & Kappas, A. (Eds.), The Oxford handbook of affective computing (pp. 131150). New York: Oxford University Press.
Cohn, J. F., & Ekman, P. (2005). Measuring facial action. In Harrigan, J. A., Rosenthal, R., & Scherer, K. R. (Eds.), The new handbook of nonverbal behavior research methods in the affective sciences (pp. 964). New York: Oxford University Press.
Coifman, K., & Almahmoud, S. (2017). Emotion flexibility and psychological risk and resilience. In Kumar, U. (Ed.), The Routledge international handbook of psychosocial resilience (pp. 114). London: Routledge.
Coifman, K. G., Flynn, J. J., & Pinto, L. A. (2016). When context matters: Negative emotions predict psychological health and adjustment. Motivation and Emotion, 40, 602624. doi:10.1007/s11031-016-9553-y
Crowell, S. E., Baucom, B. R., McCauley, E., Potapova, N. V., Fitelson, M., Barth, H., … Beauchaine, T. P. (2013). Mechanisms of contextual risk for adolescent self-injury: Invalidation and conflict escalation in mother–child interactions. Journal of Clinical Child and Adolescent Psychology, 42, 467480. doi:10.1080/15374416.2013.785360
Crowell, S. E., Baucom, B. R., Yaptangco, M., Bride, D., Hsiao, R., McCauley, E., & Beauchaine, T. P. (2014). Emotion dysregulation and dyadic conflict in depressed and typical adolescents: Evaluating concordance across psychophysiological and observational measures. Biological Psychology, 98, 5058. doi:10.1016/j.biopsycho.2014.02.009
Crowell, S. E., Beauchaine, T. P., Hsiao, R. C., Vasilev, C. A., Yaptangco, M., Linehan, M. M., & McCauley, E. (2012). Differentiating adolescent self-injury from adolescent depression: Possible implications for borderline personality development. Journal of Abnormal Child Psychology, 40, 4557. doi:10.1007/s10802-011-9578-3
Crowell, S. E., Butner, J., Wiltshire, T. J., Munion, A. K., Yaptangco, M., & Beauchaine, T. P. (2017). Evaluating emotional and biological sensitivity to maternal behavior among depressed and self-injuring adolescent girls using nonlinear dynamics. Clinical Psychological Science, 5, 272285. doi:10.1177/2167702617692861
Darwin, C. (1872). The expression of the emotions in man and animals. London: Murray.
Dibeklioğlu, H., Hammal, Z., Yang, Y., & Cohn, J. F. (2015). Multimodal detection of depression in clinical interviews. Paper presented at the 2015 ACM on International Conference on Multimodal Interaction (pp. 307–310), Seattle, WA.
Di Pierro, R., Sarno, I., Perego, S., Gallucci, M., & Madeddu, F. (2012). Adolescent nonsuicidal self-injury: The effects of personality traits, family relationships and maltreatment on the presence and severity of behaviours. European Child and Adolescent Psychiatry, 21, 511520. doi:10.1007/s00787-012-0289-2
Domitrovich, C. E., Cortes, R. C., & Greenberg, M. T. (2007). Improving young children's social and emotional competence: A randomized trial of the preschool “PATHS” curriculum. Journal of Primary Prevention, 28, 6791. doi:10.1007/s10935-007-0081-0
Du, S., Tao, Y., & Martinez, A. M. (2014). Compound facial expressions of emotion. Proceedings of the National Academy of Sciences, 111, E1454–E1462. doi:10.1073/pnas.1322355111
Duchenne, G. (1990). The mechanism of human facial expression. Translated by Cuthbertson, R. A.. New York: Cambridge University Press. (Originally published as Mecanisme de la Physionomie Humaine in 1862.)
Dys, S. P., & Malti, T. (2016). It's a two-way street: Automatic and controlled processes in children's emotional responses to moral transgressions. Journal of Experimental Child Psychology, 152, 3140. doi:10.1016/j.jecp.2016.06.011
Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion , 6 , 169200. doi:10.1080/02699939208411068
Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48, 384392. doi:10.1037/0003-066X.48.4.384
Ekman, P., & Cordaro, D. (2011). What is meant by calling emotions basic. Emotion Review , 3 , 364370. doi:10.1177/1754073911410740
Ekman, P., & Friesen, W. (1978). Facial action coding system: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychologists Press.
Ekman, P., Friesen, W. V., O'Sullivan, M., Chan, A., Diacoyanni-Tarlatzis, I., Heider, K., … Ricci-Bitti, P. E. (1987). Universals and cultural differences in the judgments of facial expressions of emotion. Journal of Personality and Social Psychology, 53, 712717. doi:10.1037/0022-3514.53.4.712
Ekman, P., & Oster, H. (1979). Facial expressions of emotion. Annual Review of Psychology, 30, 527554. doi:0.1146/annurev.ps.30.020179.002523
Ekman, P., Roper, G., & Hager, J. C. (1980). Deliberate facial movement. Child Development, 51, 886891. doi:10.2307/1129478
Ekman, P., & Rosenberg, E. L. (Eds.). (2005). What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system (2nd ed.). New York: Oxford University Press.
Fairchild, G., Van Goozen, S. H. M., Calder, A. J., Stollery, S. J., & Goodyer, I. M. (2009). Deficits in facial expression recognition in male adolescents with early-onset or adolescence-onset conduct disorder. Journal of Child Psychology and Psychiatry, 50, 627636. doi:10.1111/j.1469-7610.2008.02020.x
Fasel, B., & Luettin, J. (2003). Automatic facial expression analysis: A survey. Pattern Recognition, 36, 259275. doi:10.1016/S0031-3203(02)00052-3
Field, T. M., & Walden, T. A. (1982). Production and discrimination of facial expressions by preschool children. Child Development, 53, 12991311. doi:10.2307/1129020
Field, T. M., Woodson, R., Greenberg, R., & Cohen, D. (1982). Discrimination and imitation of facial expression by neonates. Science, 218, 179181. doi:10.1126/science.7123230
Gadea, M., Aliño, M., Espert, R., & Salvador, A. (2015). Deceit and facial expression in children: The enabling role of the “poker face” child and the dependent personality of the detector. Frontiers in Psychology, 6, 6. doi:10.3389/fpsyg.2015.01089
Gallup, A. C., O'Brien, D. T., & Wilson, D. S. (2010). The relationship between adolescent peer aggression and responses to a sequential Prisoner's Dilemma game during college: An explorative study. Journal of Social, Evolutionary, and Cultural Psychology, 4, 277289. doi:10.1037/h0099283
Girard, J. M., & Cohn, J. F. (2015). Automated audiovisual depression analysis. Current Opinion in Psychology, 4, 7579. doi:10.1016/j.copsyc.2014.12.010
Golan, O., Ashwin, E., Granader, Y., McClintock, S., Day, K., Leggett, V., & Baron-Cohen, S. (2009). Enhancing emotion recognition in children with autism spectrum conditions: An intervention using animated vehicles with real emotional faces. Journal of Autism and Developmental Disorders, 40, 269279. doi:10.1007/s10803-009-0862-9
Golan, O., Baron-Cohen, S., & Hill, J. (2006). The Cambridge Mindreading (CAM) face-voice battery: Testing complex emotion recognition in adults with and without Asperger syndrome. Journal of Autism and Developmental Disorders, 36, 169183. doi:10.1007/s10803-005-0057-y
Goodenough, F. L. (1932). Expression of the emotions in a blind-deaf child. Journal of Abnormal and Social Psychology, 27, 328333. doi:10.1037/h0076099
Gordon, I., Pierce, M. D., Bartlett, M. S., & Tanaka, J. W. (2014). Training facial expression production in children on the autism spectrum. Journal of Autism and Developmental Disorders, 44, 24862498. doi:10.1007/s10803-014-2118-6
Gross, J. J., & Levenson, R. W. (1993). Emotional suppression: Physiology, self-report, and expressive behavior. Journal of Personality and Social Psychology, 64, 970986. doi:10.1037/0022-3514.64.6.970
Gunes, H., & Pantic, M. (2010). Automatic, dimensional and continuous emotion recognition. International Journal of Synthetic Emotions, 1. doi:10.4018/jse.2010101605
Gunes, H., Schuller, B., Pantic, M., & Cowie, R. (2011). Emotion representation, analysis and synthesis in continuous space: A survey. Proceedings of the Automatic Face and Gesture Recognition Workshop, 2011 IEEE International Conference (pp. 827–834), Santa Barbara, CA.
Haines, N., Southward, M. W., Cheavens, J. S., Beauchaine, T. P., & Ahn, W.-Y. (2019). Using computer-vision and machine learning to automate facial coding of positive and negative affect intensity. PLOS ONE, 14, e0211735. doi:10.1371/journal.pone.0211735
Hamm, J., Kohler, C. G., Gur, R. C., & Verma, R. (2011). Automated facial action coding system for dynamic analysis of facial expressions in neuropsychiatric disorders. Journal of Neuroscience Methods, 200, 237256. doi:10.1016/j.jneumeth.2011.06.023
Hammal, Z., Cohn, J. F., Heike, C., & Speltz, M. L. (2015). Automatic measurement of head and facial movement for analysis and detection of infants’ positive and negative affect. Frontiers in ICT, 2, 21. doi:10.3389/fict.2015.00021
Hipwell, A., Keenan, K., Kasza, K., Loeber, R., Stouthamer-Loeber, M., & Bean, T. (2008). Reciprocal influences between girls’ conduct problems and depression, and parental punishment and warmth: A six-year prospective analysis. Journal of Abnormal Child Psychology, 36, 663677. doi:10.1007/s10802-007-9206-4
Hoffner, C., & Badzinski, D. M. (1989). Children's integration of facial and situational cues to emotion. Child Development, 60, 411422. doi:10.2307/1130986
iMotions. (2018). Facial expression analysis: The definitive guide. Retrieved from https://imotions.com/facialexpression-guide-ebook/ on 3/17/2018
Isaacowitz, D. M., Löckenhoff, C. E., Lane, R. D., Wright, R., Sechrest, L., Riedel, R., & Costa, P. T. (2007). Age differences in recognition of emotion in lexical stimuli and facial expressions. Psychology and Aging, 22, 147159. doi:10.1037/0882-7974.22.1.147
Izard, C. E., Fine, S., Schultz, D., Mostow, A., Ackerman, B., & Youngstrom, E. (2001). Emotion knowledge as a predictor of social behavior and academic competence in children at risk. Psychological Science, 12, 1823. doi:10.1111/1467-9280.00304
Izard, C. E., Huebner, R. R., Risser, D., McGinnes, G., & Dougherty, L. (1980). The young infant's ability to produce discrete emotion expressions. Developmental Psychology, 16, 132140. doi:10.1037/0012-1649.16.2.132
Izard, C. E., & Malatesta, C. Z. (1987). Perspectives on emotional development: I. Differential emotions theory of early emotional development. In Osofsky, J. D. (Ed.), Wiley series on personality processes: Handbook of infant development (pp. 494554). Oxford: Wiley.
Jack, R. E., Garrod, O. G. B., Yu, H., Caldara, R., & Schyns, P. G. (2012). Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences, 109, 72417244. doi:10.1073/pnas.1200155109
Kanluan, I., Grimm, M., & Kroschel, K. (2008). Audio-visual emotion recognition using an emotion space concept. Paper presented at the 16th European Signal Processing Conference, Lausanne, Switzerland.
Keltner, D., Moffitt, T. E., & Stouthamer-Loeber, M. (1995). Facial expressions of emotion and psychopathology in adolescent boys. Journal of Abnormal Psychology, 104, 644652. doi:10.1037/0021-843X.104.4.644
Korb, S., With, S., Niedenthal, P., Kaiser, S., & Grandjean, D. (2014). The perception and mimicry of facial movements predict judgments of smile authenticity. PLOS ONE, 9, e99194. doi:10.1371/journal.pone.0099194
Kotsia, I., & Pitas, I. (2007). Facial expression recognition in image sequences using geometric deformation features and support vector machines. IEEE Transactions on Image Processing, 16, 172187. doi:10.1109/TIP.2006.884954
Krumhuber, E. G., Skora, L., Küster, D., & Fou, L. (2016). A review of dynamic datasets for facial expression research. Emotion Review, 9, 280292. doi:10.1177/1754073916670022
Lewinski, P., den Uyl, T. M., & Butler, C. (2014). Automated facial coding: Validation of basic emotions and FACS AUs in FaceReader. Journal of Neuroscience, Psychology, and Economics, 7, 227236. doi:10.1037/npe0000028
Li, J. J., & Lansford, J. E. (2018). A smartphone-based ecological momentary assessment of parental behavioral consistency: Associations with parental stress and child ADHD symptoms. Developmental Psychology, 54, 10861098. doi:10.1037/dev0000516
Littlewort, G., Whitehill, J., Wu, T., & Fasel, I. (2011). The computer expression recognition toolbox (CERT). Paper presented at the IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG 2011), Santa Barbara, CA.
Lopes, A. T., de Aguiar, E., De Souza, A. F., & Oliveira-Santos, T. (2017). Facial expression recognition with convolutional neural networks: Coping with few data and the training sample order. Pattern Recognition, 61, 610628. doi:10.1016/j.patcog.2016.07.026
Luterek, J. A., Orsillo, S. M., & Marx, B. P. (2005). An experimental examination of emotional experience, expression, and disclosure in women reporting a history of childhood sexual abuse. Journal of Traumatic Stress, 18, 237244. doi:10.1002/jts.20027
Marsh, P., Beauchaine, T. P., & Williams, B. (2008). Dissociation of sad facial expressions and autonomic nervous system responding in boys with disruptive behavior disorders. Psychophysiology, 45, 100110. doi:10.1111/j.1469-8986.2007.00603.x
Mattson, W. I., Cohn, J. F., Mahoor, M. H., Gangi, D. N., & Messinger, D. S. (2013). Darwin's Duchenne: Eye constriction during infant joy and distress. PLOS ONE, 8, e80161. doi:10.1371/journal.pone.0080161
McDuff, D., El Kaliouby, R., Kassam, K., & Picard, R. (2010). Affect valence inference from facial action unit spectrograms. Paper presented at the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, CA.
Meltzoff, A. N., & Moore, M. K. (1977). Imitation of facial and manual gestures by human neonates. Science, 198, 7578. doi:10.1126/science.198.4312.75
Messinger, D. S., Mahoor, M., Chow, S.-M., & Cohn, J. (2009). Automated measurement of facial expression in infant-mother interaction: A pilot study. Infancy, 14, 285305. doi:10.1080/15250000902839963
Messinger, D. S., Mattson, W. I., Mahoor, M. H., & Cohn, J. F. (2012). The eyes have it: Making positive expressions more positive and negative expressions more negative. Emotion, 12, 430436. doi:10.1037/a0026498
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2017). AffectNet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, 1. doi:10.1109/TAFFC.2017.2740923
Muehlenkamp, J. J., Kerr, P. L., Bradley, A. R., & Adams Larsen, M. (2010). Abuse subtypes and nonsuicidal self-injury: Preliminary evidence of complex emotion regulation patterns. Journal of Nervous and Mental Disease, 198, 258263. doi:10.1097/NMD.0b013e3181d612ab
National Advisory Mental Health Council Workgroup on Tasks and Measures for Research Domain Criteria. (2016). Behavioral assessment methods for RDoC constructs. Bethesda, MD: National Institute of Mental Health.
Nelson, C. A. (1987). The recognition of facial expressions in the first two years of life: Mechanisms of development. Child Development, 58, 889909. doi:10.2307/1130530
Nicolaou, M. A., Gunes, H., & Pantic, M. (2011). Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Transactions on Affective Computing, 2, 92105. doi:10.1109/T-AFFC.2011.9
Noldus. (2018). FaceReader: Tool for automatic analysis of facial expression: Version 6.0 [Software]. Wageningen, Netherlands: Noldus Information Technology.
Odom, R. D., & Lemond, C. M. (1972). Developmental differences in the perception and production of facial expressions. Child Development, 43, 359369. doi:10.2307/1127541
Parrott, W. G. (2000). Emotions in social psychology: Key readings. Philadelphia, PA: Psychology Press.
Pessoa, L. (2008). On the relationship between emotion and cognition. Nature Reviews Neuroscience, 9, 148158. doi:10.1038/nrn2317
Pollak, S. D., Cicchetti, D., Hornung, K., & Reed, A. (2000). Recognizing emotion in faces: Developmental effects of child abuse and neglect. Developmental Psychology, 36, 679688. doi:10.1037/0012-1649.36.5.679
Pollak, S. D., & Sinha, P. (2002). Effects of early experience on children's recognition of facial displays of emotion. Developmental Psychology, 38, 784791. doi:10.1037/0012-1649.38.5.784
Porter, S., & ten Brinke, L. (2010). The truth about lies: What works in detecting high-stakes deception? Legal and Criminological Psychology, 15, 5775. doi:10.1348/135532509X433151
Preuschoft, S., & van Hooff, J. A. (1995). Homologizing primate facial displays: A critical review of methods. Folia Primatologica, 65, 121137. doi:10.1159/000156878
Prinz, R. J., Foster, S., Kent, R. N., & O'Leary, K. D. (1979). Multivariate assessment of conflict in distressed and nondistressed mother-adolescent dyads. Journal of Applied Behavior Analysis, 12, 691700. doi:10.1901/jaba.1979.12-691
Ramsook, K. A., Cole, P. M., & Fields-Olivieri, M. A. (2019). What emotion dysregulation looks like: Inferences from behavioral observations. In Beauchaine, T. P. & Crowell, S. E. (Eds.), The Oxford handbook of emotion dysregulation. New York: Oxford University Press.
Reed, L. I., DeScioli, P., & Pinker, S. A. (2014). The commitment function of angry facial expressions. Psychological Science, 25, 15111517. doi:10.1177/0956797614531027
Ringeval, F., Schuller, B., Valstar, M., Jaiswal, S., Marchi, E., Lalanne, D., … Pantic, M. (2015). AV+EC 2015: The first affect recognition challenge bridging across audio, video, and physiological data. Paper presented at the 5th International Workshop on Audio/Visual Emotion Challenge, Brisbane, Australia.
Robin, M., Pham-Scottez, A., Curt, F., Bigre, C. D.-L., Speranza, M., Sapinho, D., … Kedia, B. S. (2012). Decreased sensitivity to facial emotions in adolescents with borderline personality disorder. Psychiatry Research, 200, 417421. doi:10.1016/j.psychres.2012.03.032
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 11611178. doi:10.1037/h0077714
Russell, J. A., & Bullock, M. (1985). Multidimensional scaling of emotional facial expressions: Similarity from preschoolers to adults. Journal of Personality and Social Psychology, 48, 12901298. doi:10.1037//0022-3514.48.5.1290
Schlosberg, H. (1941). A scale for the judgment of facial expressions. Journal of Experimental Psychology, 29, 497510. doi:10.1037/h0061489
Schlosberg, H. (1952). The description of facial expressions in terms of two dimensions. Journal of Experimental Psychology, 44, 229237. doi:10.1037/h0055778
Schlosberg, H. (1954). Three dimensions of emotion. Psychological Review, 61, 8188. doi:10.1037/h0054570
Sikka, K., Ahmed, A. A., Diaz, D., Goodwin, M. S., Craig, K. D., Bartlett, M. S., & Huang, J. S. (2015). Automated assessment of children's postoperative pain using computer vision. Pediatrics, 136, e124e131. doi:10.1542/peds.2015-0029
Smith, C. A., & Ellsworth, P. C. (1985). Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychology, 48, 813838. doi:10.1037/0022-3514.48.4.813
Smith, E. R., & Mackie, D. M. (2015). Dynamics of group-based emotions: Insights from intergroup emotions theory. Emotion Review, 7, 349354. doi:10.1177/1754073915590614
Smith, M., & Walden, T. (1998). Developmental trends in emotion understanding among a diverse sample of African-American preschool children. Journal of Applied Developmental Psychology, 19, 177197. doi:10.1016/S0193-3973(99)80035-5
Smith, S. (2003). Digital signal processing: A practical guide for engineers and scientists. Burlington, MA: Elsevier Science.
Snyder, J., Schrepferman, L., & St Peter, C. (1997). Origins of antisocial behavior: Negative reinforcement and affect dysregulation of behavior as socialization mechanisms in family interaction. Behavior Modification, 21, 187215. doi:10.1177/01454455970212004
Song, R., Over, H., & Carpenter, M. (2016). Young children discriminate genuine from fake smiles and expect people displaying genuine smiles to be more prosocial. Evolution and Human Behavior, 37, 490501. doi:10.1016/j.evolhumbehav.2016.05.002
Sorce, J. F., Emde, R. N., Campos, J., & Klinnert, M. (1985). Maternal emotional signaling: Its effect on the visual cliff behavior of 1-year-olds. Developmental Psychology, 21, 195200. doi:10.1037/0012-1649.21.1.195
Stöckli, S., Schulte-Mecklenbeck, M., Borer, S., & Samson, A. C. (2017). Facial expression analysis with AFFDEX and FACET: A validation study. Behavior Research Methods, 26, 115. doi:10.3758/s13428-017-0996-1
Stubbs, J., Crosby, L., Forgatch, M. S., & Capaldi, D. M. (1998). Family and peer process code: A synthesis of three Oregon Social Learning Center behavior codes [Training manual]. Eugene, OR: Oregon Social Learning Center.
Thomas, L. A., De Bellis, M. D., Graham, R., & LaBar, K. S. (2007). Development of emotional facial recognition in late childhood and adolescence. Developmental Science, 10, 547558. doi:10.1111/j.1467-7687.2007.00614.x
Thompson, J. (1941). Development of facial expression of emotion in blind and seeing children. Archives of Psychology (Columbia University), 264, 47.
Thompson, R. A. (1990). Emotion and self-regulation. In Thompson, R. A. (Ed.), Nebraska Symposium on Motivation: Vol. 36. Socioemotional development (pp. 383483). Lincoln, NE: University of Nebraska Press.
Tian, Y. I., Kanade, T., & Cohn, J. F. (2001). Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23, 97115. doi:10.1109/34.908962
Troster, H., & Brambring, M. (1992). Early social-emotional development in blind infants. Child: Care, Health and Development, 18, 207227. doi:10.1111/j.1365-2214.1992.tb00355.x
Valstar, M., Gratch, J., Schuller, B., Ringeval, F., Lalanne, D., Torres, M. T., … Pantic, M. (2016). AVEC 2016: Depression, mood, and emotion recognition workshop and challenge. Paper presented at the Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge, Amsterdam, Netherlands.
Watson, D., & Tellegen, A. (1985). Toward a consensual structure of mood. Psychological Bulletin, 98, 219235. doi:10.1037/0033-2909.98.2.219
Westphal, M., Seivert, N. H., & Bonanno, G. A. (2010). Expressive flexibility. Emotion, 10, 92100. doi:10.1037/a0018420
Widen, S. C., & Russell, J. (2003). A closer look at preschoolers' freely produced labels for facial expressions. Developmental Psychology, 39, 114128. doi:10.1037/0012-1649.39.1.114
Yarczower, M., Kilbride, J. E., & Hill, L. (1979). Imitation and inhibition of facial expression. Developmental Psychology, 15, 453454. doi:10.1037/0012-1649.15.4.453
Yoshimura, S., Sato, W., Uono, S., & Toichi, M. (2014). Impaired overt facial mimicry in response to dynamic facial expressions in high-functioning autism spectrum disorders. Journal of Autism and Developmental Disorders, 45, 13181328. doi:10.1007/s10803-014-2291-7
Zhao, X., & Zhang, S. (2016). A review on facial expression recognition: Feature extraction and classification. IETE Technical Review, 33, 505517. doi:10.1080/02564602.2015.1117403
Zisner, A., & Beauchaine, T. P. (2016). Psychophysiological methods and developmental psychopathology. In Cicchetti, D. (Ed.), Developmental psychopathology: Vol. 2. Developmental neuroscience (3rd ed., pp. 832884). Hoboken, NJ: Wiley.
Zivin, G. (1977). On becoming subtle: Age and social rank changes in the use of a facial gesture. Child Development, 48, 13141321. doi:10.2307/1128489

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed