Skip to main content Accessibility help
×
Hostname: page-component-8448b6f56d-gtxcr Total loading time: 0 Render date: 2024-04-19T11:57:37.470Z Has data issue: false hasContentIssue false

28 - Social Signal Processing and Socially Assistive Robotics in Developmental Disorders

from Part IV - Applications of Social Signal Processing

Published online by Cambridge University Press:  13 July 2017

Mohamed Chetouani
Affiliation:
University Pierre et Marie Curie
Sofiane Boucenna
Affiliation:
University Pierre and Marie Curie
Laurence Chaby
Affiliation:
University Pierre and Marie Curie, University Paris Descartes
Monique Plaza
Affiliation:
University Pierre and Marie Curie, University Paris Descartes
David Cohen
Affiliation:
University Pierre et Marie Curie
Judee K. Burgoon
Affiliation:
University of Arizona
Nadia Magnenat-Thalmann
Affiliation:
Université de Genève
Maja Pantic
Affiliation:
Imperial College London
Alessandro Vinciarelli
Affiliation:
University of Glasgow
Get access

Summary

Introduction

Multimodal social-emotional interactions play a critical role in child development and this role is emphasized in autism spectrum disorders (ASD). In typically developing children, the ability to correctly identify, interpret, and produce social behaviors (Figure 28.1) is a key aspect for communication and is the basis of social cognition (Carpendale & Lewis, 2004). This process helps children to understand that other people have intentions, thoughts, and emotions and act as a trigger of empathy (Decety & Jackson, 2004; Narzisi et al., 2012). Social cognition includes the child's ability to spontaneously and correctly interpret verbal and nonverbal social and emotional cues (e.g., speech, facial and vocal expressions, posture and body movements, etc.); the ability to produce social and emotional information (e.g., initiating social contact or conversation); the ability to continuously adjust and synchronize behavior to others (i.e., parent, caregivers, peers); and the ability to make an adequate attribution about another's mental state (i.e., “theory of mind”).

Definitions and Treatments

ASDs are a group of behaviorally defined disorders with abnormalities or impaired development in two areas: (1) persistent deficits in social communication and social interaction and (2) restricted, repetitive patterns of behavior, interests, or activities. An individual with ASD has difficulty interacting with other people due to an inability to understand social cues as well as others’ behaviors and feelings. For example, children with ASD often have difficulty with cooperative play with other peers; they prefer to continue with their own repetitive activities (Baron-Cohen & Wheelwright, 1999). Persons with ASD evaluate both world and human behavior uniquely because they react in an abnormal way to input stimuli while there is problematic human engagement and inability to generalize the environment (Rajendran & Mitchell, 2000). Although ASD remains a devastating disorder with a poor outcome in adult life, there have been important improvements in treating ASD with the development of various therapeutic approaches (Cohen, 2012).

Successful autism “treatments” using educational interventions have been reported as recently as a decade ago (Murray, 1997). Since then, the literature devoted to the description and evaluation of interventions in ASD has become substantial over the last few years. From this literature, a number of conclusions can be drawn. First, there is increasing convergence between behavioral and developmental methods (Ospina et al., 2008).

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2017

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Altmann, U. (2011). Studying movement synchrony using time series and regression models. In I. A, Esposito, R, Hoffmann, S, Hübler, & B, Wrann (Eds), Program and Abstracts of the COST 2012 Final Conference Held in Conjunction with the 4th COST 2012 International Training School on Cognitive Behavioural Systems (p. 23).
Ashenfelter, K. T., Boker, S. M., Waddell, J. R., & Vitanov, N. (2009). Spatiotemporal symmetry and multifractal structure of head movements during dyadic conversation.Journal of Experimental Psychology: Human Perception and Performance, 35(4), 1072–1091.Google Scholar
Baron-Cohen, S. & Wheelwright, S. (1999). “Obsessions” in children with autism or Asperger syndrome: Content analysis in terms of core domains of cognition.The British Journal of Psychiatry, 175(5), 484–490.Google Scholar
Bernieri, F. J., Reznick, J. S., & Rosenthal, R. (1988). Synchrony, pseudo synchrony, and dissynchrony: Measuring the entrainment process in mother–infant interactions.Journal of Personality and Social Psychology, 54(2), 243–253.Google Scholar
Bird, G., Leighton, J., Press, C., & Heyes, C. (2007). Intact automatic imitation of human and robot actions in autism spectrum disorders.Proceedings of the Royal Society B: Biological Sciences, 274(1628), 3027–3031.Google Scholar
Black, M. P., Bone, D., Williams, M. E., et al. (2011). The USC CARE Corpus: Child– psychologist interactions of children with autism spectrum disorders. In: Proceedings of Inter- Speech (pp. 1497–1500).
Boker, S. M., Xu, M., Rotondo, J. L., & King, K. (2002). Windowed cross-correlation and peak picking for the analysis of variability in the association between behavioral time series.Psychological Methods, 7(3), 338–355.Google Scholar
Boucenna, S., Anzalone, S., Tilmont, E., Cohen, D., & Chetouani, M. (2014). Learning of social signatures through imitation game between a robot and a human partner.IEEE Transactions on Autonomous Mental Development, 6(3), 213–225.Google Scholar
Campbell, N. (2008). Multimodal processing of discourse information: The effect of synchrony. In Proceedings of 2008 Second International Symposium on Universal Communication (pp. 12–15).
Carpendale, J. I. M. & Lewis, C. (2004). Constructing an understanding of the mind: The development of children's social understanding within social interaction.Behavioral and Brain Sciences, 27, 79–151.Google Scholar
Carpenter, M., Nagell, K., Tomasello, M., Butterworth, G., & Moore, C. (1998). Social cognition, joint attention, and communicative competence from 9 to 15 months of age.Monographs of the Society for Research in Child Development, 63(4), 1–143.Google Scholar
Chaby, L., Chetouani, M., Plaza, M., & Cohen, D. (2012). Exploring multimodal socialemotional behaviors in autism spectrum disorders. In Workshop on Wide Spectrum Social Signal Processing, 2012 ASE/IEEE International Conference on Social Computing (pp. 950– 954).
Chamak, B., Bonniau, B., Jaunay, E., & Cohen, D. (2008). What can we learn about autism from autistic persons? Psychotherapy and Psychosomatics, 77, 271–279.Google Scholar
Chaminade, T., Da Fonseca, D., Rosset, D., et al. (2012). FMRI study of young adults with autism interacting with a humanoid robot. In Proceedings of the 21st IEEE International Symposium on Robot and Human Interactive Communication (pp. 380–385).
Cohen, D. (2012). Traumatismes et traces: donnés expérimentales.Neuropsychiatrie de l’Enfance et de l’Adolescence, 60, 315–323.Google Scholar
Cohen, D., Cassel, R. S., Saint-Georges, C., et al. (2013). Do parentese prosody and fathers’ involvement in interacting facilitate social interaction in infants who later develop autism? PLoS ONE, 8(5), e61402.Google Scholar
Costa, S., Santos, C., Soares, F., Ferreira, M., & Moreira, F. (2010). Promoting interaction amongst autistic adolescents using robots. In Proceedings of 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology (pp. 3856–3859).
Dautenhahn, K. (2003). Roles and functions of robots in human society: Implications from research in autism therapy.Robotica, 21(4), 443–452.Google Scholar
Dautenhahn, K. & Werry, I. (2004). Towards interactive robots in autism therapy: Background, motivation and challenges.Pragmatics & Cognition, 12(1), 1–35.Google Scholar
Dawson, G., Rogers, S., Munson, J., et al. (2009). Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver model.Pediatrics, 125(1), 17–23.Google Scholar
Decety, J. & Jackson, P. (2004). The functional architecture of human empathy.Behavioral and Cognitive Neuroscience Reviews, 3(2), 71–100.Google Scholar
Delaherche, E. & Chetouani, M. (2010). Multimodal coordination: Exploring relevant features and measures. In Proceedings of the 2nd International Workshop on Social Signal Processing (pp. 47–52).
Delaherche, E., Chetouani, M., Bigouret, F., et al. (2013). Assessment of the communicative and coordination skills of children with autism spectrum disorders and typically developing children using social signal processing.Research in Autism Spectrum Disorders, 7(6), 741– 756.Google Scholar
Delaherche, E., Chetouani, M., Mahdhaoui, M., et al. (2012). Interpersonal synchrony: A survey of evaluation methods across disciplines.IEEE Transactions on Affective Computing, 3(3), 349–365.Google Scholar
Demouy, J., Plaza, M., Xavier, J., et al. (2011). Differential language markers of pathology in autism, pervasive developmental disorder not otherwise specified and specific language impairment.Research in Autism Spectrum Disorders, 5(4), 1402–1412.Google Scholar
Diehl, J., Schmitt, L. M., Villano, M., & Crowell, C. R. (2012). The clinical use of robots for individuals with autism spectrum disorders: A critical review.Research in Autism Spectrum Disorders, 6(1), 249–262.Google Scholar
Duquette, A., Michaud, F., & Mercier, H. (2008). Exploring the use of a mobile robot as an imitation agent with children with low-functioning autism.Autonomous Robots, 24(2), 147– 157.Google Scholar
Feil-Seifer, D. & Mataric, M. J. (2009). Toward socially assistive robotics for augmenting interventions for children with autism spectrum disorders. In O, Khatib, V, Kumar, & G, Pappas (Eds), Experimental Robotics (vol. 54, pp. 201–210). Berlin: Springer.
Feil-Seifer, D. & Mataric, M. J. (2011). Automated detection and classification of positive vs. negative robot interactions with children with autism using distance-based features. In 6th ACM/IEEE International Conference on Human–Robot Interaction (pp. 323–330).
François, D., Powell, S., & Dautenhahn, K. (2009). A long-term study of children with autism playing with a robotic pet: Taking inspirations from non-directive play therapy to encourage children's proactivity and initiative-taking.Interaction Studies, 10(3), 324–373.Google Scholar
Fujimoto, I., Matsumoto, T., De Silva, P. R. S., Kobayashi, M., & Higashi, M. (2011). Mimicking and evaluating human motion to improve the imitation skill of children with autism through a robot.International Journal of Social Robotics, 3(4), 349–357.Google Scholar
Howlin, P., Baron-Cohen, S., & Hadwin, J. (1999). Teaching Children with Autism to Mind-Read: A Practical Guide for Teachers and Parents. New York: John Wiley & Sons.
Jones, S. (2007). Imitation in infancy the development of mimicry.Psychological Science, 18(7), 593–599.Google Scholar
Jones, S. (2009). The development of imitation in infancy.Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1528), 2325.Google Scholar
Kaliouby, R., Picard, R., & Barron-Cohen, S. (2006). Affective computing and autism.Annals of the New York Academy of Sciences, 1093, 228–248.Google Scholar
Kendon, A. (1970).Movement coordination in social interaction: Some examples described.Acta Psychologica, 32, 100–125.Google Scholar
Kjelgaard, M. & Tager-Flusberg, H. (2001). An investigation of language impairment in autism: Implications for genetic subgroups.Language and Cognitive Processes, 16(2–3), 287–308.Google Scholar
Konstantinidis, E. I., Luneski, A., Frantzidis, C. A., Pappas, C., & Bamidis, P. D. (2009). A proposed framework of an interactive semi-virtual environment for enhanced education of children with autism spectrum disorders. In Proceedings of the 22nd IEEE International Symposium on Computer-Based Medical Systems(pp. 1–6).Google Scholar
Kozima, H., Nakagawa, C., & Yasuda, Y. (2007). Children-robot interaction: A pilot study in autism therapy.Progress in Brain, 164, 385–400.Google Scholar
Liu, C., Conn, K., Sarkar, N., & Stone, W. (2008). Physiology-based affect recognition for computer-assisted intervention of children with autism spectrum disorder.International Journal of Human-Computer Studies, 66(9), 662–677.Google Scholar
Mahdhaoui, A., Chetouani, M., Cassel, R. S., et al. (2011). Computerized home video detection for motherese may help to study impaired interaction between infants who become autistic and their parents.International Journal of Methods in Psychiatric Research, 20(1), e6–e18.Google Scholar
Michelet, S., Karp, K., Delaherche, E., Achard, C., & Chetouani, M. (2012). Automatic imitation assessment in interaction.Lecture Notes in Computer Science, 7559, 161–173.Google Scholar
Mitchell, P., Parsons, S., & Leonard, A. (2006). Using virtual environments for teaching social understanding to 6 adolescents with autistic spectrum disorders.Journal of Autism and Developmental Disorders, 3(37), 589–600.Google Scholar
Murray, D. (1997). Autism and information technology: Therapy with computers. In S, Powell & R, Jordan (Eds), Autism and Learning: A Guide to Good Practice (pp. 100–117). London: David Fulton.
Nadel, J., Simon, M., Canet, P., et al. (2006). Human responses to an expressive robot. In Proceedings of the Sixth International Workshop on Epigenetic Robotics (pp. 79–86).
Narzisi, A., Muratori, F., Calderoni, S., Fabbro, F., & Urgesi, C. (2012). Neuropsychological Profile in High Functioning Autism Spectrum Disorders.Journal of Autism and Developmental Disorders, 43(8), 1895–1909.Google Scholar
Ospina, M. B., Seida, J. K., Clark, B., et al. (2008). Behavioural and developmental interventions for autism spectrum disorder: a clinical systematic review.PLoS ONE, 3(11): e3755.Google Scholar
Pantic, M., Pentland, A., Nijholt, A., & Huang, T. (2006). Human computing and machine understanding of human behavior: A survey. In Proceedings of the 8th International Conference on Multimodal Interfaces (pp. 239–248).
Papert, S. (1980). Mindstorms: Children, Computers, and Powerful Ideas. New York: Basic Books.
Pentland, A., Lazer, D., Brewer, D., & Heibeck, T. (2009). Using reality mining to improve public health and medicine.Studies in Health Technology and Informatics, 149, 93–102.Google Scholar
Picard, R. (2010). Emotion research by the people, for the people.Emotion Review, 2(3), 250–254.Google Scholar
Pierno, A., Mari, M., Lusher, D., & Castiello, U. (2008). Robotic movement elicits visuomotor priming in children with autism.Neuropsychologia, 46(2), 448–454.Google Scholar
Pioggia, G., Igliozzi, R., Ferro, M., et al. (2005). An android for enhancing social skills and emotion recognition in people with autism.IEEE Transactions on Neural Systems and Rehabilitation Engineering, 13(4), 507–515.Google Scholar
Pioggia, G., Igliozzi, R., Sica, M. L., et al. (2008). Exploring emotional and imitational androidbased interactions in autistic spectrum disorders.Journal of CyberTherapy & Rehabilitation,1(1), 49–61.Google Scholar
Rajendran, G. & Mitchell, P. (2000). Computer mediated interaction in Asperger's syndrome: The Bubble Dialogue program.Computers and Education, 35, 187–207.Google Scholar
Ramseyer, F. & Tschacher, W. (2011). Nonverbal synchrony in psychotherapy: Coordinated body movement reflects relationship quality and Outcome.Journal of Consulting and Clinical Psychology, 79(3), 284–295.Google Scholar
Ravindra, P., De Silva, S., Tadano, K., et al. (2009). Therapeutic-assisted robot for children with autism. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 3561–3567).
Rehg, J. M., Abowd, G. D., Rozga, A., et al. (2013). Decoding children's social behavior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 3414– 3421).
Ringeval, F., Demouy, J., Szaszák, G., et al. (2011). Automatic intonation recognition for the prosodic assessment of language impaired children.IEEE Transactions on Audio, Speech and Language Processing, 19(5), 1328–1342.Google Scholar
Robins, B., Dautenhahn, K., & Dubowski, J. (2006). Does appearance matter in the interaction of children with autism with a humanoid robot? Interaction Studies, 7(3), 509–542.Google Scholar
Robins, B., Dautenhahn, K., Te Boekhorst, R., & Billard, A. (2005). Robotic assistants in therapy and education of children with autism: Can a small humanoid robot help encourage social interaction skills? Universal Access in the Information Society, 4(2), 105–120.Google Scholar
Saint-Georges, C., Chetouani, M., Cassel, R., et al. (2013).Motherese in interaction: At the crossroad of emotion and cognition? (A systematic review).PLoS ONE, 8(10), e78103.Google Scholar
Saint-Georges, C., Mahdhaoui, A., Chetouani, M., et al. (2011). Do parents recognize autistic deviant behavior long before diagnosis? Taking into account interaction using computational methods.PLoS ONE, 6(7), e22393.Google Scholar
Scassellati, B. (2007). How social robots will help us to diagnose, treat, and understand autism. In S, Thrun, R, Brooks, & H, Durrant-Whyte (Eds), Robotics Research (pp. 552–563). London: Springer.
Scassellati, B., Admoni, H., & Mataric, M. (2012). Robots for use in autism research.Annual Review of Biomedical Engineering, 14, 275–294.Google Scholar
Schuller, B., Steidl, E., Batliner, A., et al. (2013). The InterSpeech 2013 computational paralinguistics challenge: Social signals, conflict, Emotion, autism. In Proceedings of the Annual Conference of the International Speech Communication Association (pp. 148–152).
Segalin, C., Pesarin, A., Vinciarelli, A., Tait, M., & Cristani, M. (2013). The expressivity of turntaking: Understanding children pragmatics by hybrid classifiers. In Proceedings of the 14th International Workshop on Image Analysis for Multimedia Interactive Services (pp. 1–4).
Simon, M., Canet, P., Soussignan, R., Gaussier, P., & Nadel, J. (2007). L’enfant face à des expressions robotiques et humaines.Enfance, 59(1), 59–70.Google Scholar
Stribling, P., Rae, J., & Dickerson, P. (2009). Using conversation analysis to explore the recurrence of a topic in the talk of a boy with an autism spectrum disorder.Clinical Linguistics & Phonetics, 23(8), 555–582.Google Scholar
Sun, X., Truong, K., Nijholt, A., & Pantic, M. (2011). Automatic visual mimicry expression analysis in interpersonal interaction. In Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition: Workshop on CVPR for Human Behaviour Analysis (pp. 40–46).
Tager-Flusberg, H. (1981). On the nature of linguistic functioning in early infantile autism.Journal of Autism and Developmental Disorders, 11, 45–56.Google Scholar
Tapus, A., Mataric, M., & Scassellati, B. (2007). Socially assistive robotics.IEEE Robotics and Automation Magazine, 14(1), 35–42.Google Scholar
Tomasello, M. & Farrar, M. (1986). Joint attention and early language.Child Development, 57(6), 1454–1463.Google Scholar
Vannetzel, L., Chaby, L., Cautru, F., Cohen, D., & Plaza, M. (2011). Neutral versus emotional human stimuli processing in children with pervasive developmental disorders not otherwise specified.Research in Autism Spectrum Disorders, 5(2), 775–783.Google Scholar
Varni, G., Volpe, G., & Camurri, A. (2010). A system for real-time multimodal analysis of nonverbal affective social interaction in user-centric media.IEEE Transactions on Multimedia, 12(6), 576–590.Google Scholar
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009). Social signal processing: Survey of an emerging domain.Image and Vision Computing, 27(12), 1743–1759.Google Scholar
Wainer, J., Ferrari, E., Dautenhahn, K., & Robins, B. (2010). The effectiveness of using a robotics class to foster collaboration among groups of children with autism in an exploratory study.Personal and Ubiquitous Computing, 14(5), 445–455.Google Scholar
Weisman, O., Delaherche, E., Rondeau, M., et al. (2013). Oxytocin shapes parental motion during father–infant interaction.Biology Letters, 9(6).Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×