Skip to main content Accessibility help
×
Hostname: page-component-848d4c4894-xm8r8 Total loading time: 0 Render date: 2024-06-16T12:25:09.684Z Has data issue: false hasContentIssue false

11 - Natural language generation for augmentative and assistive technologies

from Part IV - Engagement

Published online by Cambridge University Press:  05 July 2014

Nava Tintarev
Affiliation:
University of Aberdeen
Ehud Reiter
Affiliation:
University of Aberdeen
Rolf Black
Affiliation:
Dundee University
Annalu Waller
Affiliation:
University of Dundee
Amanda Stent
Affiliation:
AT&T Research, Florham Park, New Jersey
Srinivas Bangalore
Affiliation:
AT&T Research, Florham Park, New Jersey
Get access

Summary

Introduction

Many people are not able to communicate easily, because of diversity in their physical and/or intellectual abilities. For example, someone with cerebral palsy may not have enough control over their vocal tract and mouth to speak (or enough manual dexterity control to use sign language); and someone with autism may be physically capable, but unable to speak because of their cognitive profile.

Augmentative and alternative communication (AAC) is a sub-field of assistive technology focusing on tools to help people with major motor and/or cognitive impairments communicate better. Traditionally, AAC has used technology to provide speech output for non-speaking users, either by converting text to speech or speaking out pre-stored phrases. The initial focus of computer-based AAC systems was on giving users access to a broad set of symbols/pictures (by using some kind of menu or search system), and on using speech synthesis to automatically speak out phrases corresponding to the selected symbols. However, computers can in principle do much more than this.

In particular, since many AAC systems are intended to help users produce utterances, there seems to be a natural role for Natural Language Generation (NLG) to play, as it is after all a technology for producing language. However, the use of NLG in AAC is somewhat different from most uses of NLG. Since the goal of AAC is to help the user communicate, the NLG system must be used interactively, under the user’s control; we want to assist the user in communication, not replace the user with an automatic communicator. Also, since most human communication is social, NLG AAC systems often need to generate texts whose communicative goal is social interaction. Hence, NLG AAC systems are very different from systems that summarize information in a task-oriented context, which has been the focus of most NLG research for interactive systems.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2014

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

American Speech-Language-Hearing Association (2002). Augmentative and alternative communication: Knowledge and skills for service delivery [knowledge and skills]. Available from http://www.asha.org/policy. Accessed on 11/24/2013.
Andreasen, P., Waller, A., and Gregor, P. (1998). BlissWord – full access to Blissymbols for all users. In Proceedings of the Biennial Conference of the International Society for Augmentative and Alternative Communication, pages 167-168, Dublin, Ireland. International Society for Augmentative and Alternative Communication.Google Scholar
Beukelman, D. R. and Mirenda, P. (1998). Augmentative and Alternative Communication: Management of Severe Communication Disorders in Children and Adults. Brookes Publishing Company, Baltimore, MD, 2nd edition.Google Scholar
Biswas, P. (2006). A flexible approach to natural language generation for disabled children. In Proceedings of the International Conference on Computational Linguistics and the Annual Meeting of the Association for Computational Linguistics (COLING-ACL), pages 1-6, Sydney, Australia. Association for Computational Linguistics.Google Scholar
Biswas, P. and Samanta, D. (2008). Friend: A communication aid for persons with disabilities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 16(2):205-209.CrossRefGoogle ScholarPubMed
Black, R., Reddington, J., Reiter, E., Tintarev, N., and Waller, A. (2010). Using NLG and sensors to support personal narrative for children with complex communication needs. In Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies, pages 1-9, Los Angeles, CA. Association for Computational Linguistics.Google Scholar
de Rosis, F. and Grasso, F. (2000). Affective natural language generation. In Paiva, A., editor, Affective Interactions, pages 204-218. Springer LNCS, Berlin, Germany.Google Scholar
Demasco, P. W. and McCoy, K. F. (1992). Generating text from compressed input: An intelligent interface for people with severe motor impairments. Communications of the ACM, 35(5): 68-78.CrossRefGoogle Scholar
Demir, S., Carberry, S., and McCoy, K. F. (2011). Summarizing information graphics textually. Computational Linguistics, 38(3):527-574.Google Scholar
Dempster, M. (2008). Using natural language generation to encourage effective communication in non-speaking people. In Proceedings of the Young Researchers Consortium at the International Conference on Computers Helping People with Special Needs, Linz, Austria.Google Scholar
Dempster, M., Alm, N., and Reiter, E. (2010). Automatic generation of conversational utterances and narrative for augmentative and alternative communication: A prototype system. In Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies, pages 10-18, Los Angeles, CA. Association for Computational Linguistics.Google Scholar
Dugard, P., File, P., and Todman, J. (2011). Single-case and Small-n Experimental Designs: A Practical Guide To Randomization Tests. Routledge, New York, NY, USA, 2nd edition.Google Scholar
Ferres, L., Lindgaard, G., and Sumegi, L. (2013). Evaluating a tool for improving accessibility to charts and graphs. ACM Transactions on Computer-Human Interaction (TOCHI), 20(5).CrossRefGoogle Scholar
Gatt, A. and Reiter, E. (2009). SimpleNLG: A realisation engine for practical applications. In Proceedings of the European Workshop on Natural Language Generation (ENLG), pages 90-93, Athens, Greece. Association for Computational Linguistics.Google Scholar
Grove, N. (2010). The Big Book of Storysharing. Special Educational Needs Joint Initiative for Training (SENJIT), Institute of Education, University of London, London, UK.Google Scholar
Light, J. and Drager, K. (2007). AAC technologies for young children with complex communication needs: State of the science and future research directions. Augmentative and Alternative Communication, 23(3):204-216.Google ScholarPubMed
Liu, X., Sen, A., Bauer, J., and Zitzmann, C. (2007). A software client for wi-fi based realtime location tracking of patients. In Proceedings of the International Conference on Medical Imaging and Informatics, pages 141-150, Beijing, China. Springer.Google Scholar
McCabe, A. and Peterson, C. (1991). Getting the story: A longitudinal study of parental styles in eliciting narratives and developing narrative skill. In McCabe, A. and Peterson, C., editors, Developing Narrative Structure, pages 217-253. Lawrence Erlbaum Associates, Hillsdale, NJ.Google Scholar
McCoy, K. F., Bedrosian, J., and Hoag, L. (2010). Implications of pragmatic and cognitive theories on the design of utterance-based AAC systems. In Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies, pages 19-27, Los Angeles, CA. Association for Computational Linguistics.Google Scholar
Mellish, C. and Dale, R. (1998). Evaluation in the context of natural language generation. Computer Speech & Language, 12(4):349-373.CrossRefGoogle Scholar
Newell, A. F., Carmichael, A., Gregor, P., Alm, N., and Waller, A. (2008). Information technology for cognitive support. In Sears, A. and Jacko, J. A., editors, The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pages 811-828. Lawrence Erlbaum Associates, Mahwah, NJ, 2nd edition.Google Scholar
Nikolova, S., Tremaine, M., and Cook, P. (2010). Click on bake to get cookies: Guiding word-finding with semantic associations. In Proceedings of the ACM SIGACCESS Conference on Computers and Accessibility (ASSETS), pages 155-162, Orlando, FL. Association for Computing Machinery.Google Scholar
Polkinghorne, D. (1995). Narrative configuration in qualitative analysis. In Hatch, J. A. and Wisniewski, R., editors, Life History and Narrative, pages 5-24. Taylor & Francis, Bristol, PA.Google Scholar
Portet, F., Reiter, E., Gatt, A., Hunter, J., Sripada, S., Freer, Y., and Sykes, C. (2009). Automatic generation of textual summaries from neonatal intensive care data. Artificial Intelligence, 173(7-8):789-816.CrossRefGoogle Scholar
Quasthoff, U. M. and Nikolaus, K. (1982). What makes a good story? Towards the production of conversational narratives. In Flammer, A. and Kintsch, W., editors, Discourse Processing. North-Holland Publishing Company, Amsterdam.Google Scholar
Reddington, J. and Tintarev, N. (2011). Automatically generating stories from sensor data. In Proceedings of the International Conference on Intelligent User Interfaces (IUI), pages 407-410, Palo Alto, CA. Association for Computing Machinery.Google Scholar
Reichenbach, H. (1947). Elements of Symbolic Logic. Macmillan Co., New York, NY.Google Scholar
Reiter, E. (2007). An architecture for data-to-text systems. In Proceedings of the European Workshop on Natural Language Generation (ENLG), pages 97-104, Saarbrücken, Germany. Association for Computational Linguistics.Google Scholar
Reiter, E. and Belz, A. (2009). An investigation into the validity of some metrics for automatically evaluating NLG systems. Computational Linguistics, 35(4):529-558.CrossRefGoogle Scholar
Reiter, E., Gatt, A., Portet, F., and van der Meulen, M. (2008). The importance of narrative and other lessons from an evaluation of an NLG system that summarises clinical data. In Proceedings of the International Conference on Natural Language Generation (INLG), pages 147-156, Salt Fork, OH. Association for Computational Linguistics.Google Scholar
Roark, B., de Villiers, J., Gibbons, C., and Fried-Oken, M. (2010). Scanning methods and language modeling for binary switch typing. In Proceedings of the NAACL HLT Workshop on Speech and Language Processing for Assistive Technologies, pages 28-36, Los Angeles, CA. Association for Computational Linguistics.Google Scholar
Robertson, J. and Good, J. (2005). Story creation in virtual game worlds. Commununications of the ACM, 48(1):61-65.CrossRefGoogle Scholar
Thomas, K. E. and Sripada, S. (2008). What's in a message? Interpreting geo-referenced data for the visually-impaired. In Proceedings of the International Conference on Natural Language Generation (INLG), pages 113-120, Salt Fork, OH. Association for Computational Linguistics.Google Scholar
Todman, J. and Alm, N. (2003). Modelling conversational pragmatics in communication aids. Journal of Pragmatics, 35(4):523-538.CrossRefGoogle Scholar
Todman, J., Alm, N., Higginbotham, J., and File, P. (2008). Whole utterance approaches in AAC. Augmentative and Alternative Communication, 24(3):235-254.CrossRefGoogle ScholarPubMed
Waller, A. (2006). Personal narrative and AAC: From transactional to interactional conversation. In Proceedings of the International Society for Augmentative and Alternative Communication (ISAAC) Research Symposium, Dtlsseldorf, Germany. International Society for Augmentative and Alternative Communication.Google Scholar
Waller, A., Dennis, F., Brodie, J., and Cairns, A. Y. (1998a). Evaluating the use of TalksBac, a predictive communication device for nonfluent adults with aphasia. International Journal of Language & Communication Disorders, 33(1):45-70.Google ScholarPubMed
Waller, A., Dennis, F., Cairns, A., Whitehead, N., Brodie, J., Newell, A. F., and Morrison, K. (1998b). The future development of TalksBac: A predictive augmentative communication system for adults with nonfluent aphasia. International Journal of Language & Communication Disorders, 33(1):45-70.Google Scholar
Waller, A., Francis, J., Tait, L., Booth, L., and Hood, H. (1999). The WriteTalk project: Story-based interactive communication. In Buhler, C. and Knops, H., editors, Assistive Technology on the Threshold of the New Millennium, pages 180-184. IOS Press, Amsterdam, The Netherlands.Google Scholar
Waller, A. and Newell, A. (1997). Towards a narrative-based communication systems. European Journal of Disorders of Communication, 32(S3):289-306.CrossRefGoogle Scholar
Waller, A., O'Mara, D., Manurung, R., Pain, H., and Ritchie, G. (2005). Facilitating user feedback in the design of a novel joke generation system for people with severe communication impairment. In Proceedings of the International Conference on Human-Computer Interaction (HCII), Las Vegas, NV. Lawrence Erlbaum Associates.Google Scholar
Waller, A., O'Mara, D. A., Tait, L., Booth, L., Brophy-Arnott, B., and Hood, H. E. (2001). Using written stories to support the use of narrative in conversational interactions: Case study. Augmentative and Alternative Communication, 17(4):221-232.CrossRefGoogle Scholar
Yarrington, D., Pennington, C., Bunnell, H. T., Gray, J., Lilley, J., Nagao, K., and Polikoff, J. (2008). ModelTalker Voice Recorder (MTVR) - a system for capturing individual voices for synthetic speech. In Proceedings of the ISAAC Biennial Conference, Montreal, Canada. International Society for Augmentative and Alternative Communication.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×