To save content items to your account,
please confirm that you agree to abide by our usage policies.
If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account.
Find out more about saving content to .
To save content items to your Kindle, first ensure firstname.lastname@example.org
is added to your Approved Personal Document E-mail List under your Personal Document Settings
on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part
of your Kindle email address below.
Find out more about saving to your Kindle.
Note you can select to save to either the @free.kindle.com or @kindle.com variations.
‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi.
‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.
Many kinds of information technology can be used to make meetings more productive, some of which are related to what happens before and after meetings, while others are intended to be used during a meeting. Document repositories, presentation software, and even intelligent lighting can all play their part. However, the following discussion of user requirements will be restricted to systems that draw on the multimodal signal processing techniques described in the earlier chapters of this book to capture and analyze meetings. Such systems might help people understand something about a past meeting that has been stored in an archive, or they might aid meeting participants in some way during the meeting itself. For instance, they might help users understand what has been said at a meeting, or even convey an idea of who was present, who spoke, and what the interaction was like. We will refer to all such systems, regardless of their purpose or when they are used, as “meeting support technology.”
This chapter reviews the main methods and studies that elicited and analyzed user needs for meeting support technology in the past decade. The chapter starts by arguing that what is required is an iterative software process that through interaction between developers and potential users gradually narrows and refines sets of requirements for individual applications. Then, it both illustrates the approach and lays out specific user requirements by discussing the major user studies that have been conducted for meeting support technology.
Meeting support technology evaluation can broadly be considered to be in three categories, which will be discussed in sequence in this chapter, in terms of goals, methods, and outcomes, following a brief introduction on methodology and undertakings prior to the AMI Consortium (Section 13.1). Evaluation efforts can be technology-centric, focused on determining how specific systems or interfaces performed in the tasks for which they were designed (Section 13.2). Evaluations can also adopt a task-centric view, defining common reference tasks such as fact finding or verification, which directly support cross-comparisons of different systems and interfaces (Section 13.3). Finally, the user-centric approach evaluates meeting support technology in its real context of use, measuring the increase in efficiency and user satisfaction that it brings (Section 13.4).
These aspects of evaluation differ from the component evaluation that accompanies each of the underlying technologies described in Chapters 3 to 10, which is often a black-box evaluation based on reference data and distance metrics (although task-centric approaches have been adopted for summarization evaluation, as shown in Chapter 10). Rather, the evaluation of meeting support technology is a stage in a complex software development process for which the helix model was proposed in Chapter 11. We think back on this process in the light of evaluation undertakings, especially for meeting browsers, at the end of this chapter (Section 13.5).
Approaches to evaluation: methods, experiments, campaigns
The evaluation of meeting browsers, as pieces of software, should be related (at least in theory) to a precise view of the specifications they answer.
Bringing together experts in multimodal signal processing, this book provides a detailed introduction to the area, with a focus on the analysis, recognition and interpretation of human communication. The technology described has powerful applications. For instance, automatic analysis of the outputs of cameras and microphones in a meeting can make sense of what is happening – who spoke, what they said, whether there was an active discussion and who was dominant in it. These analyses are layered to move from basic interpretations of the signals to richer semantic information. The book covers the necessary analyses in a tutorial manner, going from basic ideas to recent research results. It includes chapters on advanced speech processing and computer vision technologies, language understanding, interaction modeling and abstraction, as well as meeting support technology. This guide connects fundamental research with a wide range of prototype applications to support and analyze group interactions in meetings.
This book is an introduction to multimodal signal processing. In it, we use the goal of building applications that can understand meetings as a way to focus and motivate the processing we describe. Multimodal signal processing takes the outputs of capture devices running at the same time – primarily cameras and microphones, but also electronic whiteboards and pens – and automatically analyzes them to make sense of what is happening in the space being recorded. For instance, these analyses might indicate who spoke, what was said, whether there was an active discussion, and who was dominant in it. These analyses require the capture of multimodal data using a range of signals, followed by a low-level automatic annotation of them, gradually layering up annotation until information that relates to user requirements is extracted.
Multimodal signal processing can be done in real time, that is, fast enough to build applications that influence the group while they are together, or offline – not always but often at higher quality – for later review of what went on. It can also be done for groups that are all together in one space, typically an instrumented meeting room, or for groups that are in different spaces but use technology such as videoconferencing to communicate. The book thus introduces automatic approaches to capturing, processing, and ultimately understanding human interaction in meetings, and describes the state of the art for all technologies involved.
In this paper, we describe a system for coreference resolution and emphasize the role of evaluation for its design. The goal of the system is to group referring expressions (identified beforehand in narrative texts) into sets of coreferring expressions that correspond to discourse entities. Several knowledge sources are distinguished, such as referential compatibility between a referring expression and a discourse entity, activation factors for discourse entities, size of working memory, or meta-rules for the creation of discourse entities. For each of them, the theoretical analysis of its relevance is compared to scores obtained through evaluation. After looping through all knowledge sources, an optimal behavior is chosen, then evaluated on test data. The paper also discusses evaluation measures as well as data annotation, and compares the present approach to others in the field.
Email your librarian or administrator to recommend adding this to your organisation's collection.