Why interactive information retrieval and not just information retrieval?

It is legitimate for the reader to ask why a distinction is made by this book between the terms ‘interactive information retrieval’ and ‘information retrieval’. Surely, in the current context of information seeking through web search engines, it is clear that information retrieval necessarily involves some form of interaction between searcher and system. An answer to this question lies in the early history of computerized information retrieval systems, especially in their evaluation as carried out by different disciplinary groups.

If we consider libraries as prototypical information retrieval systems, we can see one aspect of their history as moving from ‘closed stack’ libraries, in which the information seeker can interact with the information objects in the collection only through some intermediary who controls access to those objects, to ‘open stack’ libraries, in which the information seeker is able to interact directly with the information objects (although, of course, often with the benefit of an inter - mediary's assistance, and in both cases usually with the benefit of some ordering of the collection, and/or of some index to it). From this point of view, we can see at least three ways in which interaction in the information retrieval system occurs: between the information seeker and an intermediary; between the intermediary and the collection; and between the information seeker and the collection. To these three we might also add interaction of information seeker and intermediary with the index to the collection. The goal of all of this interaction is, ultimately, that the information seeker finds the information object(s) that are useful in achieving the goal that led to the information-seeking behavior (cf. Belkin, 2010).

Although the term ‘information retrieval’ was coined in 1950 by C. N. Mooers (Mooers, 1950) it came into general use only in the late 1950s, when computers began to be used for the purposes of automatically characterizing the contents of documents, and retrieving them from document collections. The characterizing of the documents was understood as indexing them (on the example of traditional library and documentation practice), and their retrieval was construed as the characterization of an information seeker's information problem as a query put to the collection, couched in the vocabulary of the indexing language.

Introduction

Multimedia here refers to images, audio and video. Multimedia information retrieval means the process of searching for and finding image, audio and video documents through a multimedia search engine. This chapter will take image retrieval as an example to discuss interaction models and interfaces developed for multimedia information retrieval and to illustrate information-seeking behaviour in relation to image search.

Current image search engines are mainly based on keyword annotations or information extracted from the image's context (e.g. web page text). This approach has three limitations. First, the manual annotation of images requires significant effort and thus may not be practical for large image collections. Second, as the complexity of the image increases, capturing image content by text alone becomes increasingly more difficult. Finally, it relies on the user being able to articulate and enter a text description of their information need using the same vocabulary (and language) as the text annotations.

In seeking to overcome these limitations, content-based image retrieval was proposed in the early 1990s (Rui et al., 1998). It searches using images rather than keywords as the query (discussed in more detail in Chapter 13, ‘Multimedia: information representation and access’). Content-based image retrieval systems have since been primarily used for image searches on collections with limited annotations, or for image searches where annotation is not required, such as trademark search (Eakins, Riley and Edwards, 2003). More recently, Google has launched a new application called ‘Google Goggles’ for Google Android mobile phones, which is a content-based search application and allows people to search for more information about a famous landmark or work of art simply by submitting a photo of that object (Jamaal, 2010).

A basic content-based image retrieval system interprets the content (e.g. colour, texture, shape and so on) of the images in a query and in the collection, calculates the similarity between the images in the query and the object images in the collection and ranks the object images in the collection according to their degree of relevance to the images in the query (Marques and Furht, 2002).

Introduction

Inherent in the concept of interactive information retrieval is the notion that we interact with some search user interface (SUI) beyond the submission of an initial query. Perhaps the most familiar SUI to many is the streamlined experience provided by Google, but many more exist in online retail, digital archives, within-website (vertical) search, legal records and elsewhere. Amazon, for example, provides a multitude of different features that together make a flexible, interactive and highly suitable gateway between users and products.

The aim of this chapter is to provide a framework for thinking about the elements that make up different SUI designs, taking into account when and where they are typically used.

Search: the way we usually see it

The SUI that many people now see daily is Google, and Figure 8.1 overleaf shows the 14 notable SUI features it provides for users on its search engine results page (SERP). The most common feature searchers expect to see is the query box (#1 in Figure 8.1), which in Google provides a maintained context so that the query can easily be edited or changed without going to the previous page. Searchers are free to enter whatever they like, including special operators that imply specific phrases or make sure certain words are not included. The second most obvious feature is the display of results (#2), which is usually ordered by how relevant they are to the search terms. Results typically highlight how they relate to the search terms by showing parts in bold font. Users are typically able to view additional results using the pagination control (#3).

We also see many control and modifier SUI features. Google provides fixed options across the top (#4) and relevant options down the left (#5) for specializing the search towards certain types of results. Further, Google allows users to restrict their results (#6), or change how they are shown (#7). It is typical for search engines to provide an advanced search to help define searches more specifically (#8). Finally, most search engines provide recommendations for related queries (#9).

Web search is unique

Web search is different from other types of information retrieval. The scale and diversity of web content is several orders of magnitude larger than what is found in traditional information retrieval corpora. Web corpora contain billions of web pages with content ranging from images to blogs to technical articles. Similarly, the scale and variety of the people who issue web queries and the tasks that underlie those queries are immense. Web search tools are used for everything from simple navigational queries to complex research tasks that extend over time. Although there are some common motivations, strategies, tasks and information targets, there is a long tail of uncommon behavior that accounts for a significant portion of all web search interactions. We begin this chapter by looking more closely at what makes web search unique. We then show how these unique aspects have led to interesting approaches to ranking, including the use of machine learning and rich features. We describe how people interact with web search results, how rich behavioral data can be used to personalize the search experience, and the challenges and opportunities that these capabilities pose for evaluation. We conclude by looking at emerging trends in web search to give a glimpse of what web search tools of the future might look like.

Large-scale, diverse content

The web is very large; the number of documents it contains is estimated in the tens of billions, with many pages generated on the fly based on underlying databases or user interaction. Before the rise of the web, search engines typically only dealt with static corpora of, at most, millions of documents. The billions of web documents search tools must contend with come in a larger variety of formats than typically dealt with in traditional information retrieval, including blogs, online stores, government pages, forums and news sites. Text can be extracted from many of these document types (e.g. HTML, PDF, Word), but not necessarily from all (e.g. images, video). The content people search for within this large and diverse corpus is broad. In addition to documents, people look for particular images, answers to their questions, entities, templates and applications.

Modern information environments are large, dynamic and complex. In the last decade alone there has been an explosion in the amount of information available to be found and the type of tools one can use to find information. In only a few decades we have moved from interacting with relatively small, carefully curated collections of information, such as those at our local university library or public library, to interacting with large volumes of heterogeneous, unclassified and mixed quality data. There are few industries, research disciplines or human activities that do not require information and one's success in these areas often depends directly on one's ability to find useful information.

Since the development of computerized systems for information access in the late 1950s many interdisciplinary researchers have tried to understand how such systems should be designed, implemented and evaluated. Developing a successful information system requires a solid understanding of users (how users think about, find and make decisions about information), of systems (how information objects should be represented and searched) and of interfaces (how the user and system should interact). However, historically, research about these different aspects has been largely unconnected, especially with regard to userand system-centred research. Much research from the systems field has been largely uninformed by an understanding of information seeking and much research in information seeking lacks a solid perspective of how systems are designed and implemented.

The bridge between user-centred and system-centred research is known as interactive information retrieval, and this is the term we most often use to describe our research alliances. Interactive information retrieval is a rich area of research that uses research from information behaviour and seeking to inform the design of interactive search systems at interface and system levels. The study of interactive information retrieval is useful not only for researchers who wish to design new systems but for anyone who needs to use, commission or evaluate search systems.

This book is an overview of the different specialties that comprise interactive information retrieval. We have called it Interactive Information Seeking, Behaviour and Retrieval in order to cover all important aspects of information retrieval – behaviour and seeking, interaction and retrieval. It shows how foundational work from studies of human information interactions has led to the design of the systems we use every day to find information.

Introduction

We build information retrieval systems to help people satisfy their information needs. Although new computer interfaces have increased the ways that users can satisfy their information needs, the predominant interfaces require users to describe their needs with words. Similarly, most retrieval systems represent the items in their collection using words. The words provided by the user are then compared to the words attached to the items in the collection. If the user's words match those of the item, then that item might be what the user wants. If the words do not match, then the item is not returned by the retrieval system. While this explanation is a simplification of matters, the way we represent documents is the first step towards obtaining a high quality retrieval system. We begin by discussing issues in representing text items that pertain to both manual and automatic representation techniques.

Text representation

Textual items in our collections could be books, journal articles, web pages, XML documents, emails, word processing files and so forth. These items vary in length and structure and may contain non-text items such as images. In all cases, we will refer to the text items in a collection as documents or items, but it is important to remember that there is a large variety of possible text items.

Although there are many possible representations of documents, our focus will be on representations that use words or tokens derived from words. The process of deciding which words to use to describe a document is called indexing and the chosen words are called index terms. Sometimes we want to represent documents with more than words and then it makes sense to talk about the use of features, which are more generic than index terms. An example of a non-word feature could be the number of words in a document.

When we, as humans, manually index documents, we look at the item, read it, and then make decisions ourselves about what index terms to use. When we automatically index, we write computer algorithms to process digital forms of the documents and make the decisions about index terms.

Information interaction and behaviour was first recognized as an area for research in the early 1960s, and a few studies had been conducted as early as the 1930s (e.g. Akers, 1931). It has expanded greatly with the introduction of computer-based information systems – and particularly with the arrival of the web. The definition of the field is still evolving and its title has gradually changed. Nevertheless, its goal has always been to investigate the behaviour of people when they interact with information (see Chapter 2, ‘Information Behaviour and Seeking’, for a description of the main components of research in human information-seeking behaviour).

This chapter analyses the approaches that human information behaviour researchers use when they investigate information-seeking behaviour – how people look for information when they use electronic information systems. The development of these approaches followed those in other social sciences, if with some delay. While human information behaviour studies in the first two decades were guided by the same approach (materialized predominantly through large-scale questionnaires), later various approaches were used. These are not isolated from one another, but connected through several dimensions that point to their commonalities and differences.

In this chapter I explain some of these dimensions and discuss the contributions of human information behaviour studies to research and design. The dimensions I consider are philosophical stance, research setting, level of control, instruments for data collection, level of generalization, nature of data and analysis, and method of reasoning.

The philosophical stance

All researchers have a philosophical stance that guides their work, whether or from one project to another. A researcher's stance shapes the decisions about what research questions to investigate and these questions guide the study design, including the approach to be applied. Human information behaviour research has been directed by several stances and discussing all of them requires a book of its own. Here I adapt a simplistic dichotomy – that between the positivist and the non-positivist stances. The positivist stance was the first to influence human information behaviour and information retrieval research. Information retrieval evaluation investigations have been guided only by positivism – which has often been named ‘the scientific method’ – and is still the dominant stance in human information behaviour research.

Search user interfaces are at the centre of interactive information seeking and retrieval. Yes, search user interfaces include the jobs that the quotation above enumerates and even more. While this book is not only about interfaces, it is also about those jobs and even more about that more.

This book reports on the recent advances on interaction topics reflected in its title. However, the historic roots of these topics go deep – and like all roots they are entangled and not that visible. But we can learn from them in terms of problems addressed and solutions attempted. Research on information-seeking behaviour, under various names and orientations, goes back many decades. So let us start with a bit of general history of the topics covered by the book; more specific history is presented in Chapter 1.

Activities associated with what we now call information retrieval existed and were practised long before Calvin Mooers (1919–1994, physicist and information retrieval pioneer) coined the term information retrieval in 1951. They were rooted in libraries but left them to become part of technology.

Mooers (1951) defined information retrieval as follows:

Not surprisingly, interaction is not mentioned in this definition – at the time the concept and term (in its present denotation and connotation) did not exist, even though interaction was practised from the start using ‘whatever systems, techniques, or machines’ were available. Even ‘intellectual aspects … for search’ imply interaction. As it turned out, the ‘systems, techniques or machines’ on the one hand and ‘intellectual aspects’ on the other became two major branches of problem orientation and inspiration for information retrieval research and practice.

Introduction

Information seeking is a core human life activity, driven by information needs, which are situation-driven phenomena emerging from people's desire to tackle a problem (Belkin, Oddy and Brooks, 1982a, 1982b) or address an incompleteness (Mackay, 1960; Belkin, 1980; Dervin, 1983). Search systems provide a means for people to find the information they seek rapidly. Interaction between searchers and these systems is important for effective information retrieval, whose interaction is primarily manifested in mainstream search systems, such as web search engines, through multi-turn queries comprising dialogue (input by the user) and search results (output by the system). Query formulation can be an intellectually demanding process that has a direct impact on the relevancy of search results (Croft and Thompson, 1987). When searchers are unfamiliar with the collection makeup and retrieval environment or information needs are vague or complex, query formulation challenges are magnified (Furnas et al., 1987; Salton and Buckley, 1990; Spink, Greisdorf and Bateman, 1998). To help users, search systems can provide support beyond processing rudimentary queries and providing search results. Systems can help their users determine the relevant items, create effective search queries, and make search decisions such as determining how best to use the query. Among other benefits, these techniques can help searchers better specify well defined information needs or further develop their previously ill-defined needs. In this chapter we describe techniques that have emerged from the information retrieval research community in recent decades to support search interaction. We begin with a brief description of how interaction is traditionally modeled in information-seeking research.

Modeling information seeking

Information retrieval traditionally conforms to a query-centric model of information seeking known as specified searching, where the query conceived by a searcher and presented to the system is assumed to specify the information that the searcher is trying to locate (Oddy, 1977). This model assumes that the information being sought exists in the document collection being searched and that the goal of the search process is to retrieve that information. Although this paradigm permits query refinement, information-seeking behavior is constrained to analytical search strategies that yield precise results with minimal need for result set examination and item comparison (Marchionini, 2006b). Such information-seeking scenarios would benefit from support for lookup searches and other specialized search support.

Introduction

A substantial body of literature exists on empirical studies and theoretical discussions of information behavior. Taking a literary-warrant approach, we note that research in human information behavior has investigated a broad range of topics: information needs, information seeking, looking for information, searching information retrieval systems, browsing information resources, finding inform ation, relevance judgment, information use, information generation, infor - mation communication, information sharing, information transfer, personal information management, accidental information encountering, serendipitous information, information avoidance, affect associated with information, information habit, information style, and any information-related human behavior.

As a preferred term, ‘information behavior’ has its evolutionary path. In the early 19th century, studies about information use focused on usage of information resources and channels or systems (e.g. journals, interpersonal communications and libraries). After World War 2, the rapid development in science and technology led to the advent of online information retrieval systems, which marked a new era of information storage, dissemination and retrieval. User studies in the late 1960s and 1970s expanded from investigating information use to observing information need, for example by scientists and engineers in their work environments. It was also during this period that much debate was devoted to the concept of relevance as it was a key element for two traditional measures of information retrieval performance: recall (the percentage of the relevant documents being retrieved) and precision (the percentage of the retrieved documents that are relevant). In the late 1970s and 1980s, the information world witnessed the transition from card catalogs and print indexes and abstracts to online public access catalogs (OPACs) and various commercial online information retrieval services.

The term ‘information seeking’ occurred in the literature as a short form for information need, seeking and use. The first International Conference on Information Seeking in Context (ISIC), held in Finland in 1996, was significant as the meaning of context was debated; a substantial number of papers reported on research of real users taking qualitative, naturalistic or multiple approaches; and research findings included not only purposive, active information-seeking behavior such as work-task-related behavior, but also unintentional, passive information behavior such as accidental information encountering and information avoidance.

Introduction

This chapter considers the social component of interactive information retrieval: what is the role of other people in searching and browsing? For simplicity we begin by considering situations without computers. After all, you can interactively retrieve information without a computer; you just have to interact with someone or something else. Such an analysis can then help us think about the new forms of collaborative interactions that extend our conceptions of information search, made possible by the growth of networked ubiquitous computing technology.

Information searching and browsing have often been conceptualized as solitary activities; however, they always have a social component. We may talk about ‘the’ searcher or ‘the’ user of a database or information resource. Our focus may be on individual uses and our research may look at individual users. Our experiments may be designed to observe the behaviors of individual subjects. Our models and theories derived from our empirical analyses may focus substantially or exclusively on an individual's evolving goals, thoughts, beliefs, emotions and actions. Nevertheless there are always social aspects of information seeking and use present, both implicitly and explicitly.

We start by summarizing some of the history of information access with an emphasis on social and collaborative interactions. Then we look at the nature of recommendations, social search and interfaces to support collaboration between information seekers. Following this we consider how the design of interactive information systems is influenced by social elements.

Background

The history of organized information is, necessarily, one of social relationships. The organizers structured documents to ease retrieval, possibly for themselves in the first instance, but then for users who wished to access the collection. Personal libraries, organizational libraries, and then public libraries provided access to information resources hundreds of years before computers arrived. These libraries contained, or indeed embodied, two main social relationships between the librarians and the users. The simplest relationship was a dialogue between the information seeker and the librarian, which has evolved into the reference interview. The other relationship was indirect, via the structure of the collection; where the librarians arranged the items to facilitate retrieval. These arrangements have evolved into classification systems such as the Dewey Decimal Classification. As the literary record grew through books and journals the connections between published works, authors, readers and information seekers have become ever more complex.