Have you heard the parable of the man who lost his car keys? Walking to his car from the office in the dark he fumbles for the keys to open the door, and drops them in the gutter. But the light in the gutter is poor, so he undertakes his search on the pavement, where the light from the streetlight is brighter and it's easier to see.

Unable to find his keys, he ends up walking home.

Introduction

We often look for information to help us answer our questions (or find our car keys) in the places where it is easiest to look, where information is easiest to find, even if it isn't the right information to answer our particular questions. The ‘streetlight effect’ (Freedman, 2010) means that we end up focusing our time and attention on the information and data we discover: how we can make it better, find more of it and so on. We forget the questions that we originally wanted to answer. We are so busy looking where the light is good that we forget what we were looking for and why it was important.

The streetlight effect reminds us that just because we have information it doesn't mean that it is the right information, or that it is telling us what we really want to know. There is also little benefit in collecting all this information – like searching in the light – if, in the end, you find little that you can actually use. If you can't act on the information to inform and improve what you are doing, what is the rationale for collecting all this information? If you're not going to find your keys where the light is good, there is little point in looking there.

Increasingly the scholarly community understands that what we measure should be what is most important to us. Yet the most important things are often obscured by poor light. As research increasingly migrates to the web, with both online journals and more informal channels such as Twitter, blogs and Facebook, what we measure is changing as well. It no longer makes sense for us to base our metrics solely on previous standards of success – such as the number of times an article is cited in other articles – if increasingly we also value tweets or likes, or a quote in a Wikipedia article.