Domain names, URLs and web pages

Domains

Become familiar with the major domain identifiers, such as .com, .co.uk, .org and country codes. Organisations usually try and register memorable combinations of names and identifiers, but these days if they are registering a domain for the first time they may have to take whatever they can get, so if you're not finding the site that you expect at a .com address, try the .org or even the .org.uk, version for example. In the last few years the number of different domain names has increased, so we now have ones such as .accountant, .club, .help, and .science. A complete list is available on Wikipedia at https://en.wikipedia.org/wiki/List_of_Internet_top-level_domains. You can use the site: search function to limit your search to any of these new domains, and it should give you a smaller set of results, but you'll probably still need to do a more general domain search to find everything that you need.

It's worthwhile checking to see who a domain is registered to, if you have any doubts about the validity or authority of a particular site. In the UK the organisation responsible for domain name registrations is Nominet and they have a very useful Whois tool at www.nominet.org.uk/uk-domainnames/ about-domain-names/domain-lookup-whois/whois-tool that you can use to check a domain – as long as it has .uk in the address. You can see who a domain is registered to; either a person or an organisation, their address, when the domain was registered to them and when it is going to expire. It's not a foolproof method, because a registrant can pay extra to have their details redacted and it won't find addresses that end in .com either – even if they are based in the UK. If you have no luck with Nominet, do a simple whois search within your favourite search engine and it will return a variety of tools that you can use to explore domain ownership.

Welcome to this, the fifth edition of Expert Internet Searching, which began its life back at the beginning of the century, when it was called The Advanced Internet Searcher's Handbook. That very first edition, published towards the end of 1999, didn't even mention Google. Most of the search engines that I wrote about no longer exist and over the years functionality has waxed and waned.

We're now in a world of user-generated content, access to data not just at the touch of a button but by simply asking the question into our living rooms where our mobile assistants can research and answer the question as soon as we have finished speaking it. ‘News’ now means things that happened in the last few seconds, instead of the last few hours or the last day or week, and we've seen the rise of ‘fake news’. While we have such easy access to information, and everyone can contribute to it, the other side of that coin is that everyone does it, only not for the greater good, but to promote their own political or religious agenda, and in doing so accuracy and truth become immediate casualties.

We all still see the rather silly claim that ‘it's all on Google, so why do we need libraries and librarians?’ but I firmly believe that information professionals are needed now more than ever. It's in part our role to help stem the tide of fake news, to open people's eyes to the rich abundance of information available in so many different formats, and to assist them in working out what they need to know and the best way of getting it. When I was a child and I told my careers officer that I wanted to be a librarian she said ‘is it because you like books?’ and I said ‘No, it's because I want the power’. It's even more true today than it was rather too many decades ago. The more information that we have at our fingertips, the more useful and powerful we become. However, we don't get that by simply visiting Google at every opportunity.

Introduction

We don't know how many search engines are available on the internet, but we can be certain that there are plenty more than we're aware of. As you have already seen, and will continue to see in the course of reading this book, there are plenty of options available to a searcher. Given that they all have their own advantages and disadvantages there's little point in just sticking to one. However, to go from one search engine to another and then another is going to be time-consuming and confusing, so while we might all agree that it's probably not best practice to just go to just one engine, it's understandable.

However, there's a type of search engine that helps us to overcome that problem, though if I'm being strictly accurate it's one type of engine with two variations: multi-search and meta-search. A multi-search engine can perhaps best be described as a search engine that doesn't actually run searches itself, but provides searchers with the ability to search a number of other engines from a portal or launch pad page. A meta-search engine, on the other hand, will take a query, pass it onto other search engines on behalf of the searcher, take the results that are returned, de-duplicate them and provide a single list of results.

There are many advantages to using this type of search engine. They can save users a lot of time, since results are generally returned more quickly with a single mouse click instead of the searcher typing in one search engine address after another, visiting, and then re-running the same search. Secondly, they're a good way to really highlight which results all the search engines think are the best, rather than relying on one engine to do that job for you. Thirdly, they're an excellent reminder to searchers that there's more to life than Google, and it's a good way to introduce them to others. Finally, because the searcher is interrogating a search engine via a third party, the engine can't personalise results, since it doesn't know who they are for.

Introduction

If I asked you which reference book you used in your work, you'd probably think that you'd misheard me, because of course you probably use quite a lot of them. Similarly if I asked which professional journal your library subscribed to, you'd think that was an odd question as well. However, in my work as a trainer people often think that just using one search engine is a perfectly acceptable way to work, and a common question that I'm asked is ‘What's the “best” search engine?’

There is no best search engine, as there is no best reference book or professional journal. We match the appropriate tool to the question that we've been asked, and if we don't get the answer we need, we just simply move onto the next resource until we have been able to come to a conclusion. Search engines are exactly the same. Google is a great search engine, there's no doubt about that, but as we've seen it does have its drawbacks. Consequently it's a good idea to have various alternatives ready to hand if you can't find what you need, or you simply want an alternative viewpoint on your query. This chapter will look at some of those engines, and in common with the approach that I used with Google I'll look at how they can be used, any interesting new search functions and of course, a critical look at their weaknesses.

Bing

Bing is the Microsoft alternative to Google. Microsoft has had a very chequered career with the internet and has tried several previous incarnations of search to try and compete with Google. This is the only one that's really had any success, and it doesn't look as though it's liable to change or be morphed into anything else by Microsoft at the moment. Much of the functionality is the same as Google's offerings and the data collection method is the same; that is to say, spidering websites, copying the contents of pages back to home base and then indexing them.

Introduction

I'll start this chapter by asking you a question: ‘How many search engines do you think that there are on the internet?’ The chances are fairly high (and I know this, because I ask this question a lot) that your immediate reaction will be to provide a figure in the region of perhaps ten or a dozen. If you're wildly optimistic, you might reach for several hundred, or even a thousand. In actual fact, no one has any idea as to how many search engines are available, and there are a few reasons for this. First, there's no central database of them, so we can't get a definitive number. Secondly, as we saw in the previous chapter, the market is very volatile, with engines disappearing and others appearing on a regular basis. Thirdly, it depends on your definition of a search engine, and I'll come to that in a moment. However, that doesn't help us reach any sort of answer at all. I used to keep a listing of country-based search engines, but when it reached 5000 it was simply too much work to keep up to date. I ran several searches to see if I could track down any sort of figure but found very little. I ran searches for ‘400,000 search engines’ and was able to increase this to ‘600,000’, mainly from sites offering to submit your website to that number of engines. I also ran a search for the phrase ‘use our search engine’ and had over 800,000 results. None of these are figures that I'm prepared to trust, but I've always gone with a figure that suggests up to about 400,000 of them, and no one has contradicted me yet, so that's a figure that I'm going with, but remember, that's little more than a shot in the dark!

Introduction

In the 1990s this class of search engine was extremely popular, and Yahoo! was leading the charge. It's hard to believe that at one point Yahoo! was worth over US$100 billion (www.youtube.com/watch?v=5DpspOXs1rM) and it was a huge brand, closely associated with search and its search directory. It acted as a portal to the internet when there really wasn't very much organisation at all. Websites were springing up at the rate of thousands per day and early engines such as AltaVista simply couldn't keep up with this flood. Rather than try and seek out all of these websites Yahoo! and others like them took a different approach – web authors came to them, looked through their search directory hierarchy and found an appropriate niche for their site. This would then be submitted and once approved, the website would be included in the directory. The portal approach used directory search to encourage people to visit and to stay on the website. Coupled with news, video and messaging services it was hoped that a compelling offer would ensure that consumers wouldn't look anywhere else.

However, this failed to take into account the fact that people wanted to search more quickly and effectively, and by tying advertisements to the type of searches that people were running Google was able to take an increasingly larger share of the marketplace. The growth of Facebook, Twitter and YouTube meant that people were also getting their news from various different places, and communication tools such as Messenger and WhatsApp further depreciated the value of the portal approach. Finally, because of the sheer amount of growth of websites, and importantly the increase in technology, speed of spidering websites and fall in storage costs meant that free-text engines were able to index far more, and more effectively, than directory-based engines. As users deserted the sinking ship of Yahoo! and similar sites, advertising revenue decreased and it simply wasn't possible to change course and go from a directorybased to a free-text approach, although Yahoo! did attempt that with their ‘Yahoo!-Microsoft Search Alliance’ in December 2009. In late December 2014 Yahoo! closed its Directory offering, 20 years after it began.

Introduction

In the earlier days of internet search video and multimedia was very much the poor relation. Most people used dial-up connections to access the internet and the equipment needed to create video or make voice recordings was either expensive to purchase or complicated to use – often both. By 2016, 81% of UK households had a broadband connection (either fixed or mobile) and 42% of users had taken up the option of superfast fixed broadband by the end of 2015 (www.ofcom.org.uk/about-ofcom/latest/media/facts). The total number of global broadband subscribers had grown to 10% of the world's population by the end of 2014; in the developed world this figure was 27% (https://en. wikipedia.org/wiki/Global_Internet_usage). Due in part to the cheap avail ability of webcams and more importantly the increased sophistication of smartphones, 400 hours’ worth of video is uploaded to YouTube every minute (up from the 72 hours I referenced in the previous edition of the book), and more than 1 billion hours of YouTube footage is watched every day, which is a tenfold increase since 2012. That means that if you wanted to watch an entire day's worth of uploaded video it would take you approximately 65 years to do so (https://youtube.googleblog.com/2017/02/you-know-whats-cool-billionhours. html). Meanwhile, on Facebook an average of 100 million hours of video was watched every day in January 2016, and over 8 billion views daily (http:// expandedramblings.com/index.php/by-the-numbers-7-amazing-facebook-stats).

I think we can safely say that sharing and watching video on the internet has become a serious business. Of course, a large amount of the content shared and viewed is simply for entertainment (although how one defines that is of course open to question; what is entertainment for one person may well be serious research for another), but with news items being shared on video sites, ‘how-to’ videos, educational material, television programmes and so on, content found on video has to be taken into account when searching. In this chapter I'll look at some of the ways in which searchers can utilise the of lood of content that's available and suggest some search engines to use in order to retrieve it.

Introduction

It may seem a ridiculous thought, but at one point in time it was possible to write a book about internet search and not include any references to Google, though you'd need to go back to the first edition of this book, written at the end of the last century, to find it. The rise of Google has been unstoppable, and it would be quite easy to just write about the search engine and the company behind it for chapter after chapter. A search on Amazon for ‘Google’ results in over 650,000 items for sale (up from 50,000 in 2013) and covers books, apps, videos, mobile phones, hardware and software. Google stopped being just a search engine many years ago, and is now involved in an increasing number of ventures from mobile phones to cloud computing to consumer services and more. It's the world's most dominant search engine, indexes trillions of pages and is searched billions of times every day. In August 2015 Google announced plans to reorganise all of its different interests into an overarching company called Alphabet.

Its success is mirrored by its failures, however. It has tried several times to enter the social networking environ ment to take on Facebook, but each of these attempts has met either with complete failure, with Google Wave and Google Buzz, or very limited success, with Google+. Google glasses failed to take off, despite all of the hype surrounding them. It's also had its fair share of failures in the software field as well, and closes down almost as many projects as reach fruition.

I've called this chapter ‘The world according to Google’ because it's very easy to spend your entire internet working life just using its various products and because of the way in which it tries to ‘mould’ its search results to you it's all too easy to limit what you do and see to these results.

Introduction

In Chapter 5 we saw that clustering search engines such as Carrot2 were able to display results in a non-linear form, by the use of folders and grouping like results together. There are various advantages to this approach, such as preventing searchers from thinking that the first result is in some way ‘the best’. A logical next step on from this category is to be able to see the websites themselves directly in the search results, rather than having to click on a link to go to them. The tile approach is also becoming more popular, based on the increased use of mobile devices. This chapter will look at search engines that use this approach to display results. However, that's only one side of the visual search coin – rather than seeing search results portrayed in a visual format you might actually just want to search for images themselves (which of course has a very visual component in the way that results are displayed) and this chapter will also look at some image search options in detail.

A visual search engine will display results by showing searchers the actual web pages themselves that are returned from the search that's been run. Some engines (particularly free-text) may include a thumbnail of a screenshot, but that's more to simply help identify a site, rather than offering any practical use. However, those in the visual search engines category (and it should be admitted that it's a very small category) do rather more than that – the results are the web pages themselves. Searchers can f lick from one screen to the next and get a very good indication of what's on the web pages from the SERPs, without going directly to the sites in question. There are a number of advantages to this approach; users can view a page safe in the knowledge that no one at the website will know that they are looking at it, which is helpful when it comes to competitive intelligence.

Introduction

It's an unfortunate fact of life that not everything can be put into neat little accurately labelled boxes, and search engines are no exception to this rule. Much though it grieves me, it really is necessary to have a chapter that looks at search engines that don't fit neatly into any of the other categories that I have used in the book. It may be because the search engine is so unusual that it defies any sort of categorisation (as you'll see later, Bananaslug fits that description perfectly), or because it covers content that isn't included elsewhere. Whatever the reason, there are several engines that really deserve inclusion in the book, so this chapter is where you'll find them.

Academic search engines

Academia

Academia is part social network, part database, part search engine and you can find it at www.academia.edu. It defines itself as a ‘platform for academics to share research papers. The company's mission is to accelerate the world's research.’ Almost 50 million academics have joined the service (up from the 2 million mentioned in the last edition of this book, illustrating how quickly the service has grown), adding almost 18 million papers and listing over 2 million research interests. Academia offers a simple search box, with instant suggestions as you type. Results are broken down into a variety of areas; people, documents, journals and jobs for example. Documents may range from papers that have been uploaded by individual researchers, which can be downloaded and read in a PDF format, through to links to web pages or to other social media sites such as Pinterest. If you need to find an expert in a particular area, this is a good place to start, as you can quickly see their interests, what they have published, who they are following and their recent activities on the site.

Introduction

When writing a new edition of an existing book, one of the decisions that you have to take is how much to change, and how much to keep the same. In all of the previous editions of this title I've included an introduction to the internet, and clearly I'm doing so again – why change a winning formula? However in truth, it's not really about that, it's rather more to do with the fact that I strongly believe that an introduction really IS required. As you might imagine, I talk to people all of the time about the internet, and the majority of them are not internet or information professionals, they are the average person on the street, or in my case, out walking their dogs in the park. Their knowledge of the internet is about as limited as my knowledge of the way in which the internal combustion engine works, which is to say, limited in the extreme. Of course, if you're someone who works with the internet on a regular, daily basis you're probably far more aware of the internet and what it does and doesn't do. However, it's always a good idea to have a refresher every now and then, and even if you read this chapter in a previous edition of the book it's worth scanning through it again because information does change year on year.

What doesn't change, however, is the old time-worn phrase ‘It's all on Google’. Having spent some time in 2016 looking at two big political events, the British European Union referendum and the American Presidential election, it's very easy to see that not only is it not all on Google but Google may very well just provide you with the information that it thinks you want to see, limiting your choice even further than what is available in its databases.