Before Bruce Chatwin died in 1989, he and Paul Theroux discussed whether travel writing was legitimised primarily by fact or fiction. Although both authors routinely drew from the intellectual fields of literature, science, anthropology, poetry, politics and history, Theroux believed that travel writers ‘record what the eye sees’, whereas Chatwin – who hated the term travel writer – preferred to ‘embroider’ his adventures with fictional embellishments. Chatwin was much happier concealing the mundane aspects of his own journey in favour of more fantastic literary descriptions, whereas Theroux believed the challenge of travel writing was including the ‘how’ of travel – the minutiae about how you get from one place to another – alongside personal observations and descriptions of foreign cultures. Chatwin and Theroux never doubt these twinned authorities, they simply disagree over which should be emphasised, and which is more persuasive for the reader. The different positions advocated by Chatwin and Theroux reveal the tension at the heart of all travel writing: it must authorise itself through both fact and fiction. This chapter examines the different ways in which facts and fictions are arranged in travel writing and argues that these competing authorities cannot be divorced from the legacy of colonialism.