You work for an airline in the operations department. They ask you to reach into the historical company and industry datasets and figure out the expected loading for every flight in their book for this coming year. This ties directly into projected revenue. In the back of your mind, you know that expenses for the company need to be offset by revenue. It doesn't help to be overly optimistic because the risk of not meeting the projections carries with it the risk of future disappointment.

What should you do? Passenger boardings are cyclical. There are risks of a downturn. But a downturn could impact profitability of the entire company. The price of oil and jet fuel is important. Mergers happen every year which change the competitive landscape and make some routes more efficient.

Time series analysis can certainly help. Just like confidence intervals in statistics (please see also the Appendix), there is a band of uncertainty around any of the projections. The expected airline passenger loadings is a random variable. Passenger boarding is one of several arenas we will explore with the help of tools in R.

Examining Time Series

We begin with a quick survey of the types of times series we will model. We can use the quantmod and PerformanceAnalytics packages, firstly. We will define the vector of symbols to be downloaded. The symbols GSPC, VIX, TNX, refer to the S&P 500 index, the CME volatility index, and the ten-year treasury yield, respectively. Use the getSymbols() function to download the time series for the symbols in sym.vec between the dates of January 3, 2003 and September 10, 2015. If the quantmod and PerformanceAnalytics packages are not yet installed, they can be installed with the commands install.packages(“quantmod”, dependencies=TRUE) and install.packages(“PerformanceAnalytics”, dependencies=TRUE).

The first plot is of the S&P 500, shown in Figure 6.1.We see the peak of the market in early 2007 and then the steep decline as the housing crisis hit.We see the market bottom out in mid 2009 and begin a large rally that lasts through mid 2015. We see minor corrections in 2011 and 2012 due to uncertainty surrounding the government debt in the Euro zone, and a significant sell-off in mid 2015 due to uncertainty surrounding China's economy.

When traveling across the agricultural American Midwest, one can hear AM radio stations broadcast the futures prices of corn, soybeans, wheat, and other commodities every weekday at various times of the day. Iowa and Illinois lead the nation in corn production. Kansas leads the nation in wheat production. Listening to these farm reports is entertaining. Included in the broadcasts are very detailed weather reports. Weather is critical to many producers’ livelihoods. After a few weeks of listening to these broadcasts, we learn that nobody truly knows for sure whether the agricultural market prices will go up or down on a trading day. Hedging the production chain price risk, especially for the farmers, who seasonally grow the crops, is achieved with futures and option securities. Producers very often want to lock in a price for delivering corn at the end of the season, or want to be compensated for a significant reduction in the agricultural price in order to guarantee recovering their fixed costs over the upcoming days and months.

From Chapter 14, we have gained more familiarity with the random walk processes assumed by the option models. We continue the option theme from the prior chapter and examine a very popular model for pricing European options. The most famous and widely accepted model of option valuation model is known as the Black–Scholes model of 1973 (Black and Myron, 1973). It revolutionized the pricing and trading of options which, prior to this, were priced in rather arbitrary ways. Black and Scholes relied upon the stochastic calculus. Stochastic calculus was invented by Itô in 1951 to address the need for a calculus for random variables as functions over time, like our stock market prices (Ito, 1951). Together with Merton, Black and Scholes were awarded the 1997 Nobel Prize in Economic Sciences for this invention. We discuss the Black–Scholes model here in order to complete our tour of financial analytics. We will try to make minimal use of stochastic calculus.

We saved this more mathematical material for the end of the book because it involves the more complex type of security: options. As mentioned in Chapter 14, options are derivative securities.

In 1994 the Channel Tunnel opened between England and France, allowing high-speed Eurostar trains to whisk passengers from the continent to the United Kingdom and back on a grand scale. What an amazing engineering feat it was for the time (beyond many people's earlier imaginations), yet we take it for granted today. In 1994, Grumman Aerospace Corporation, the chief contractor on the Apollo Lunar Module, was acquired by Northrop Corporation to form the new aerospace giant, Northrop Grumman. It was the prime contractor of the newly deployed advanced technology B-2 Stealth Bomber. On a much more mundane and personal scale, also in 1994, in a townhouse just outside the City of Chicago, I was performing a tedious daily exercise: looking up daily closing prices each evening in a stack of Investor's Business Daily newspapers for the two stock investments that were about to be purchased. This was not only to find out their running rate of return but also to find out their historical volatility relative to other stocks before entering into the positions. Doing this manual calculation was slow and tedious. The WorldWideWeb was introduced in the form of the Mosaic browser the next year. It was not long before Yahoo! was posting stock quotes and historical price charts, as well as technical indicators on the charts, available on demand for free in just a few seconds via the new web browsers.

The advent of spreadsheet software took analysts to a new level of analytical thinking. No longer were live, human-operated calculations limited to a single dimension. Each row or column could present a time dimension, a production category, a business scenario. And the automated dependency feature made revisions quite easy. Now spreadsheets can be used for a prototype for a more sophisticated and permanent analytical product: the large-scale, analytical computer program.

With modern programming languages like R and Python®, a skilled analyst can now design their analytic logic with significantly less effort than before, using resources such as Yahoo! or other free services for historical quotes. It has been said that Python's terse syntax allows for programs with the same functionality as their Java equivalents, yet four times smaller, and we suspect that R is similar. A small financial laboratory can be built on a laptop costing less than $200 in a matter of weeks, simulating multiple market variables as required.

Statistics is a mathematical science which is concerned with collecting and organizing data and conducting experiments involving random variables which represent the data. These random variables can represent natural or simulated events. The amazing attribute of statistics is its ability to explain the organization of the data we observe. In this chapter we will cover some basic formulas that will lay a foundation for the subsequent analytics framework.

A discussion of statistics is necessary for any treatment of financial analytics. In order to discuss investments in financial instruments from a quantitative perspective, a certain amount of preliminary background is needed. This will provide a higher level of accuracy. We will not be able to do our job well without it. This background will be conveniently stated in terms of formulas, beginning here and continuing throughout the book. Formulas provide crisp specifications for the computer instructions in R.

We begin with probability with discrete outcomes. After completing the first three sections, the reader is invited to visit the Appendix for a review of the many potential probability distributions and statistical analysis concepts that are used in analytics.

Probability

In probability we are concerned with the likelihood of events. An event A is defined such that ∅ ⊆ A ⊆ S, where ∅ is defined as the null space or empty set and S is defined as the sample space or set of all possible outcomes. We also set that the probability of event A occurring as 0 ≤ P(A) ≤ 1, where the probability of the null space is P(∅) = 0 and the probability of the sample space is P(S) = 1. We define the complement Ac as the set satisfying 1) A ∪ Ac = S and 2) A ∩ Ac = ∅. While they might seem superfluous, these two conditions simply make mathematically rigorous the requirement that (1) an event must either happen or not happen, and (2) an event may not both happen and not happen.

The data mining and machine learning literature is now flush with scenarios about predicting baseball players’ salaries from their prior year's number of hits and walks and about predicting the product sales from the prices, customer income, and level of advertising. These are amazing and noteworthy stories. They inspire data scientists to continue their cause. The classic examples feature a large two-dimensional array of cases as rows with the independent variables, also known as the stimulus variables, and the predictable variables, also known as the response variables, as columns. If we are predicting an athlete's salary, this figure is hand-tuned by the people who negotiate contracts. Better athletic production yields better salaries as athletes are constantly compared to one other. The salary is a figure updated usually one time per year at most. And only a handful of people are involved in setting the athlete's salary or the price of a consumer item. So these represent ideal cases in predictability.

Unfortunately, prediction in the case of financial analytics never turns out to be as accurate as in these sports and marketing areas. There is just more random noise in the financial markets with prices that are updated every second of every trading day. Thousands of participants are involved. Every security is affected by many other securities. For example, oil prices are affected not only by oil supply and demand and by the volume of trades at each incremental oil price level but by interest rates and various foreign exchange rates. Nevertheless, one can try these financial predictions using the same techniques in order to experiment and observe what can be predicted.

The process of attempting prediction yields at least two benefits. On the one hand, it may be possible to predict attributes from combinations of other attributes: in this case, response variables from stimulus variables. If this was the case, we can stand alongside those other successes in other areas of data science. On the other hand, prediction may not be possible or even all that useful, however; the collection exercise, getting all the data into rows and columns of the array, provides observations that can be made in an unsupervised learning sense. And discoveries can be made by applying thresholds or sorting and filtering the attributes to find maximally performing securities, as we are able to observe in Chapter 7.