Although experiments and data are crucial to the fields of science and engineering, scientists and engineers also know experiments and data are messy. The number of ways data collection can fail – people forget to turn on the right switch at the right time, insects fly into critical components of an experiment, broken instruments, etc. – are myriad. Data-analysis routines then need to handle the cases of missing or bad data.

In Chapter 5, we learned about array syntax and how this enables us to use arrays not only to hold a collection of numbers but also to do calculations with those collections. In this chapter, we further explore how to analyze collections of data, such as arrays. Array syntax is powerful but there are many kinds of calculations that require the ability of examining and analyzing individual elements of an array.

In previous chapters, we looked at one-dimensional and two-dimensional arrays and used them in a variety of tasks, including data analysis, prognostic modeling, and the management of the output from multiple model runs. Arrays are, however, not limited to one or two dimensions. If we have a three-dimensional physical space – made up of length, width, and height – the measurements of some quantity every meter in length, width, and height in that space (at the same instant in time) would be nicely stored as elements in a three-dimensional array. If all such measurements are taken at regular intervals in time – say every hour – those values could be stored as elements in a four-dimensional array.

When we first learn to program, our programs are usually very short, limited in their functionality, and seldom more than a single file. For such short programs, written by a single person, we can keep track of the different versions of our program and describe to a new user how to install the program by writing out a description and instructions in a few files of documentation (perhaps a README.txt file or a short user’s manual written in Word).

In Section 10.2.4, we examined copying and deleting files and directories. The tools we used – copytree, rmtree, and move – automatically handled directory trees. For instance, a copy operation using copytree on a directory copies not just the topmost directory but all files and subdirectories underneath the topmost directory, and so on to every file and subdirectory. In the present chapter, we explore how a programming language implements the idea of “do-this-action-over-nested-levels” through the principle of recursion.

In Section 11.2, we discussed the importance of testing our programs in order for us to have confidence they are working the way they should. We discussed basic tests and the use of a debugger to help us step through our program slowly to find bugs. As our programs become larger, however, we need a more formalized framework and set of tools to help us in our testing. In this chapter, we introduce two such tools: linting or static checking, and formal unit testing frameworks. We conclude by describing the “test-driven development” mindset, which uses all these tools. That mindset can help us to write bulletproof programs.

In the we saw how we can use Python as a basic calculator. Along the way, we were introduced to expressions, variables, and environments to type in and run Python code. In this chapter, we will extend our use of Python as a basic calculator to tasks associated with a scientific calculator.

Compared with their predecessors, modern computers are fast. Even wearable computing devices today boast processing power that only a few human generations ago would have placed them in the pantheon of the fastest computers on the planet. No amount of computational power, however, will do us any good if the program we write makes poor use of processor cycles and memory. Throughout the present work, we have focused on trying to “get the job done” and have written our programs with a focus on accomplishing the science or engineering task at hand rather than the speed at which the program accomplishes the task.

In previous chapters, we have used looping and array syntax to analyze data in arrays of a number of dimensions. When we used looping constructs to examine the elements of arrays, we also found that branching statements like the if statement enable us to do different calculations depending upon whether certain conditions are true or not. Is it possible, however, to include branching logic into array syntax and avoid writing loops while still doing different calculations depending on whether certain conditions are true or not? In this chapter, we examine array functions and constructs that enable us to do exactly that.

In Chapter 9, we were introduced to objects and object-oriented programming (OOP). In the objects we considered, however, the classes those objects were instances of were all written by others. We just made use of the templates or common patterns that describe the array, list, string, and other objects discussed in that chapter to make arrays, lists, strings, etc.