Book contents
- Frontmatter
- Contents
- List of Figures
- Preface
- 1 The Key Idea
- 2 Approaches to Parallel Software Development
- 3 Architectural Background
- 4 Models and Their Properties
- 5 The Categorical Data Type of Lists
- 6 Software Development Using Lists
- 7 Other Operations on Lists
- 8 A Cost Calculus for Lists
- 9 Building Categorical Data Types
- 10 Lists, Bags, and Finite Sets
- 11 Trees
- 12 Arrays
- 13 Graphs
- 14 Conclusions
- A C++ Library for Lists
- B Historical Background
- References
- Index
3 - Architectural Background
Published online by Cambridge University Press: 25 October 2009
- Frontmatter
- Contents
- List of Figures
- Preface
- 1 The Key Idea
- 2 Approaches to Parallel Software Development
- 3 Architectural Background
- 4 Models and Their Properties
- 5 The Categorical Data Type of Lists
- 6 Software Development Using Lists
- 7 Other Operations on Lists
- 8 A Cost Calculus for Lists
- 9 Building Categorical Data Types
- 10 Lists, Bags, and Finite Sets
- 11 Trees
- 12 Arrays
- 13 Graphs
- 14 Conclusions
- A C++ Library for Lists
- B Historical Background
- References
- Index
Summary
In this chapter, we explore the constraints imposed on models by the properties of parallel architectures. We are only concerned, of course, about theoretical properties, because we cannot predict technological properties very far into the future. Recent foundational results, particularly by Valiant [200], show that arbitrary parallel programs can be emulated efficiently on certain classes of parallel architectures, but that inefficiencies are unavoidable on others. Thus a model of parallel computation that expresses arbitrary computations cannot be efficiently implementable over the full range of parallel architecture classes. The difficulty lies primarily in the volume of communication that takes place during computations. Thus we are driven to choose between two quite different approaches to designing models: accepting some inefficiency, or restricting communication in some way.
Parallel Architectures
We consider four architecture classes:
shared-memory MIMD architectures, consisting of processors executing independently, but communicating through a shared memory, visible to them all;
distributed-memory MIMD architectures, consisting of processors executing independently, each with its own memory, and communicating using an interconnection network whose capacity grows as p log p, where p is the number of processors;
distributed-memory MIMD architectures, consisting of processors executing independently, each with its own memory, and communicating using an interconnection network whose capacity grows only linearly with the number of processors (that is, the number of communication links per processor is constant);
SIMD architectures, consisting of a single instruction stream, broadcast to a set of data processors whose memory organisation is either shared or distributed.
- Type
- Chapter
- Information
- Foundations of Parallel Programming , pp. 15 - 26Publisher: Cambridge University PressPrint publication year: 1994