Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- Foreword
- Acknowledgements
- List of acronyms
- Part 1 Fundamentals
- Part II Application development
- Part III System architecture
- Part IV Application design and analytics
- 9 Design principles and patterns for stream processing applications
- 10 Stream analytics: data pre-processing and transformation
- 11 Stream analytics: modeling and evaluation
- Part V Case studies
- Part VI Closing notes
- Keywords and identifiers index
- Index
- References
9 - Design principles and patterns for stream processing applications
from Part IV - Application design and analytics
Published online by Cambridge University Press: 05 March 2014
- Frontmatter
- Dedication
- Contents
- Preface
- Foreword
- Acknowledgements
- List of acronyms
- Part 1 Fundamentals
- Part II Application development
- Part III System architecture
- Part IV Application design and analytics
- 9 Design principles and patterns for stream processing applications
- 10 Stream analytics: data pre-processing and transformation
- 11 Stream analytics: modeling and evaluation
- Part V Case studies
- Part VI Closing notes
- Keywords and identifiers index
- Index
- References
Summary
Overview
In the preceding chapters we described the stream processing programming model and the system architecture that supports it. In this chapter we will describe the principles of stream processing application design, and provide patterns to illustrate effective and efficient ways in which these principles can be put into practice.
We look at look at functional design patterns [1] and principles that describe effective ways to accomplish stream processing tasks, as well as non-functional ones [1] that address cross-cutting concerns such as scalability, performance, and fault tolerance.
This chapter is organized as follows. Section 9.2 describes functional design patterns and principles, covering the topics of edge adaptation, flow manipulation, and dynamic adaptation. Section 9.3 describes non-functional design patterns and principles, covering the topics of application composition, parallelization, optimization, and fault tolerance.
Functional design patterns and principles
We start by examining functional design patterns and principles, covering edge adaptation, flow manipulation, and dynamic adaptation.
9.2.1 Edge adaptation
SPAs consume data from external sources available in various different formats and accessible by employing different protocols. Similarly, results produced by streaming applications are often consumed by external systems in various formats and through different protocols. We term the process of interacting with external systems to receive and send data as edge adaptation.
- Type
- Chapter
- Information
- Fundamentals of Stream ProcessingApplication Design, Systems, and Analytics, pp. 275 - 341Publisher: Cambridge University PressPrint publication year: 2014