Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- Foreword
- Acknowledgements
- List of acronyms
- Part 1 Fundamentals
- Part II Application development
- Part III System architecture
- Part IV Application design and analytics
- 9 Design principles and patterns for stream processing applications
- 10 Stream analytics: data pre-processing and transformation
- 11 Stream analytics: modeling and evaluation
- Part V Case studies
- Part VI Closing notes
- Keywords and identifiers index
- Index
- References
10 - Stream analytics: data pre-processing and transformation
from Part IV - Application design and analytics
Published online by Cambridge University Press: 05 March 2014
- Frontmatter
- Dedication
- Contents
- Preface
- Foreword
- Acknowledgements
- List of acronyms
- Part 1 Fundamentals
- Part II Application development
- Part III System architecture
- Part IV Application design and analytics
- 9 Design principles and patterns for stream processing applications
- 10 Stream analytics: data pre-processing and transformation
- 11 Stream analytics: modeling and evaluation
- Part V Case studies
- Part VI Closing notes
- Keywords and identifiers index
- Index
- References
Summary
Overview
Continuous operation and data analysis are two of the distinguishing features of the stream processing paradigm. Arguably, the way in which SPAs employ analytics is what makes them invaluable to many businesses and scientific organizations.
In earlier chapters, we discussed how to architect and build a SPA to perform its analytical task efficiently. Yet, we haven't yet addressed any of the algorithmic issues surrounding the implementation of the analytical task itself.
Now that the stage is set and we have the knowledge and tools for engineering a high-performance SPA, we switch the focus to studying how existing stream processing and mining algorithms work, and how new ones can be designed. In the next two chapters, we will examine techniques drawn from data mining, machine learning, statistics, and other fields and show how they have been adapted and evolved to perform in the context of stream processing.
This chapter is organized as follows. The following two sections provide a conceptual introduction of the mining process in terms of its five broad steps: data acquisition, pre-processing, transformation, as well as modeling and evaluation (Section 10.2), followed by a description of the mathematical notation to be used when discussing specific algorithms (Section 10.3).
Since many issues associated with data acquisition were discussed in Section 9.2.1, in this chapter, we focus on the second and third steps, specifically on the techniques for data pre-processing and transformation.
- Type
- Chapter
- Information
- Fundamentals of Stream ProcessingApplication Design, Systems, and Analytics, pp. 342 - 387Publisher: Cambridge University PressPrint publication year: 2014
References
- 2
- Cited by