Book contents
- Frontmatter
- Dedication
- Contents
- Preface
- Foreword
- Acknowledgements
- List of acronyms
- Part 1 Fundamentals
- Part II Application development
- 3 Application development – the basics
- 4 Application development – data flow programming
- 5 Large-scale development – modularity, extensibility, and distribution
- 6 Visualization and debugging
- Part III System architecture
- Part IV Application design and analytics
- Part V Case studies
- Part VI Closing notes
- Keywords and identifiers index
- Index
- References
5 - Large-scale development – modularity, extensibility, and distribution
from Part II - Application development
Published online by Cambridge University Press: 05 March 2014
- Frontmatter
- Dedication
- Contents
- Preface
- Foreword
- Acknowledgements
- List of acronyms
- Part 1 Fundamentals
- Part II Application development
- 3 Application development – the basics
- 4 Application development – data flow programming
- 5 Large-scale development – modularity, extensibility, and distribution
- 6 Visualization and debugging
- Part III System architecture
- Part IV Application design and analytics
- Part V Case studies
- Part VI Closing notes
- Keywords and identifiers index
- Index
- References
Summary
Overview
In this chapter, we study modularity, extensibility, and distribution. As part of modularity, we look at how types, functions, primitive, and composite operators can be organized into toolkits to facilitate the development of large-scale applications.
As part of extensibility, we discuss how to extend a SPS with new cross-domain and domain-specific operators. As part of distribution, we discuss mechanisms to manage and control how the logical view of a SPA maps to a corresponding physical view to be deployed on a set of distributed hosts.
This chapter is organized as follows. Section 5.2 discusses modularity and extensibility, providing the foundations for structuring large, complex, and multi-component applications. Section 5.3 tackles the issue of distributed programming and deployment.
Modularity and extensibility
Large-scale software engineering and system development in programming languages such as C++ and Java relies on creating a complete application from a collection of smaller building blocks. The design of complex SPAs is no different. Such a design and development approach is referred to as modularity [1].
The reliance on a modular design aims at mitigating engineering complexity by organizing an application into modules or components, where the relationship between these components is expressed in terms of interactions through well-defined component interfaces.
In earlier chapters we discussed how composite operators facilitate the creation of modular SPAs.
- Type
- Chapter
- Information
- Fundamentals of Stream ProcessingApplication Design, Systems, and Analytics, pp. 148 - 177Publisher: Cambridge University PressPrint publication year: 2014