Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
  1. Home
  2. Browse subjects
  3. Computer Science
  4. Knowledge Management, Databases and Data Mining
  5. Content listing

Refine search

Refine search

Access:
Content type:
Publication date:
Apply   From and To filters
Subject: Show more
Journals Show more
Publishers: Show more
Societies Show more
Series: Show more
Collections: Show more

Actions for selected content:

Select all | Deselect all
  • View selected items
  • Export citations
  • Download PDF (zip)
  • Save to Kindle
  • Save to Dropbox
  • Save to Google Drive

Save Search

You can save your searches here and later view and run them again in "My saved searches".

Please provide a title, maximum of 40 characters.
Cancel
×

1550 results in Knowledge Management, Databases and Data Mining

Page 38 of 78

  • Preface

  • Parteek Bhatia
  • Book:
    Data Mining and Data Warehousing
    Published online:
    26 April 2019
    Print publication:
    16 May 2019, pp xxxi-xxxii

  • Summary

    In the modern age of artificial intelligence and business analytics, data is considered as the oil of this cyber world. The mining of data has huge potential to improve business outcomes, and to carry out the mining of data there is a growing demand for database mining experts. This book intends training learners to fill this gap.

    This book will give learners sufficient information to acquire mastery over the subject. It covers the practical aspects of data mining, data warehousing, and machine learning in a simplified manner without compromising on the details of the subject. The main strength of the book is the illustration of concepts with practical examples so that the learners can grasp the contents easily. Another important feature of the book is illustration of data mining algorithms with practical hands-on sessions on Weka and R language (a major data mining tool and language, respectively). In this book, every concept has been illustrated through a step-by-step approach in tutorial form for self-practice in Weka and R. This textbook includes many pedagogical features such as chapter wise summary, exercises including probable problems, question bank, and relevant references, to provide sound knowledge to learners. It provides the students a platform to obtain expertise on technology, for better placements.

    Video sessions on data mining, machine learning, big data and DBMS are also available on my YouTube channel. Learners are requested to subscribe to this channel https://www.youtube.com/user/parteekbhatia to get the latest updates through video sessions on these topics.

    Your suggestions for further improvements to the book are always welcome. Kindly e-mail your suggestions to parteek.bhatia@gmail.com.

    I hope you enjoy learning from this book as much as I enjoyed writing it.

  • 7 - Cluster Analysis

  • Parteek Bhatia
  • Book:
    Data Mining and Data Warehousing
    Published online:
    26 April 2019
    Print publication:
    16 May 2019, pp 155-205

  • Summary

    Chapter Objectives

    ✓ To comprehend the concept of clustering, its applications, and features.

    ✓ To understand various distance metrics for clustering of data.

    ✓ To comprehend the process of K-means clustering.

    ✓ To comprehend the process of hierarchical clustering algorithms.

    ✓ To comprehend the process of DBSCAN algorithms.

    Introduction to Cluster Analysis

    Generally, in the case of large datasets, data is not labeled because labeling a large number of records requires a great deal of human effort. The unlabeled data can be analyzed with the help of clustering techniques. Clustering is an unsupervised learning technique which does not require a labeled dataset.

    Clustering is defined as grouping a set of similar objects into classes or clusters. In other words, during cluster analysis, the data is grouped into classes or clusters, so that records within a cluster (intra-cluster) have high similarity with one another but have high dissimilarities in comparison to objects in other clusters (inter-cluster), as shown in Figure 7.1.

    The similarity of records is identified on the basis of values of attributes describing the objects. Cluster analysis is an important human activity. The first human beings Adam and Eve actually learned through the process of clustering. They did not know the name of any object, they simply observed each and every object. Based on the similarity of their properties, they identified these objects in groups or clusters. For example, one group or cluster was named as trees, another as fruits and so on. They further classified the fruits on the basis of their properties like size, colour, shape, taste, and others. After that, people assigned labels or names to these objects calling them mango, banana, orange, and so on. And finally, all objects were labeled. Thus, we can say that the first human beings used clustering for their learning and they made clusters or groups of physical objects based on the similarity of their attributes.

    Applications of Cluster Analysis

    Cluster analysis has been widely used in various important applications such as:

  • • Marketing: It helps marketers find out distinctive groups among their customer bases, and this knowledge helps them improve their targeted marketing programs.

  • • Land use: Clustering is used for identifying areas of similar land use from the databases of earth observations.

  • • Insurance: Clustering is helpful for recognizing clusters of insurance policyholders with a high regular claim cost.

  • 6 - Implementing Classification in Weka and R

  • Parteek Bhatia
  • Book:
    Data Mining and Data Warehousing
    Published online:
    26 April 2019
    Print publication:
    16 May 2019, pp 128-154

  • Summary

    Chapter Objectives

    ✓ To demonstrate the use of the decision tree

    ✓ To apply the decision tree on a sample dataset

    ✓ To implement a decision tree process using Weka and R

    Building a Decision Tree Classifier in Weka

    In this chapter, we will learn how Weka's decision tree feature helps to classify unknown samples of a dataset based on its attribute values. When Weka's decision tree is applied to an unknown sample, the decision tree classifies the sample into different classes such as Class A, Class B and Class C as shown in Figure 6.1.

    For example, if we want to predict the class of an unknown sample of a flower based on the length and width dimensions of its Sepal and Petal. The first step would be to measure Sepal length and width and Petal length and width of an unknown flower and compare these dimensions to the values of the samples in our dataset of known species. The decision tree algorithm of Weka will help in creating decision rules to predict the class of unknown flower automatically as shown in Figure 6.2.

    As shown in Figure 6.2, the dimensions of an unknown sample of flower will be matched with the rules generated by the decision tree. First, the rules will be matched to determine whether the sample belongs to Setosa class or not, if yes, the unknown sample will be classified as setosa. If not, the unknown sample will be checked for being of the Virginica class. If it matches with the conditions of the Virginica class, it will be labeled as Virginica, otherwise Versicolor. It is important to note that it would not be simple to create these rules on the basis of the values of single attribute as shown in Table 6.1. It is clear that for the same Sepal width, the flower may be of Setosa or Versicolor or Virginica, making it unclear which species an unknown flower belongs to on the basis of Sepal width alone. Thus, the decision tree must make its prediction based on all four flower dimensions.

    Due to such overlaps, the decision tree cannot predict with 100% accuracy the class of flower, but can only determine the likelihood of an unknown sample belonging to a particular class. In real situations the decision tree algorithm works on the basis of probability.

  • 2 - Introduction to Data Mining

  • Parteek Bhatia
  • Book:
    Data Mining and Data Warehousing
    Published online:
    26 April 2019
    Print publication:
    16 May 2019, pp 17-27

  • Summary

    Chapter Objectives

    ✓ To learn about the concepts of data mining.

    ✓ To understand the need for, and the applications of data mining

    ✓ To differentiate between data mining and machine learning

    ✓ To understand the process of data mining.

    ✓ To understand the difference between data mining and machine learning.

    Introduction to Data Mining

    In the age of information, an enormous amount of data is available in different industries and organizations. The availability of this massive data is of no use unless it is transformed into valuable information. Otherwise, we are sinking in data, but starving for knowledge. The solution to this problem is data mining which is the extraction of useful information from the huge amount of data that is available.

    Data mining is defined as follows:

    ‘Data mining is a collection of techniques for efficient automated discovery of previously unknown, valid, novel, useful and understandable patterns in large databases. The patterns must be actionable so they may be used in an enterprise's decision making.’

    From this definition, the important take aways are:

  • • Data mining is a process of automated discovery of previously unknown patterns in large volumes of data.

  • • This large volume of data is usually the historical data of an organization known as the data warehouse.

  • • Data mining deals with large volumes of data, in Gigabytes or Terabytes of data and sometimes as much as Zetabytes of data (in case of big data).

  • • Patterns must be valid, novel, useful and understandable.

  • • Data mining allows businesses to determine historical patterns to predict future behaviour.

  • • Although data mining is possible with smaller amounts of data, the bigger the data the better the accuracy in prediction.

  • • There is considerable hype about data mining at present, and the Gartner Group has listed data mining as one of the top ten technologies to watch.

    • Need of Data Mining

    Data mining is a recent buzz word in the field of Computer Science. It is a computing process that uses intelligent mathematical algorithms to extract the relevant data and computes the probability of future actions. It is also known as Knowledge Discovery in Data (KDD).

  • 13 - Data Warehouse Schema

  • Parteek Bhatia
  • Book:
    Data Mining and Data Warehousing
    Published online:
    26 April 2019
    Print publication:
    16 May 2019, pp 405-415

  • Summary

    Chapter Objectives

    ✓ To understand the concept of dimension, measure and the fact table

    ✓ To able to apply different schema of data warehouse designs such as Star Schema, Snowflake Schema and Fact Constellation Schema on real world applications.

    ✓ To understand the differences between these schemas, their strengths and weakness.

    Introduction to Data Warehouse Schema

    Logical descriptions of database are known as Schema. It is the blueprint of the entire database. It defines how the data are organized and how the relations among them are associated. Data warehouse schema consists of the name and description of records including associated data items and aggregates. A database uses relational models whereas a data warehouse uses different types of schema, namely, Star, Snowflake, and Fact Constellation.

    To start discussion on these schemas, it is important to understand the basic terminology used in this process, which is discussed below.

    Dimension

    The term ‘dimension’ in data warehousing is a collection of reference information about a measurable event. These events are stored in a fact table and are known as facts. The dimensions are generally the entities for which an organization wants to preserve records. The descriptive attributes are organized as columns in dimension tables by a data warehouse. For example, a student's dimension attributes could consist of first and last name, roll number, age, gender, or an address dimension that would include street name, state, and country attributes.

    A dimension table consists of a primary key column that uniquely identifies each record (row) of dimension. A dimension is a framework that consists of one or more hierarchies that classify data. Usually dimensions are de-normalized tables and may have redundant data.

    Let us take a quick recap of the concepts of normalization and de-normalization, as they will be used in this chapter. Normalization is a process of breaking up a larger table into smaller tables free of any possible insertion, updation or deletion anomalies. Normalized tables have reduced redundancy of data. In order to get full information, these tables are usually joined.

    In de-normalization, smaller tables are merged to form larger tables to reduce joining operations. De-normalization is particularly performed in those cases where retrieval is a major requirement and insert, update, and delete operations are minimal, as in case of historical data or data warehouse. These de-normalized tables will have redundancy of data.

  • 1 - Beginning with Machine Learning

  • Parteek Bhatia
  • Book:
    Data Mining and Data Warehousing
    Published online:
    26 April 2019
    Print publication:
    16 May 2019, pp 1-16

  • Summary

    Chapter Objectives

    ✓ To understand the concept of machine learning and its applications.

    ✓ To understand what are supervised and unsupervised machine learning strategies.

    ✓ To understand the concept of regression and classification.

    ✓ To identify the strategy to be applied to a given problem.

    Introduction to Machine Learning

    Machine Learning (ML) has emerged as the most extensively used tool for web-sites to classify surfers and address them appropriately. When we surf the Net, we are exposed to machine learning algorithms multiple times a day, often without realizing it. Machine learning is used by search engines such as Google and Bing to rank web pages or to decide which advertisement to show to which user. It is used by social networks such as Facebook and Instagram to generate a custom feed for every user or to tag the user by the picture that was uploaded. It is also used by banks to detect whether an online transaction is genuine or fraudulent and by e-commerce websites such as Amazon and Flipkart to recommend products that we are most likely to buy. Even email providers such as Gmail, Yahoo, and Hotmail use machine learning to decide which emails are spam and which are not. These are only a few examples of applications of machine learning.

    The ultimate aim of machine learning is to build an Artificial Intelligence (AI) platform that is as intelligent as the human mind. We are not very far from this dream and many AI researchers believe that this goal can be achieved through machine learning algorithms that try to mimic the learning processes of a human brain.

    Actually, ML is a branch of AI. Many years ago researchers tried to build intelligent programs with pre-defined rules like in the case of a normal program. But this approach did not work as there were too many special cases to be considered. For instance, we can define rules to find the shortest path between two points. But it is very difficult to make rules for programs such as photo tagging, classifying emails as spam or not spam, and web page ranking. The only solution to accomplish these tasks was to write a program that could generate its own rules by examining some examples (also called training data). This approach was named Machine Learning. This book will cover state of art machine learning algorithms and their deployment.

  • 3 - Beginning with Weka and R Language

  • Parteek Bhatia
  • Book:
    Data Mining and Data Warehousing
    Published online:
    26 April 2019
    Print publication:
    16 May 2019, pp 28-54

  • Summary

    Chapter Objectives

    ✓ To learn to install Weka and the R language

    ✓ To demonstrate the use of Weka software

    ✓ To experiment with Weka on the Iris dataset

    ✓ To introduce basics of R language

    ✓ To experiment with R on the Iris dataset

    About Weka

    In this book, all data mining algorithms are explained with Weka and R language. The learner can perform and apply these algorithms easily using these well-know data mining tool and language. Let's first discuss the Weka tool.

    Weka is an open-source software under the GNU General Public License System. It was developed by the Machine Learning Group, University of Waikato, New Zealand. Although named after a flightless New Zealand bird, ‘WEKA’ stands for Waikato Environment for Knowledge Analysis. The system is written using the object oriented language Java. Weka is data mining software and it is a set of machine learning algorithms that can be applied to a dataset directly, or called from your own Java code. Weka contains tools for data pre-processing, classification, regression, clustering, association rules, and visualization.

    The story of the development of Weka is very interesting. It was initially developed by students of University of Waikato, New Zealand, as part of their course work on data mining. They had implemented all major machine learning algorithms as part of lab work for this course. In 1993, the University of Waikato began development of the original version of Weka, which became a mix of Tcl/Tk, C, and Makefiles. In 1997, the decision was made to redevelop Weka from scratch in Java, including implementations of modeling algorithms. In 2006, Pentaho Corporation acquired an exclusive license to use Weka for business intelligence.

    This chapter will cover the installation of Weka, datasets available and will guide the learner about how to start experimentation using Weka. Later on we will discuss another data mining tool, R. Let us first discuss the installation process for Weka, step-by-step.

    Installing Weka

    Weka is freely available and its latest version can be easily downloaded from https://www.cs.waikato. ac.nz/ml/weka/downloading.html as shown in Figure 3.1.

    To work more smoothly, you must first download and install Java VM before downloading Weka.

Data Mining and Data Warehousing
  • Principles and Practical Techniques
  • Parteek Bhatia
  • Published online:
    26 April 2019
    Print publication:
    16 May 2019
  • Written in lucid language, this valuable textbook brings together fundamental concepts of data mining and data warehousing in a single volume. Important topics including information theory, decision tree, Naïve Bayes classifier, distance metrics, partitioning clustering, associate mining, data marts and operational data store are discussed comprehensively. The textbook is written to cater to the needs of undergraduate students of computer science, engineering and information technology for a course on data mining and data warehousing. The text simplifies the understanding of the concepts through exercises and practical examples. Chapters such as classification, associate mining and cluster analysis are discussed in detail with their practical implementation using Weka and R language data mining tools. Advanced topics including big data analytics, relational data models and NoSQL are discussed in detail. Pedagogical features including unsolved problems and multiple-choice questions are interspersed throughout the book for better understanding.

  • 2 - Architecture and Categorization of DBMSs

  • from Part I - Databases and Database Design
  • Wilfried Lemahieu, KU Leuven, Belgium, Seppe vanden Broucke, KU Leuven, Belgium, Bart Baesens, KU Leuven, Belgium
  • Book:
    Principles of Database Management
    Published online:
    17 August 2019
    Print publication:
    12 July 2018, pp 20-37

  • Summary

    Chapter Objectives

    In this chapter, you will learn to:

  • • identify the key components of a DBMS architecture;

  • • understand how these components work together for data storage, processing, and management;

  • • categorize DBMSs based upon data model, degree of simultaneous access, architecture, and usage.

    • Opening Scenario

    To kick-start its business, Sober purchased the customer database of Mellow Cab, a firm that recently stepped out of the taxi business. Unfortunately, the database has been handed over in a legacy CODASYL format that Sober is not familiar with. Sober also needs a new database to store transaction details whenever passengers book either a ride-hailing or ride-sharing service. Other data (e.g., multimedia) are an option they are interested in. Sober wants to continuously store the location of its taxis and periodically review hot-spot pick-up and drop-off locations. Sober is looking at ways to manage all these data sources in the optimal way.

    As discussed in Chapter 1, a DBMS supports the creation, usage, and maintenance of a database. It consists of several modules, each with their specific functionality, that work together according to a predefined architecture. In this chapter, we zoom into this internal architecture and provide a categorization of DBMSs along various dimensions. The overview of the chapter is straightforward. We start by discussing the components that together make up a DBMS. Next, we provide a classification of DBMSs in terms of data model, degree of simultaneous access, architecture, and usage.

    Architecture of a DBMS

    As discussed before, a DBMS needs to support various types of data management-related activities, such as querying and storage. It also must provide interfaces to its environment. To achieve both of these goals, a DBMS is composed of various interacting modules that together make up the database management system architecture. Figure 2.1 shows an overview of the key components of a DBMS architecture. We review each component in more detail in what follows.

    Figure 2.1 is by no means exhaustive. Depending upon the vendor and implementation, some components may be left out and others added. On the left, you can see various ways of interacting with the DBMS. DDL statements create data definitions that are stored in the catalog. Interactivequeries are typically executed from a front-end tool, such as a command-line interface, simple graphical user interface, or forms-based interface. Applications interact with the DBMS using embedded DML statements.

  • 4 - Organizational Aspects of Data Management

  • from Part I - Databases and Database Design
  • Wilfried Lemahieu, KU Leuven, Belgium, Seppe vanden Broucke, KU Leuven, Belgium, Bart Baesens, KU Leuven, Belgium
  • Book:
    Principles of Database Management
    Published online:
    17 August 2019
    Print publication:
    12 July 2018, pp 79-90

  • Summary

    Chapter Objectives

    In this chapter, you will learn to:

  • • identify the basic concepts of data management;

  • • understand the role and importance of catalogs, metadata, data quality, and data governance;

  • • identify key roles in database modeling and management;

  • • understand the differences between an information architect, database designer, data owner, data steward, database administrator, and data scientist.

    • Opening Scenario

    Sober realizes that the success of its entire business model depends on data, and it wants to make sure the data are managed in an optimal way. The company is looking at how to organize proper data management and wondering about the corresponding job profiles to hire. The challenge is twofold. On the one hand, Sober wants to have the right data management team to ensure optimal data quality. On the other hand, Sober only has a limited budget to build that team.

    In this chapter, we zoom into the organizational aspects of data management. We begin by elaborating on data management and review the essential role of catalogs and metadata, both of which were introduced in Chapter 1. We then discuss metadata modeling, which essentially follows a similar database design process to that which we described in Chapter 3. Data quality is also extensively covered in terms of both its importance and its underlying dimensions. Next, we introduce data governance as a corporate culture to safeguard data quality. We conclude by reviewing various roles in data modeling and management, such as information architect, database designer, data owner, data steward, database administrator, and data scientist.

    Data Management

    Data management entails the proper management of data and the corresponding data definitions or metadata. It aims at ensuring that (meta-)data are of good quality and thus a key resource for effective and efficient managerial decision-making. In the following subsections we first review catalogs, the role of metadata, and the modeling thereof. This is followed by a discussion on data quality and data governance.

    Catalogs and the Role of Metadata

    The importance of good metadata management cannot be understated. In the past this was often neglected, resulting in significant problems when applications needed to be updated or maintained. In the file-based approach to data management, the metadata were stored in each application separately, creating the issues discussed in Chapter 1.

  • 1 - Fundamental Concepts of Database Management

  • from Part I - Databases and Database Design
  • Wilfried Lemahieu, KU Leuven, Belgium, Seppe vanden Broucke, KU Leuven, Belgium, Bart Baesens, KU Leuven, Belgium
  • Book:
    Principles of Database Management
    Published online:
    17 August 2019
    Print publication:
    12 July 2018, pp 3-19

  • Summary

    Chapter Objectives

    In this chapter, you will learn to:

  • • understand the differences between the file versus database approach to data management;

  • • discern the key elements of a database system;

  • • identify the advantages of database systems and database management.

    • Opening Scenario

    Since Sober is a startup company, it must carefully decide how it will manage all its data. The company is thinking about storing all its data in Word documents, Excel files, and maybe some other files (e.g., Notepad) as well.

    In this chapter, we discuss the fundamental concepts of database management. Many ideas presented here are elaborated in later chapters. We kick off by reviewing popular applications of database technology, and follow this by defining key concepts such as a database and a database management system, or DBMS. Next, we step back in time and discuss the file-based approach and contrast it with the database approach to data management. We then zoom into the elements of a database system. We conclude by discussing the advantages of database design.

    Applications of Database Technology

    Data are everywhere and come in different shapes and volumes. These data need to be stored and managed using appropriate data management or database technologies. Think about the storage and retrieval of traditional numeric and alphanumeric data in an application developed to keep track of the number of products in stock. For each product, the product number, product name, and available quantity needs to be stored. Replenishment orders need to be issued as soon as the quantity drops below the safety limit. Every replenishment order has an order number, order date, supplier number, supplier name, and a set of product numbers, names, and quantities.

    Database technology is not just for traditional numeric and alphanumeric data. It can also store multimedia data such as pictures, audio, or video – YouTube and Spotify support the querying of music based upon artist, album, genre, playlist, or record label. Biometric data, including fingerprints and retina scans, are often used for security, such as border control as you enter a country. Information is also gathered by wearables, such as a Fitbit or an Apple Watch, which continuously monitor and analyze your health and fitness. Geographical information systems (GIS) applications, such as Google Maps, store and retrieve all types of spatial or geographical data.

  • 11 - NoSQL Databases

  • from Part II - Types of Database Systems
  • Wilfried Lemahieu, KU Leuven, Belgium, Seppe vanden Broucke, KU Leuven, Belgium, Bart Baesens, KU Leuven, Belgium
  • Book:
    Principles of Database Management
    Published online:
    17 August 2019
    Print publication:
    12 July 2018, pp 300-348

  • Summary

    Chapter Objectives

    In this chapter, you will learn:

  • • what is meant by “NoSQL” and how this movement differs from earlier approaches;

  • • to understand the differences between key–value-, tuple-, document-, column-, and graph-based databases;

  • • about the defining characteristics of NoSQL databases, such as their capability to scale horizontally, their approach toward data replication and how this relates to eventual consistency, their APIs, and how they are queried and interacted with;

  • • how to query NoSQL databases, using MapReduce, Cypher, and other approaches.

    • Sober has been happily using its relational database system for quite a while now, but is experiencing some limits of its existing system. In particular, due to the growth of its mobile applications (now serving many users simultaneously), some queries are performing slower, causing users to wait a few seconds when working with the app. Sober's database administrators have identified that this is mainly due to the normalized approach of its relational database model, causing many queries to involve joining tables, which results in bottlenecks when thousands of users are hailing cabs simultaneously.

    Another issue is that the mobile developers regularly want to experiment with new features (to do so, a “beta” version of the app is rolled out to a subset of users), though given the emphasis on data consistency put forward by RDBMSs, changing existing data schemas so they can handle new data fields is a time-intensive effort. The app developers are not very familiar with this, causing back-and-forth between the development team and the database administrators.

    Given these new issues, Sober is thinking of different scenarios to handle its growth. One option would be to roll out additional servers to handle the increased usage, but keep using the RDBMS system to do so. Another possibility would be to explore a NoSQL database solution. Sober's database administrator has heard about newer approaches such as MongoDB, Cassandra, and others, which offer strong scaling opportunities and a schema-free approach to data management.

    Relational database management systems (RDBMSs) put a lot of emphasis on keeping data consistent. They require a formal database schema, and new data or modifications to existing data are not accepted unless they comply with this schema in terms of data type, referential integrity, etc. Sometimes this focus on consistency may become a burden, because it induces (in some cases unnecessarily) overhead and hampers scalability and flexibility.

  • 16 - Data Distribution and Distributed Transaction Management

  • from Part III - Physical Data Storage, Transaction Management, and Database Access
  • Wilfried Lemahieu, KU Leuven, Belgium, Seppe vanden Broucke, KU Leuven, Belgium, Bart Baesens, KU Leuven, Belgium
  • Book:
    Principles of Database Management
    Published online:
    17 August 2019
    Print publication:
    12 July 2018, pp 516-548

  • Summary

    Chapter Objectives

    In this chapter, you will learn to:

  • • grasp the basics of distributed systems and distributed databases;

  • • discern key architectural implications of distributed databases;

  • • understand the impact of fragmentation, allocation, and replication;

  • • identify different types of transparency;

  • • understand the steps in distributed query processing;

  • • understand distributed transaction management and concurrency control;

  • • grasp the impact of eventual consistency and BASE transactions.

    • Opening Scenario

    As Sober envisions growing as part of its long-term strategy, it wants to have a careful understanding of the data implications involved. More specifically, the company wants to know if it would make sense to distribute its data across a network of offices and work with a distributed database. Sober wants to know the impact of data distribution on query processing and optimization, transaction management, and concurrency control.

    In this chapter, we focus on the specifics of distributed databases (i.e., systems in which the data and DBMS functionality are distributed over different nodes or locations on a network). First, we discuss the general properties of distributed systems and offer an overview of some architectural variants of distributed database systems. Then, we tackle the different ways of distributing data over nodes in a network, including the possibility of data replication. We also focus on the degree to which the data distribution can be made transparent to applications and users. Then, we discuss the complexity of query processing and query optimization in a distributed setting. A next section is dedicated to distributed transaction management and concurrency control, focusing on both tightly coupled and loosely coupled settings. The last section overviews the particularities of transaction management in Big Data and NoSQL databases, which are often distributed in a cluster set-up, presenting BASE transactions as an alternative to the traditional ACID transaction paradigms.

    Distributed Systems and Distributed Databases

    Ever since the early days of computing, which were dominated by monolithic mainframes, distributed systems have had their place in the ICT landscape. A distributed computing system consists of several processing units or nodes with a certain level of autonomy, which are interconnected by a network and which cooperatively perform complex tasks. These complex tasks can be divided into subtasks as performed by the individual nodes.

  • 20 - Analytics

  • from Part IV - Data Warehousing, Data Governance, and (Big) Data Analytics
  • Wilfried Lemahieu, KU Leuven, Belgium, Seppe vanden Broucke, KU Leuven, Belgium, Bart Baesens, KU Leuven, Belgium
  • Book:
    Principles of Database Management
    Published online:
    17 August 2019
    Print publication:
    12 July 2018, pp 664-730

  • Summary

    Chapter Objectives

    In this chapter, you will learn to:

  • • understand the key steps of the analytics process model;

  • • identify the skill set of a data scientist;

  • • preprocess data for analytics using denormalization, sampling, exploratory data analysis, and dealing with missing values and outliers;

  • • build predictive analytical models using linear regression, logistic regression, and decision trees;

  • • evaluate predictive analytical models by splitting up the dataset and using various performance metrics;

  • • build descriptive analytical models using association rules, sequence rules, and clustering;

  • • understand the basic concepts of social network analytics;

  • • discern the key activities during post-processing of analytical models;

  • • identify the critical success factors of analytical models;

  • • understand the economic perspective on analytics by considering the total cost of ownership (TCO) and return on investment (ROI) and how they are affected by in- versus outsourcing, on-premise versus cloud solutions, and open versus commercial software;

  • • improve the ROI of analytics by exploring new sources of data, increasing data quality, securing management support, optimizing organizational aspects, and fostering cross-fertilization;

  • • understand the impact of privacy and security in a data storage, processing, and analytics context.

    • Opening Scenario

    Now that Sober has made its first steps in business intelligence, it is eager to take this to the next level and explore what it could do with analytics. The company has witnessed extensive press and media coverage on predictive and descriptive analytics and wonders what these technologies entail and how they could be used to its advantage. It is actually thinking about analyzing its booking behavior, but is unsure how to tackle this. Given that Sober is a startup, it also wants to know the economic and privacy implications of leveraging these technologies.

    In this chapter, we extensively zoom into analytics. We kick-off by providing a bird's eye overview of the analytics process model. We then give examples of analytics applications and discuss the data scientist job profile. We briefly zoom into data pre-processing. The next section elaborates on different types of analytics: predictive analytics, descriptive analytics, and social network analytics. We also discuss the post-processing of analytical models. Various critical success factors for analytical models are clarified in the following section. This is followed by a discussion on the economic perspective of analytics.

  • 13 - Physical Database Organization

  • from Part III - Physical Data Storage, Transaction Management, and Database Access
  • Wilfried Lemahieu, KU Leuven, Belgium, Seppe vanden Broucke, KU Leuven, Belgium, Bart Baesens, KU Leuven, Belgium
  • Book:
    Principles of Database Management
    Published online:
    17 August 2019
    Print publication:
    12 July 2018, pp 395-429

  • Summary

    Chapter Objectives

    In this chapter, you will learn to:

  • • grasp the basic concepts of physical database organization;

  • • identify various database access methods;

  • • understand how individual devices can be pooled and managed as so-called enterprise storage subsystems;

  • • understand the importance of business continuity.

    • Opening Scenario

    Now that Sober knows what indexes it needs, it wants to understand how these indexes can be implemented in SQL. The company is also curious to find out how the query optimizer works and how it decides on the access path to the data for a given query. Sober wonders what type of storage hardware it should adopt. Finally, the company wants to figure out how it can guarantee its uninterrupted functioning, despite possible planned or unplanned downtime of the hard- and software.

    This chapter deals with physical database organization in its broadest sense. A primary focus is on physical database design – the translation of a logical data model into an internal data model, including the design of indexes to speed up data access. In this respect, this chapter builds on the insights gained from Chapter 12 in relation to physical record and file organization. However, we also zoom out to other concerns of physical data storage, such as technological and management aspects, as well as the important area of business continuity.

    First, we discuss the concepts of tablespaces and index spaces, as well as other basic building blocks of physical database organization. The principles of file organization acquired in the previous chapter are then applied in the context of database organization. Here, we deal with the role of the query optimizer and how the latter uses indexes and other techniques to determine the optimal access plan to execute a query. We also pay particular attention to different alternatives for join implementation. In the second half of this chapter, we return to storage hardware, but now from a more global perspective than in Chapter 12. We deal with RAID technology and different approaches to networked storage, such as SAN and NAS. We conclude with a discussion on how these techniques can be applied in the context of business continuity. By no means do we claim exhaustiveness with respect to the topics covered in this chapter. Rather, we aim to provide the reader with a broad perspective on the many angles and concerns of physical data storage and management.

Page 38 of 78