Book contents
- Frontmatter
- Contents
- Foreword
- Preface
- Part I Density-Ratio Approach to Machine Learning
- Part II Methods of Density-Ratio Estimation
- 2 Density Estimation
- 3 Moment Matching
- 4 Probabilistic Classification
- 5 Density Fitting
- 6 Density-Ratio Fitting
- 7 Unified Framework
- 8 Direct Density-Ratio Estimation with Dimensionality Reduction
- Part III Applications of Density Ratios in Machine Learning
- Part IV Theoretical Analysis of Density-Ratio Estimation
- Part V Conclusions
- List of Symbols and Abbreviations
- Bibliography
- Index
8 - Direct Density-Ratio Estimation with Dimensionality Reduction
from Part II - Methods of Density-Ratio Estimation
Published online by Cambridge University Press: 05 March 2012
- Frontmatter
- Contents
- Foreword
- Preface
- Part I Density-Ratio Approach to Machine Learning
- Part II Methods of Density-Ratio Estimation
- 2 Density Estimation
- 3 Moment Matching
- 4 Probabilistic Classification
- 5 Density Fitting
- 6 Density-Ratio Fitting
- 7 Unified Framework
- 8 Direct Density-Ratio Estimation with Dimensionality Reduction
- Part III Applications of Density Ratios in Machine Learning
- Part IV Theoretical Analysis of Density-Ratio Estimation
- Part V Conclusions
- List of Symbols and Abbreviations
- Bibliography
- Index
Summary
The approaches of direct density-ratio estimation explained in the previous chapters were shown to be promising in experiments with naive kernel density estimation in experiments. However, these methods still perform rather poorly when the dimensionality of the data domain is high.
The purpose of this chapter is to introduce ideas for mitigating this weakness, following Sugiyama et al. (2010a, 2011b). A basic assumption behind the approaches explained here is that the difference between the two distributions in the density ratio (i.e., the distributions corresponding to the numerator and denominator of the density ratio) does not spread over the entire data domain, but is confined in a low-dimensional subspace – which we refer to as the heterodistributional subspace. Once the heterodistributional subspace can be identified, the density ratio is estimated only within this subspace. This will lead to more stable and reliable estimations of density ratios. Such an approach is called direct density-ratio estimation with dimensionality reduction (D3; pronounced “D-cube”).
In this chapter, two approaches to D3 are described. In Section 8.1, a heuristic method based on discriminant analysis is explained. This method is shown to be computationally very efficient, and thus is very practical. On the other hand, in Section 8.2, a more theory-oriented approach based on divergence maximization is introduced. This method is justifiable under general settings, and thus it has a wider applicability. Numerical examples are shown in Section 8.3, and the chapter is concluded in Section 8.4.
- Type
- Chapter
- Information
- Density Ratio Estimation in Machine Learning , pp. 89 - 116Publisher: Cambridge University PressPrint publication year: 2012