Skip to main content Accessibility help
×
Hostname: page-component-7479d7b7d-q6k6v Total loading time: 0 Render date: 2024-07-10T20:54:23.962Z Has data issue: false hasContentIssue false

Six - Kullback-Leibler Information and the Maximum Empirical Exponential Likelihood

from Part II - Formulation and Solution of Stochastic Inverse Problems

Published online by Cambridge University Press:  05 June 2012

George G. Judge
Affiliation:
University of California, Berkeley
Ron C. Mittelhammer
Affiliation:
Washington State University
Get access

Summary

Introduction

In this chapter, we continue to build on the extremum (E) estimation and estimating equations (EE) concepts introduced in Chapter 3 and the maximum empirical likelihood (MEL) approach demonstrated in Chapters 4 and 5. Our focus is on methods of estimation and inference involving stochastic inverse problems of the form (Y, X)⇒(η, σ2, ε). In this context, in this chapter we introduce the Kullback-Leibler Information Criterion (KLIC) and the maximum empirical exponential likelihood (MEEL) function and estimator.

Solutions to Systems of Estimating Equations and Kullback-Leibler Information

The procedures examined in the previous three chapters involved solutions relating to systems of estimating equations (EEs). In both the MEL and EE contexts, the EEs were empirical analogs to expectation relations of the type E[h(Y, X, θ)] = 0. Estimators for θ, defined by the solution to moment equations, subsume a large collection of familiar estimation procedures that include linear and nonlinear least squares. The idea of basing an estimator definition on the solution to EEs can be applied quite generally to extremum problems, as demonstrated in Chapter 3.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Brown, L. D 1986 Fundamentals of Statistical Exponential FamiliesHayward, CAInstitute of Mathematical StatisticsGoogle Scholar
Csiszar, I 1991 Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse ProblemsAnnals of Statistics 19 2032CrossRefGoogle Scholar
Gokhale, D. VKullback, S 1978 The Information in Contingency TablesNew YorkMarcel DekkerGoogle Scholar
Golan, AJudge, G. GMiller, D 1996 Maximum Entropy Econometrics: Robust Estimation with Limited DataNew YorkJohn Wiley & SonsGoogle Scholar
Good, I. J 1963 Maximum Entropy for Hypothesis Formulation, Especially for Multidimensional Contingency TablesAnnals of Mathematical Statistics 34 911CrossRefGoogle Scholar
Imbens, G 1997 One-Step Estimators for Over-Identified Generalized Method of Moment ModelsReview of Economic Studies 64 359CrossRefGoogle Scholar
Imbens, GSpady, RJohnson, P 1998 Information Theoretic Approaches to Inference in Moment Condition ModelsEconometrica 66 333CrossRefGoogle Scholar
Jaynes, E. T 1957 Information Theory and Statistical MechanicsPhysics Review 106 620CrossRefGoogle Scholar
Jaynes, E. T 1957 Information Theory and Statistical Mechanics IIPhysics Review 108 171CrossRefGoogle Scholar
Jaynes, E. T 1963 Information Theory and Statistical MechanicsFord, K. WStatistical Physics181New YorkW. A. BenjaminGoogle Scholar
Jaynes, E. T 1984 Prior Information and Ambiguity in Inverse ProblemsMcLaughlin, D. WInverse Problems151Providence, RIAmerican Mathematical SocietyGoogle Scholar
Judge, G. GBock, M. E 1978 The Statistical Implications of Pre-Test and Stein-Rule Estimators in EconometricsAmsterdamNorth-HollandGoogle Scholar
Judge, G. GMittelhammer, R 2004 A Semiparametric Like Basis for Combining Estimators under Quadratic LossJournal of American Statistical Association 99 479CrossRefGoogle Scholar
Judge, G. GMittelhammer, R 2007 Estimation and Inference in Case of Competing Sets of Estimating EquationsJournal of Econometrics 138 513CrossRefGoogle Scholar
Kitamura, YStutzer, M 1997 An Information-Theoretic Alternative to Generalized Method of Moments EstimationEconometrica 65 861CrossRefGoogle Scholar
Kullback, S 1987 Letter to the Editor: The Kullback-Leibler DistanceAmerican Statistician 41 340Google Scholar
Kullback, S 1959 Information Theory and StatisticsNew YorkJohn Wiley & SonsGoogle Scholar
Kullback, SLeibler, R. A 1951 On Information and SufficiencyAnnals of Mathematical Statistics 22 79CrossRefGoogle Scholar
Mittelhammer, RJudge, GMiller, D 2000 Econometric FoundationsNew YorkCambridge University PressGoogle Scholar
Owen, A 1990 Empirical Likelihood Ratio Confidence RegionsAnnals of Statistics 18 90CrossRefGoogle Scholar
Shannon, C. E 1948 A Mathematical Theory of CommunicationBell System Technical Journal 27 379CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×