Non-parametric estimation

This course presents some of the classical and modern methods for the construction of non-parametric density or regression estimators. Both theoretical and practical aspects are covered

The objective of this course is twofold: on the one hand, to understand the construction of estimators in order to be able to adapt them to other estimation contexts and to understand the mathematical results that guarantee the soundness of these approaches but also their limits, especially in high dimension. A second objective is to understand the issues of parameter selection from a practical point of view through the numerical implementation of algorithms for window selection or model selection. At the end of this course, the student should have a toolbox for the practical implementation of these methods.

Undergraduate analysis and probability course,




Recommended prerequisites: knowledge of a programming language, HAX710X inferential statistics course, HAX814X Linear Model course

Continuous control

• Concepts of risk and error criteria
• Introduction to non-parametric estimation: the empirical distribution function
• Density kernel estimators:
  • Bochner's lemma, quadratic risk
  • usual nuclei, higher order nuclei
  • selection of the optimal window: plug-in method, cross-validation, other adaptive methods
  • convergence speed, comparison with parametric speed
• Density projection estimators:
  • Fourier bases, polynomial bases, Haar wavelets. 
  • minimum contrast estimator, quadratic risk study
  • selection of the dimension of the projection subspace: method of Barron, Birgé, Massart (1999)
  • notion of adaptive estimator (for nested models).
• Nadaraya-Watson estimators of the regression function: quotient approach. 
• Least squares estimators: link with the multivariate linear model.
  • model selection, adaptation.
• Local polynomial regression: practical implementation of splines
• Scourge of dimension to estimate a multivariate density and/or regression function of several variables. Visualization in dimension 2: example of geo-spatialized data. Some tracks for dimension higher than 1: single-index models, additive models.
• Bootstrap: construction of confidence intervals

Hourly volumes:
CM: 21h
TD:
TP:
Field: