Non-parametric estimation

This course presents some of the classical and modern methods for constructing non-parametric density or regression estimators. Both theoretical and practical aspects are covered

The aim of this course is twofold: firstly, to understand the construction of estimators so as to be able to adapt them to other estimation contexts, and to understand the mathematical results that guarantee the soundness of these approaches, but also their limitations, particularly in high dimensions. A second objective is to understand the challenges of parameter selection from a practical point of view, through the numerical implementation of algorithms for window selection or model selection. At the end of this course, students should have a toolbox for practical implementation of these methods.

Undergraduate analysis and probability course,




Recommended prerequisites: knowledge of a programming language, HAX710X inferential statistics course, HAX814X Linear Model course

Full continuous assessment

• Notions of risk and error criteria
• Introduction to non-parametric estimation: the empirical distribution function
• Density kernel estimators :
  • Bochner lemma, quadratic risk
  • common kernels, higher-order kernels
  • optimal window selection: plug-in method, cross-validation, other adaptive methods
  • convergence speed, comparison with parametric speed
• Density projection estimators :
  • Fourier bases, polynomial bases, Haar wavelets. 
  • minimum contrast estimator, quadratic risk study
  • selection of projection subspace dimension: method of Barron, Birgé, Massart (1999)
  • notion of adaptive estimator (for nested models).
• Nadaraya-Watson estimators of the regression function: quotient approach. 
• Least squares estimators: link with the multivariate linear model.
  • model selection, adaptation.
• Local polynomial regression: practical implementation of splines
• The scourge of dimension to estimate a multivariate density and/or a multivariate regression function. Visualization in dimension 2: example of geo-spatialized data. Some ideas for dimensions greater than 1: single-index models, additive models.
• Bootstrap: building confidence intervals

Timetable:
CM: 21h
TD:
TP:
Field :