Nonparametric estimation

This course presents some of the classical and modern methods for constructing nonparametric density or regression estimators. Both theoretical and practical aspects are covered.

The objective of this course is twofold: on the one hand, it aims to provide an understanding of how estimators are constructed so that they can be adapted to other estimation contexts and to understand the mathematical results that guarantee the validity of these approaches, but also their limitations, particularly in high dimensions. The second objective is to understand the challenges of parameter selection from a practical standpoint through the numerical implementation of algorithms for window selection or model selection. By the end of this course, students should have a toolbox for the practical implementation of these methods.

Undergraduate courses in analysis and probability:




Recommended prerequisites: knowledge of a programming language, HAX710X course in inferential statistics, HAX814X course in linear modeling.

Continuous assessment

• Concepts of risk and error criteria
• Introduction to nonparametric estimation: the empirical distribution function
• Kernel density estimators:
  • Bochner lemma, quadratic risk
  • usual nuclei, higher-order nuclei
  • selection of the optimal window: plug-in method, cross-validation, other adaptive methods
  • convergence speed, comparison with parametric speed
• Density projection estimators:
  • Fourier bases, polynomial bases, Haar wavelets. 
  • minimum contrast estimator, quadratic risk study
  • selection of the dimension of the projection subspace: method by Barron, Birgé, Massart (1999)
  • concept of adaptive estimator (for nested models).
• Nadaraya-Watson estimators of the regression function: quotient approach. 
• Least squares estimators: link to the multivariate linear model.
  • model selection, adaptation.
• Local polynomial regression: practical implementation of splines
• Dimension curse for estimating multivariate density and/or regression functions of several variables. Visualization in 2 dimensions: example of geospatial data. Some approaches for dimensions greater than 1: single-index models, additive models.
• Bootstrap: construction of confidence intervals

Hours:
CM: 21 hours
TD:
TP:
Fieldwork: