Nonparametric estimation

This course presents some of the classical and modern methods for constructing nonparametric density estimators and regression estimators. It covers both theoretical and practical aspects.

The objective of this course is twofold: first, to understand how estimators are constructed in order to be able to adapt them to other estimation contexts and to understand the mathematical results that ensure the soundness of these approaches, as well as their limitations, particularly in high dimensions. A second objective is to grasp the challenges of parameter selection from a practical standpoint through the numerical implementation of algorithms for window selection or model selection. By the end of this course, students should have a toolkit for the practical implementation of these methods.

Undergraduate courses in analysis and probability:




Recommended prerequisites: familiarity with a programming language; HAX710X (Inferential Statistics); HAX814X (Linear Models)

Comprehensive Continuous Monitoring

• Concepts of Risk and Error Criteria
• Introduction to Nonparametric Estimation: The Empirical Distribution Function
• Kernel density estimators:
  • Bochner lemma, quadratic risk
  • common kernels, higher-order kernels
  • Selection of the optimal window: plug-in method, cross-validation, other adaptive methods
  • Convergence rate, comparison with the parametric rate
• Density projection estimators:
  • Fourier bases, polynomial bases, Haar wavelets. 
  • Minimum-contrast estimator, study of quadratic risk
  • Selection of the dimension of the projection subspace: the method of Barron, Birgé, and Massart (1999)
  • the concept of an adaptive estimator (for nested models).
• Nadaraya-Watson estimators of the regression function: the quotient approach. 
• Least-squares estimators: connection to the multivariate linear model.
  • model selection, customization.
• Local Polynomial Regression: Practical Implementation of Splines
• Dimensionality problem when estimating a multivariate density and/or a regression function with multiple variables. 2-dimensional visualization: example of geospatial data. Some approaches for dimensions greater than 1: single-index models, additive models.
• Bootstrap: Constructing Confidence Intervals

Hours:
Lectures: 9:00 p.m.
Tutorials:
Lab:
Fieldwork: