Stochastic processes

The first part of this course covers additional topics in probability theory: conditional expectation and Gaussian vectors. The second part introduces one of the main families of discrete-time stochastic processes: Markov chains. These are sequences of dependent random variables, whose dependency relationship is relatively simple since each variable depends only on the previous one. They are also a very powerful modeling tool. We will study the main properties of these processes, as well as their long-run behavior and the estimation of their parameters.

The course objectives are
- to be able to perform calculations involving expected values and conditional distributions
- to be able to model an experiment using a Markov chain
- to be able to calculate the quantities of interest (probability and time to first occurrence of certain events)
- to be able to determine the asymptotic behavior of the process.

Probability course (L3 level): random variables and random vectors, convergence criteria for sequences of random variables, convergence of sequences of independent and identically distributed random variables. (+ characteristic function for Gaussian vectors)
Linear algebra: matrix calculus, eigenvalues, solving linear systems, linear recurrence sequences
 

Recommended prerequisites: Measure Theory

1 Measurability 

1.1 Tribes.

1.2 Random Processes 

1.3 Filtration 

1.4 Downtime 

2 Conditional Expectation 

2.1 Conditional Probability Given an Event 

2.2 Conditional expectation with respect to a tribe.

2.3 Conditional Expectation and Independence.

2.4 Conditional Laws

3 Markov Chains

3.1 Stochastic Matrices 

3.1.1 Definition and Graphical Representation 

3.1.2 Communicating Classes 

3.1.3 Frequency

3.2 Markov Processes

3.2.1 Definition of a Markov chain

3.2.2 Markov Property 

3.3 Transition Issues 

3.4 Classification of Markov Chains

3.4.1 Recurrence and Transience 

3.4.2 Relationship to the Class Structure

3.5 Asymptotic Behavior

3.5.1 Invariant Law 

3.5.2 Convergence to the invariant distribution

3.5.3 Ergodic Theorem 

3.5.4 Statistics of Markov Chains 

Hourly course loads:
Lectures: 21
Tutorials: 21
Lab sessions:
Fieldwork: