Stochastic processes

The first part of this course covers additional probability theory topics: conditional expectation, Gaussian vectors. The second part introduces one of the main families of discrete-time stochastic processes: Markov chains. These are sequences of dependent random variables, whose dependency relationship is relatively simple since each variable depends only on the previous one. They are also a very powerful modeling tool. We will study the main properties of these processes, as well as their long-term behavior and the estimation of their parameters.

The objectives of the course are:
- to be able to perform calculations of expected values and conditional laws
- to be able to model an experiment using a Markov chain
- to be able to calculate relevant quantities (probability and time to occurrence of certain events)
- to be able to determine the asymptotic behavior of the process.

Probability course at level L3: random variables and vectors, convergence modes of sequences of random variables, convergence of sequences of independent and identically distributed random variables. (+ characteristic function if Gaussian vectors)
Linear algebra: matrix calculus, eigenvalues, solving linear systems, linear recurrence sequences
 

Recommended prerequisites: Measurement theory

1 Measurability 

1.1 Tribes.

1.2 Random processes 

1.3 Filtration 

1.4 Downtime 

2 Conditional expectation 

2.1 Conditional probability with respect to an event 

2.2 Conditional expectation with respect to a tribe.

2.3 Conditional expectation and independence.

2.4 Conditional laws

3 Markov chains

3.1 Stochastic matrices 

3.1.1 Definition and graphical representation 

3.1.2 Communicating classes 

3.1.3 Frequency

3.2 Markov processes

3.2.1 Definition of a Markov chain

3.2.2 Markov property 

3.3 Passage problems 

3.4 Classification of Markov chains

3.4.1 Recurrence and transience 

3.4.2 Link to the class structure

3.5 Asymptotic behavior

3.5.1 Invariant law 

3.5.2 Convergence toward the invariant law

3.5.3 Ergodic theorem 

3.5.4 Markov chain statistics 

Hours:
CM: 21
TD: 21
TP:
Fieldwork: