Introduction to Stochastic Processes
Course facts
- Payment: Free, no certificate
- Credits: Non Credit
- Learning style: Self-paced, start anytime
- Resources: Lecture Notes; Problem Set Solutions; Problem Sets
- Before you start: 18.440 Probability and Random Variables or 6.041SC Probabilistic Systems Analysis and Applied Probability
MIT OpenCourseWare details
- As taught in: Spring 2015
- Department: Mathematics
| Course Details | ||||||
|---|---|---|---|---|---|---|
|
||||||
| Outcomes | ||||||
|
||||||
| Price & Terms | ||||||
|
||||||
| Technical | ||||||
|
||||||
| Specification | ||||||
|
||||||
An introduction to stochastic processes: Markov chains, mixing times, random walks, martingales and Poisson processes, with assignment solutions.
Random systems that evolve step by step can be studied with a small set of ideas, and this class introduces them. It starts with finite Markov chains and how quickly they mix, then moves through random walks on networks, countable state spaces and martingales. The Galton-Watson tree, Poisson processes and continuous time Markov chains close the term, and assignments come with solutions.
What You Will Achieve
- Find stationary distributions and use time-reversal
- Bound mixing times of Markov chains
- Analyze random walks on networks and hitting times
- Apply the optional stopping theorem to martingales
Program Overview
Finite Markov Chains
The first lectures cover chains and mixing.
- Stationary distribution
- Time-reversal
- Lower bounds on mixing times
Random Walks and Countable Chains
The middle lectures extend these ideas.
- Random walk on networks
- Hitting times
- Countable state space chain
Martingales and Continuous Time
The later lectures add new tools.
- Optional stopping theorem
- Galton-Watson tree
- Poisson process
- Infinitesimal generator
Course Completion
Job Outlook
These tools are used in mathematics, statistics, operations research and computer science, where random processes need rigorous analysis.
Good for: Mathematics students who have taken probability at the level of MIT’s 18.440 and know linear algebra.
Keep in mind: Exercises in the first three assignments come from Markov Chains and Mixing Times, which you need. Exams were closed book in the class, but no exam papers are listed here.
Bottom line: A short introduction to Markov chains and martingales, with solved assignments for self-study.
Frequently Asked Questions
Which textbooks are recommended?
Markov Chains and Mixing Times by Levin, Peres and Wilmer, Probability with Martingales by Williams, and Markov Chains by Bremaud.
Are assignment solutions available?
Yes. All five assignments have published solutions, and those for the first three are courtesy of Cheng Mao.
How long does it spend on martingales?
Four lectures, on conditional expectation and the introduction to martingales, the optional stopping theorem, convergence and uniform integrability.
User Reviews
Write your review
You must be logged in to post a review.
Reviews can only be posted by registered learners. Log in or create a free account, then you will return here to post.
There are no reviews yet.