Reinforcement Learning with Gymnasium in Python
Code reinforcement learning agents in Python with Gymnasium, using policy iteration, Monte Carlo, SARSA and Q-learning on Frozen Lake and similar tasks.
Reinforcement learning trains an agent to pick the actions that collect the most reward as it interacts with an environment. The lessons turn that loop into working code: defining states, actions and rewards, estimating how good a policy is, and improving it, both when the rules of the environment are known and when they have to be learned through experience. You write these algorithms in Python against Gymnasium environments such as Frozen Lake, Mountain Car and CliffWalking.
What You Will Achieve
- Create Gymnasium environments, visualize their states and step an agent through actions in code.
- Compute discounted returns, state-values and Q-values, and use them to compare and improve policies.
- Implement policy iteration, value iteration, Monte Carlo prediction, SARSA and Q-learning in Python.
- Choose actions with epsilon-greedy and decayed epsilon-greedy strategies, and apply the Expected SARSA and Double Q-learning update rules.
Program Overview
Reinforcement Learning Framework and Gymnasium
Sets out the agent-environment loop and how returns are discounted, then puts the ideas to work in Gymnasium environments.
- How RL differs from other machine learning areas and where it applies
- The RL interaction loop with episodic and continuous tasks
- Discounted returns for agent strategies
- Mountain Car and Frozen Lake environments
Model-Based Learning with Markov Decision Processes
Builds a custom Frozen Lake MDP and uses its known transition probabilities and rewards to evaluate, compare and improve policies.
- State and action spaces, transition probabilities and rewards
- Deterministic policies and state-value functions
- Action-value functions and policy improvement
- Policy iteration and value iteration
Model-Free Learning and Advanced Strategies
Trains agents through experience on 8×8 and slippery Frozen Lake, then refines the updates and tunes exploration on CliffWalking and multi-armed bandits.
- First-visit and every-visit Monte Carlo
- Temporal difference learning with SARSA and Q-learning
- Expected SARSA and Double Q-learning
- Exploration-exploitation tradeoff and multi-armed bandits
Certificate Awarded (Paid)
Job Outlook
These skills apply to machine learning work where an agent has to learn a sequence of decisions through rewards, including Python projects that train agents in simulated environments and decision problems under uncertainty.
Good for: Python programmers who want to code classic reinforcement learning algorithms themselves and test each update rule on Gymnasium environments such as Frozen Lake.
Keep in mind: Mountain Car is only set up and visualized; the agents are trained on Frozen Lake, CliffWalking and multi-armed bandit problems.
Bottom line: A code-first route through MDPs, Monte Carlo, temporal difference methods and Q-learning variants, with every update rule written and checked in Python.
Frequently Asked Questions
Which Gymnasium environments does the course use?
You set up and visualize Mountain Car, then spend most of the practice on Frozen Lake, including a custom version for the MDP lessons, the 8x8 map and a slippery variant. CliffWalking is used for the exploration strategies, and you build a multi-armed bandit of your own.
How do the model-based and model-free parts differ?
In the model-based lessons the transition probabilities and rewards of the MDP are known, so policy iteration and value iteration can compute a better policy directly. The model-free methods, Monte Carlo, SARSA and Q-learning, estimate values from episodes the agent generates as it acts in the environment.
What do Expected SARSA and Double Q-learning add?
They come in the final chapter as refinements of the earlier updates. Expected SARSA is presented as a way to make policy updates more accurate, and Double Q-learning as a way to reduce the overestimation bias of standard Q-learning. You write each update rule and apply it to an environment.
What happens in the multi-armed bandit lessons?
The closing lessons treat the bandit as a decision problem under uncertainty. You create a multi-armed bandit, solve it with the exploration strategies covered earlier in the chapter, and assess how quickly the agent's choices converge.


There are no reviews yet.