https://yeongseung.github.io/tags/cartpole/Recent content in CartPole on Yeongseung's BlogHugo -- 0.160.1enTue, 20 Jan 2026 16:14:29 +0900https://yeongseung.github.io/posts/mastering_cartpole/post/Tue, 20 Jan 2026 16:14:29 +0900https://yeongseung.github.io/posts/mastering_cartpole/post/<h1 id="introduction">Introduction</h1> <p>If you are interested in reinforcement learning, you have probably heard of <a href="https://gymnasium.farama.org/environments/classic_control/cart_pole/">CartPole</a> <a href="#ref-6">[6]</a>, often considered the &ldquo;Hello World&rdquo; environment of the field. This is partly because the task is simple to understand, and partly because each episode ends quickly, allowing you to observe learning progress within a short time.</p> <figure class="figure-center"> <img src="https://yeongseung.github.io/posts/Mastering_CartPole/cartpole.png" width="300"> <figcaption>Figure 1. CartPole environment visualization</figcaption> </figure> <p>Looking back, I have run many experiments in CartPole before, but not in a structured or systematic way, which caused me to forget most of the details over time. In this post, I aim to consolidate those trials into a coherent narrative, focusing in particular on why Q-learning can become unstable and how techniques such as experience replay, Double DQN, and failure-focused sampling can improve robustness.</p>