Week 5!
Hello everyone, and welcome back to Week 5 of my Senior Research Project on Reinforcement Learning!
This week, I spent more time exploring complex variations of the CartPole problem, pushing beyond the basic reinforcement learning models I used before. I started researching Deep Q-Learning (DQL) and how neural networks play a role in improving decision-making. Reading through lecture notes from CSC 480/580 and CSC 580 FA22 helped me understand how Q-functions are approximated using deep learning, and how experience replay helps stabilize training. It was fascinating to see how these ideas expand on traditional Q-learning, making it more scalable and efficient for complex environments.
I also spent time learning about neural networks in reinforcement learning. While I’ve encountered neural networks before, it was interesting to see them applied in this context, where they help approximate action values instead of just making predictions like in traditional supervised learning. Understanding how gradient descent updates Q-values and how function approximation works gave me a much better intuition for how deep reinforcement learning actually operates.
Thank you so much for reading through my Week 5 update. If you have any questions, please don’t hesitate to ask.
Until next time, I will see you later!