RL From Scratch

Must Read

Implementations

• DQNValue-Based

  Deep Q-Network - neural network approximates Q(s,a), stabilised with experience replay and a frozen target network (Mnih et al., 2015).

  DiscreteOff-policyMLP
  →
• Q-learningValue-Based

  Tabular off-policy TD control - a Q-table for small discrete Gym envs like FrozenLake and CliffWalking (Watkins, 1989). The foundation DQN later approximates with a net.

  DiscreteOff-policyTabular
  →

About NeatRL

Citation

@misc{singh2026neatrl,
  title={NeatRL: Readable Single-File Reinforcement Learning Implementations in PyTorch},
  author={Yuvraj Singh},
  year={2026},
  howpublished={\url{https://github.com/YuvrajSingh-mist/NeatRL}},
  doi={10.5281/zenodo.23198154},
}