Reinforcement Learning is a study of an agent learning through interactions with the environment. Thus, to test and compare results of different reinforcement learning algorithms, we need testbed environments. By far the most commonly used testbed has been 57 Atari 2600 games. However, different environments require different exploration schemes and different algorithms. Thus, it is important to have various environments.

We attempt to provide a high-quality guide for Reinforcement Learning environments.

OpenAI Gym

OpenAI Gym is a toolkit for developing and comparing reinforcement learning algorithms. The gym library is a collection of environments that makes no assumptions about the structure of your agent. Gym comes with a diverse suite of environments, ranging from classic video games such as Atari 2600 and continuous control tasks.

Read more

Unity ML-Agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source Unity plugin that enables games and simulations to serve as environments for training intelligent agents.

Read more


PySC2 is DeepMind’s Python component of the StarCraft II Learning Environment (SC2LE). It exposes Blizzard Entertainment’s StarCraft II Machine Learning API as a Python RL Environment. This is a collaboration between DeepMind and Blizzard to develop StarCraft II into a rich environment for RL research. PySC2 provides an interface for RL agents to interact with StarCraft 2, getting observations and sending actions.

Read more

PyGame Learning Environment

PyGame Learning Environment (PLE) is a learning environment, mimicking the Arcade Learning Environment interface, allowing a quick start to Reinforcement Learning in Python. The goal of PLE is allow practitioners to focus design of models and experiments instead of environment design.

Read more

OpenSim RL

OpenSim RL allows you to synthesize physiologically accurate movement by combining biomechanical expertise embedded in the OpenSim simulation software with state-of-the-art control strategies using Deep Reinforcement Learning.

Read more