Overview

Space Invaders is a two-dimensional shooter game in which the player controls a laser cannon by moving it horizontally across the bottom of the screen and firing at descending aliens. The aim is to defeat five rows of eleven aliens—some versions feature different numbers—that move horizontally back and forth across the screen as they advance toward the bottom of the screen. The player defeats an alien, and earns points, by shooting it with the laser cannon. As more aliens are defeated, the aliens’ movement and the game’s music both speed up. Defeating the aliens brings another wave that is more difficult, a loop which can continue without end.

The aliens attempt to destroy the cannon by firing at it while they approach the bottom of the screen. If they reach the bottom, the alien invasion is successful and the game ends. A special “mystery ship” will occasionally move across the top of the screen and award bonus points if destroyed. The laser cannon is partially protected by several stationary defense bunkers—the number varies by version—that are gradually destroyed by numerous blasts from the aliens or player. A game will also end if the player’s last laser base is destroyed.

Description from Wikipedia

State of the Art

Human Starts

Result Method Type Score from
50699.3 ApeX DQN DQN Distributed Prioritized Experience Replay
23846.0 A3C LSTM PG Asynchronous Methods for Deep Learning
15730.5 A3C FF (4 days) PG Asynchronous Methods for Deep Learning
12629.0 RainbowDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
9063.0 PERDDQN (prop) DQN Prioritized Experience Replay
8978.0 DuelingPERDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
6368.6 DistributionalDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
5993.1 DuelingDDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
3912.1 PERDDQN (rank) DQN Prioritized Experience Replay
2628.7 DDQN DQN Deep Reinforcement Learning with Double Q-learning
2214.7 A3C FF (1 day) PG Asynchronous Methods for Deep Learning
1697.2 NoisyNetDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
1696.9 PERDQN (rank) DQN Prioritized Experience Replay
1464.9 Human Human Massively Parallel Methods for Deep Reinforcement Learning
1449.7 DQN2015 DQN Massively Parallel Methods for Deep Reinforcement Learning
1293.8 DQN2015 DQN Dueling Network Architectures for Deep Reinforcement Learning
1183.29 GorilaDQN DQN Massively Parallel Methods for Deep Reinforcement Learning
182.6 Random Random Massively Parallel Methods for Deep Reinforcement Learning

No-op Starts

Result Method Type Score from
54681.0 ApeX DQN DQN Distributed Prioritized Experience Replay
19723.0 ACKTR PG Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
18789.0 RainbowDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
15311.5 DuelingPERDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
7696.9 PERDDQN (prop) DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
6869.1 DistributionalDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
6427.3 DuelingDDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
5909.0 NoisyNet-DuelingDQN DQN Noisy Networks for Exploration
5747.0 C51 Misc A Distributional Perspective on Reinforcement Learning
3154.6 DDQN DQN Deep Reinforcement Learning with Double Q-learning
2865.8 PER DQN Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
2865.8 PERDDQN (rank) DQN Dueling Network Architectures for Deep Reinforcement Learning
2589.7 DDQN+PopArt DQN Learning values across many orders of magnitude
2525.5 DDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
2186.0 NoisyNet-DQN DQN Noisy Networks for Exploration
2145.5 NoisyNetDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
1976 DQN2015 DQN Human-level control through deep reinforcement learning
1883.41 GorilaDQN DQN Massively Parallel Methods for Deep Reinforcement Learning
1692.3 DQN2015 DQN Dueling Network Architectures for Deep Reinforcement Learning
1668.7 Human Human Dueling Network Architectures for Deep Reinforcement Learning
1652 Human Human Human-level control through deep reinforcement learning
1283.0 DQN DQN Noisy Networks for Exploration
1158.0 DuelingDQN DQN Noisy Networks for Exploration
1126.0 NoisyNet-A3C PG Noisy Networks for Exploration
1034.0 A3C PG Noisy Networks for Exploration
267.9 Contingency Misc Human-level control through deep reinforcement learning
250.1 Linear Misc Human-level control through deep reinforcement learning
148 Random Random Human-level control through deep reinforcement learning

Normal Starts

Result Method Type Score from
3690 Human Human Playing Atari with Deep Reinforcement Learning
1213.9 ACER PG Proximal Policy Optimization Algorithms
942.5 PPO PG Proximal Policy Optimization Algorithms
744.5 A2C PG Proximal Policy Optimization Algorithms
581 DQN2013 DQN Playing Atari with Deep Reinforcement Learning
568.4 TRPO (single path) PG Trust Region Policy Optimization
450.2 TRPO (vine) PG Trust Region Policy Optimization
271 Sarsa Misc Playing Atari with Deep Reinforcement Learning
268 Contingency Misc Playing Atari with Deep Reinforcement Learning
179 Random Random Playing Atari with Deep Reinforcement Learning