Overview

Beamrider takes place above Earth’s atmosphere, where a large alien shield called the Restrictor Shield surrounds the Earth. The player’s objective is to clear the Shield’s 99 sectors of alien craft while piloting the Beamrider ship. The Beamrider is equipped with a short-range laser lariat and a limited supply of torpedoes. The player is given three at the start of each sector.

To clear a sector, fifteen enemy ships must be destroyed. A “Sentinel ship” will then appear, which can be destroyed using a torpedo (if any remain) for bonus points. Some enemy ships can only be destroyed with torpedoes, and some must simply be dodged. Occasionally during a sector, “Yellow Rejuvenators” (extra lives) appear. They can be picked up for an extra ship, but if they are shot they will transform into ship-damaging debris.

Description from Wikipedia

State of the Art

Human Starts

Result Method Type Score from
72233.7 ApeX DQN DQN Distributed Prioritized Experience Replay
37412.2 DuelingPERDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
31181.3 PERDDQN (rank) DQN Prioritized Experience Replay
26172.7 PERDDQN (prop) DQN Prioritized Experience Replay
24622.2 A3C LSTM PG Asynchronous Methods for Deep Learning
22707.9 A3C FF (4 days) PG Asynchronous Methods for Deep Learning
21768.5 RainbowDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
17417.2 DDQN DQN Deep Reinforcement Learning with Double Q-learning
15241.5 NoisyNetDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
15002.4 DistributionalDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
14961.0 Human Human Massively Parallel Methods for Deep Reinforcement Learning
14591.3 DuelingDDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
13235.9 A3C FF (1 day) PG Asynchronous Methods for Deep Learning
12041.9 PERDQN (rank) DQN Prioritized Experience Replay
9743.2 DQN2015 DQN Dueling Network Architectures for Deep Reinforcement Learning
8672.4 DQN2015 DQN Massively Parallel Methods for Deep Reinforcement Learning
3822.07 GorilaDQN DQN Massively Parallel Methods for Deep Reinforcement Learning
254.6 Random Random Massively Parallel Methods for Deep Reinforcement Learning

No-op Starts

Result Method Type Score from
63305.2 ApeX DQN DQN Distributed Prioritized Experience Replay
30276.5 DuelingPERDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
23384.2 PER DQN Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
23384.2 PERDDQN (rank) DQN Dueling Network Architectures for Deep Reinforcement Learning
22430.7 PERDDQN (prop) DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
20793.0 NoisyNet-DQN DQN Noisy Networks for Exploration
18501.0 NoisyNet-DuelingDQN DQN Noisy Networks for Exploration
16926.5 Human Human Dueling Network Architectures for Deep Reinforcement Learning
16850.2 RainbowDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
16298.0 DuelingDQN DQN Noisy Networks for Exploration
14074.0 C51 Misc A Distributional Perspective on Reinforcement Learning
13772.8 DDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
13581.4 ACKTR PG Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
13213.4 DistributionalDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
12534.0 NoisyNetDQN DQN Rainbow: Combining Improvements in Deep Reinforcement Learning
12164.0 DuelingDDQN DQN Dueling Network Architectures for Deep Reinforcement Learning
11237.0 NoisyNet-A3C PG Noisy Networks for Exploration
10564.0 DQN DQN Noisy Networks for Exploration
9214.0 A3C PG Noisy Networks for Exploration
8627.5 DQN2015 DQN Dueling Network Architectures for Deep Reinforcement Learning
8299.4 DDQN+PopArt DQN Learning values across many orders of magnitude
7654.0 DDQN DQN Deep Reinforcement Learning with Double Q-learning
6846 DQN2015 DQN Human-level control through deep reinforcement learning
5775 Human Human Human-level control through deep reinforcement learning
5401.4 DuelingPERDDQN DQN Deep Q-Learning from Demonstrations
5173.3 DQfD Imitation Deep Q-Learning from Demonstrations
3302.91 GorilaDQN DQN Massively Parallel Methods for Deep Reinforcement Learning
1743 Contingency Misc Human-level control through deep reinforcement learning
929.4 Linear Misc Human-level control through deep reinforcement learning
363.9 Random Random Human-level control through deep reinforcement learning

Normal Starts

Result Method Type Score from
7456 Human Human Playing Atari with Deep Reinforcement Learning
4092 DQN2013 DQN Playing Atari with Deep Reinforcement Learning
3863.3 ACER PG Proximal Policy Optimization Algorithms
3031.7 A2C PG Proximal Policy Optimization Algorithms
1743 Contingency Misc Playing Atari with Deep Reinforcement Learning
1590.0 PPO PG Proximal Policy Optimization Algorithms
1425.2 TRPO (single path) PG Trust Region Policy Optimization
996 Sarsa Misc Playing Atari with Deep Reinforcement Learning
859.5 TRPO (vine) PG Trust Region Policy Optimization
354 Random Random Playing Atari with Deep Reinforcement Learning