Deep Reinforcement Learning with Double Q-Learning

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

The popular Q-learning algorithm is known to overestimate action values under certain conditions. It was not previously known whether, in practice, such overestimations are common, whether they harm performance, and whether they can generally be prevented. In this paper, we answer all these questions affirmatively. In particular, we first show that the recent DQN algorithm, which combines Q-learning with a deep neural network, suffers from substantial overestimations in some games in the Atari 2600 domain. We then show that the idea behind the Double Q-learning algorithm, which was introduced in a tabular setting, can be generalized to work with large-scale function approximation. We propose a specific adaptation to the DQN algorithm and show that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.

Last synced 2026-08-22 14:19 UTC

• Reward Shaping based on Trajectory Quality for offline and hybrid reinforcement learning

  Information Processing & Management 2026 · 0 citations

• ACID: beta-testing active inference for active cyber-defence

  Journal of Reliable Intelligent Environments 2026 · 0 citations

• A Semantic-Group Self-Attention Deep Q-Network with Coverage-Constrained Experience Replay and Lyapunov Safety Constraint for 3D UAV Path Planning

  Algorithms 2026 · 0 citations