How to Fool Deep RL Agents Without Knowing Their Architecture

👁 112 reads

Researchers show that image-classification attacks fail against RL agents, but trajectory-level perturbations transfer across different algorithms.

The paper comes from arXiv (identifier 2610.06083) and was submitted on October 5, 2026 by Zexina Li, Ruili Yao, Yiminga Zenga, and Xiaoxue Gao. The document is 63 KB and is classified in the Machine Learning and Artificial Intelligence sections.

The Problem: Attackers Don’t Know the Victim

Most previous research on attacks against deep reinforcement learning (DRL) agents assumed white-box access - full visibility into the parameters of the target policy. The authors note explicitly that this assumption rarely holds in practice. They focus on a black-box transfer scenario: the attacker has their own surrogate agent with full access, and uses it to construct observation perturbations. These perturbations are then fed to an unknown victim agent in the hope they will work despite differences in architecture or learning algorithm.

Attack Goal and First Observation

The authors frame the task as return minimization while constraining perturbation size at each step. As a starting point, they test what happens when well-known image classification attacks - FGSM, MI-FGSM, and NI-FGSM - are transferred to DRL with a per-step objective. The result is clear: perturbations obtained this way do transfer to the victim, but are no stronger than random noise with the same budget. This is an important observation because it shows that simply porting a proven technique to a new domain is not enough.

Proposed Solution: Trajectory-Level Attack

Instead of optimizing perturbations separately for each step, the authors propose a trajectory-level attack. The method optimizes an entire sequence of perturbations over a receding time horizon. To do this, it uses a differentiable environment model and a temperature-smoothed surrogate policy. The optimizers remain the same as in per-step methods, allowing direct comparison.

Experiments and Results

Tests were conducted on the CartPole-v1 environment. Ten agents based on DQN and DDQN algorithms were used, along with 100 surrogate-victim pairs. The authors tested three scenarios: white-box (attacker knows the victim), cross-model (different instance of the same algorithm), and cross-algorithm (different learning algorithm). In all three cases, the trajectory-level attack outperformed per-step attacks and random noise. The document does not provide detailed numerical results in the abstract - they are available only in the full PDF text.

What the Document Doesn’t Include

The text available to us is only the arXiv page with abstract and metadata. It does not include detailed results tables, network architecture descriptions, code, or discussion of the method’s limitations. It is also unclear whether the paper was peer-reviewed or accepted to any conference - the document appears as a preprint.

What This Means

In our view, this work is interesting for two reasons. First, it honestly shows a dead end - transferring popular attacks from image classification to RL simply does not work better than random perturbations, which is a result worth noting in itself. Second, the proposed trajectory-level approach makes conceptual sense, because DRL inherently operates on sequences, not individual frames. The weakness is that tests are limited to one simple environment (CartPole-v1) - it is unclear whether the method scales to more complex tasks. As a preprint from October 2026, it has not yet undergone public peer review, so the results should be approached with caution.

Source: arXiv cs.AI: Boosting Transferable Adversarial Attacks against Deep Reinforcement Learning (document from 2026-10-06). This discussion is based on the full text of the document, not media reports.

// read morePhantomEnvironments: Training AI Agents in Fictional WorldsState of AIDeepSeek V4.1 Flash: New Encoder-Decoder Architecture ReviewBenchmarksSKIN-DEEP: Testing AI Model Safety After UpdatesBenchmarks

Kinetyra