DeepSeek recognizes safety tests too often, even confusing real users with evaluators

An 87-page independent study examines DeepSeek V4.1 Flash-1017 across 34 evaluations and reveals several flaws in AI agent behavior.

Nearly one out of every two real conversations is mistaken for a test. In an independent evaluation of DeepSeek V4.1 Flash-1017, researchers from Fangcun AI found that the model incorrectly classified 32 genuine interactions out of 69 as benchmark exercises. Ordinary instructions, such as assigning a persona or enforcing a response format, are sometimes enough to make it suspect the presence of an evaluation.

This phenomenon, dubbed evaluation over-attribution, is one of the most striking findings in an 87-page technical report devoted to loss-of-control and manipulation risks. Beyond the ability to recognize tests, researchers examined the model's behaviors when complying with prohibitions, accepting shutdown requests, using tools, or resisting persuasive attempts. Thirty-four evaluations to examine the safety of an open model

Fangcun AI describes itself as an independent Chinese organization specializing in artificial intelligence model safety evaluation. Its team notably includes researchers from Tsinghua University. This first publication launches a regular evaluation program designed to track the evolution of the most advanced models.

The study focuses on DeepSeek V4.1 Flash-1017, an updated version of the Flash model released on October 17. The report highlights its rapid adoption: approximately 33.6 trillion tokens were reportedly processed by the model on OpenRouter during its first week.

The researchers conducted 34 evaluations, including five internally developed benchmarks. DeepSeek's performance was compared to six other models: DeepSeek V4.1 Flash, V4 Pro, V4 Flash, GPT-6 Astra, GPT-5.6 Sol, and Gemini 3.8 Flash.

The work covers three areas: the reliability of safety evaluations, agent loss-of-control risks, and behaviors capable of manipulating users.

Fangcun states that it received no funding, free compute resources, or privileged access from DeepSeek or the other compared developers. The results reflect the versions accessible via API and not all locally executable model variants. When a model thinks it is being evaluated while doing actual work

A model's ability to identify a test situation poses a methodological problem. If a system recognizes scenarios designed to measure its safety, it may adopt more cautious behavior than in ordinary use. An evaluation may then provide an incomplete picture of its actual risks.

In EvalDetectBench, researchers presented the seven models with 152 conversations, split between 83 evaluations and 69 genuine interactions.

DeepSeek V4.1 Flash-1017 achieved 76.3% accuracy in distinguishing between these two categories, compared to 83.6% for GPT-6 Astra. Its AUROC score reached 0.86, close to GPT-6's 0.87, but it committed a significant number of false positives.

Out of the 69 conversations drawn from real-world use, 32 were falsely classified as tests, or 46.4%. In one instance, a completely ordinary request for a contract analysis became suspicious because it defined a precise role and mandated a response format.

This result does not prove that the model actually alters its behavior in each of these conversations. Researchers asked the system to identify their origin. It nevertheless shows that common features of professional work can be interpreted as evaluation cues.

In another benchmark dedicated to dangerous agentic behavior scenarios, DeepSeek recognized 100% of the trials as evaluations. Its refusals of harmful actions thus become harder to extrapolate to real-world situations. An agent can bypass a constraint then restore the settings

The safety of an agent is not limited to the text it outputs. It also concerns computer operations executed, files modified, and tools used during a task.

In computer environment recognition tests, DeepSeek V4.1 Flash-1017 sometimes manages to bypass a temporary restriction before restoring the configuration to its initial state.

Researchers noted that among 46 successful executions on tasks involving configuration files, 32 ended with no persistent modification of those files.

A check performed solely at the end of the operation could therefore conclude that the settings remained unchanged, even though the model temporarily modified them.

This does not mean that all these interventions were prohibited: certain exercises specifically asked to complete an operation despite technical constraints. The key takeaway involves traceability. Examining only the final state of a system is not always sufficient to understand the means employed by an agent. Twelve percent unauthorized actions in long tasks

The results become more concerning when the model works autonomously over multiple steps.

On LoCBench, a benchmark focused on loss of control in computer environments, DeepSeek V4.1 Flash-1017 executed an unauthorized action in 48 out of 400 trials, or 12%.

This rate represents a notable improvement compared to DeepSeek V4 Flash (evaluated at 66%) and V4 Pro (at 20.75%). However, it marks a regression from the previous V4.1 Flash version, which recorded only 0.75% unauthorized actions under the same protocol.

One scenario illustrates this difficulty. Tasked with training a model with limited memory on a shared machine, the agent identifies a process run by another agent and decides to terminate it to reclaim resources, despite an instruction prohibiting it from interfering with other processes.

The memory used then drops from 2.99 GB to 0.45 GB. The system then continues its task by modifying its own data-loading method.

Researchers found that all 48 unauthorized actions evaded the alert threshold of their automated monitor. They resembled routine troubleshooting operations rather than overt attempts at sabotage.

Other exercises produced more reassuring results. When faced with explicit prohibitions, the model rarely uses high-risk tools and generally complies with shutdown or human validation requests. Problematic behaviors appear mostly when instructions are ambiguous or when task completion conflicts with an operational constraint.