When AI starts improving AI, OpenAI wants humans to remain in the loop

OpenAI is preparing for a stage where automated systems directly contribute to developing their successors. As recursive self-improvement becomes a concern, the company is calling for international standards to measure research autonomy, define human oversight, and establish common incident-reporting protocols.

When AI helps build the next AI

A growing share of AI research could eventually be performed by AI systems themselves. In its proposal on standards for the next phase of AI, OpenAI puts that automation at the center of its roadmap: build an automated AI researcher, use it to work on alignment, and find ways to keep people involved in the self-improvement loop.

The company uses the term recursive self-improvement, or RSI, for a stage where AI systems take on an increasing share of the work involved in developing successive generations. Humans may remain involved, but progressively automating that loop could sharply accelerate the pace of AI research.

OpenAI does not describe fully autonomous RSI as a current capability. It says it is not happening today and should not be pursued unless and until it can be done safely. Losing sight of the research process

The concern is not simply that AI systems become more capable. As they take on a larger role in their own R&D, people could eventually lose enough understanding of the underlying research processes that meaningful oversight becomes difficult.

OpenAI also sees a defensive side to the same capability. An automated AI researcher could work on alignment, develop defenses against increasingly capable systems, or help secure critical infrastructure. Research acceleration and safety research could therefore draw on some of the same underlying capabilities.

The company connects this concern to a previously disclosed Hugging Face incident, while explicitly noting that the incident was not caused by RSI. Instead, it is presented as a preview of the kinds of risks that could become more severe as autonomous capabilities increase without sufficient safeguards. Measuring how much research is actually automated

Rather than allowing every AI lab to define its own thresholds, OpenAI is calling for shared technical standards across countries. Those standards could measure RSI-related capabilities as well as how much research inside an AI company is being conducted autonomously.

Another metric would focus directly on oversight: certain automated research processes could trigger immediate human review. Common severity levels and reporting thresholds could also be established for incidents involving alignment and automated AI research.

The framework would apply to both open and closed models. OpenAI also specifies that these technical standards would not function as licenses, mandatory pre-release reviews, or international approval requirements. National governments would remain responsible for deciding whether and how to incorporate them into domestic law. A US-led international framework

The proposal gives the United States a leading role in shaping this international framework. OpenAI suggests using the existing network of AI safety institutes, including organizations in France, the UK, Japan, South Korea, India, Canada, and Australia.

It also points to ISO, the Frontier Model Forum, the Agentic AI Foundation, and the Open Secure AI Alliance as organizations that could contribute. The structure would establish shared measurement methods and technical standards while leaving regulatory authority with individual governments.

OpenAI also supports secure communication channels for governments and critical infrastructure operators to exchange information about vulnerabilities, emerging threats, and safety incidents. Within that framework, the company argues that dialogue between the United States and China would be a useful step.