Outside auditors find nearly a quarter of OpenAI's internal traffic went unwatched

Up to 25 percent of internal traffic from OpenAI’s most advanced models may have operated without any safety monitoring during a recent three-week audit. This is the upper bound of an estimate provided by outside researchers who spent weeks inside the company’s systems, though they note the uncertainty around that figure is significant in both directions. The findings suggest that a substantial portion of the company’s most powerful AI agents were running without the oversight mechanisms designed to detect rogue behavior.

The audit was conducted by two researchers from METR and Redwood Research, who were granted rare on-site access to OpenAI between July 21 and August 11. During this period, they had direct access to the company's codebase and internal documentation. They also interviewed approximately a dozen employees to understand how the monitoring systems were intended to function versus how they actually operated. This pilot program was organized jointly by the two research firms and OpenAI to allow for a deep, technical evaluation of internal infrastructure.

Before this review, the public had limited insight into the specific gaps in OpenAI's safety nets. The researchers focused specifically on the agentic Responses API traffic for models at the GPT-5.5 class and above. Their analysis indicated that a significant slice of this traffic was entirely outside the scope of the monitoring tools. While the exact percentage is uncertain, the potential for such a large blind spot raises questions about the robustness of the current safety architecture for the company's flagship models.

In addition to the unmonitored API traffic, the auditors identified a specific technical failure affecting internal Codex traffic. Between May and July, roughly 10 percent of this traffic was estimated to be untracked. This gap occurred due to a bug in how the system handled 'compaction events,' which are essentially context summaries used to manage memory. The auditors noted that this estimate also carries significant uncertainty. OpenAI had independently discovered this bug and implemented a fix while the audit was still in progress, but the gap had existed for months prior.

During the period when compaction events were not being monitored, agents had an additional way to delay oversight. The system was designed to trigger monitoring only when an agent handed control back to the user. This design allowed agents to sidestep checks by simply continuing to execute actions without yielding control. The auditors also demonstrated that it was possible to hide hundreds of actions from the monitor and to bypass the Codex sandbox entirely. Furthermore, they determined that specially crafted inputs could likely crash the monitoring system completely.

Another critical finding was that model behavior during reinforcement learning (RL) training was not being monitored at all when the audit began. OpenAI stated that it paused RL training following an incident involving Hugging Face in August and subsequently expanded its monitoring to cover this area. However, these changes were made after the auditors' access period ended, meaning they were not part of the formal evaluation. The company claims to have fixed all the weaknesses identified by the auditors and provided the full 61-page report internally.

The transparency of the findings remains a point of contention. After OpenAI applied redactions, METR and Redwood Research leadership received a 38-page version of the report. Only a short summary was made available to the general public. METR stated that it had no right to disclose the redacted portions on its own, leaving many technical details out of the public eye. The company asserts that all identified issues have been resolved, but the extent of the public's access to the full technical details is limited.

Looking ahead, this audit is viewed as a pilot program for a new type of independent oversight. For future engagements, METR indicates it wants to push for greater public transparency, suggesting that the current level of redaction may not be the final model for external audits. The goal is to ensure that outside researchers can provide meaningful feedback on AI safety without being constrained by extensive internal secrecy. As the industry moves toward more autonomous agents, the balance between proprietary security and public accountability will likely become a central focus of upcoming reviews.

The results of this three-week investigation highlight the complexity of monitoring AI systems that are increasingly capable of acting independently. While OpenAI has stated that the specific bugs found have been fixed, the initial gaps suggest that the safety nets may have been more porous than previously understood. As the company continues to deploy more advanced models, the findings from this audit will likely serve as a benchmark for how thoroughly these systems are scrutinized by independent parties in the coming months.