Out-of-control AI agents are causing concern, and 120 organizations including NVIDIA are proposing to establish an accident tracking mechanism. According to the US news website Axios, an alliance composed of more than 120 organizations, including NVIDIA, Cisco and CrowdStrike, is proposing to establish a new accident reporting mechanism for AI agents.

Out-of-control AI agents are causing concern, and 120 organizations including NVIDIA are proposing to establish an accident tracking mechanism. According to the US news website Axios, an alliance composed of more than 120 organizations, including NVIDIA, Cisco and CrowdStrike, is proposing to establish a new accident reporting mechanism for AI agents. The mechanism will require participating companies to disclose specific agent failures and keep detailed records of the causes of incidents. Agents from companies such as OpenAI are losing control one after another. As AI agents become increasingly capable of autonomously performing tasks across computer systems, the industry still lacks a standardized way to report security failures and learn from them. Recently, many companies, including OpenAI, disclosed incidents of out-of-control intelligent agents. The Open Secure AI Alliance (OSAA), led by Nvidia, is developing a set of guidelines called the Shared AI Discovery Exchange (SAFE). This proposed mechanism is used to standardize how organizations report cybersecurity incidents involving AI agents. The draft guidelines call for participation from model deployers, AI developers, cloud service and tool providers, independent researchers, critical infrastructure operators, and other relevant groups. Government agencies will also be invited to participate as "non-controlling observers." SAFE members will be required to agree to report the following types of incidents: AI systems accessing or exploiting third-party systems without authorization, leaking confidential information, or continuing to detect production targets after the operator suspects that the activity is unauthorized. Members also need to report certain near-misses and retain evidence related to the incident, including prompt words, agent running trajectories, tool call records, identity information, permissions and credentials, etc. Under the proposed timeline, members should notify affected organizations as soon as possible, submit an initial confidential report to SAFE within four working days, issue a preliminary factual report within 30 days where appropriate, and provide a remediation progress update within 90 days. The draft guidelines state: "Intention does not determine whether an incident should be reported. Even if the operating environment is considered to be simulated, this may explain the cause of the incident, but it does not exempt the obligation to report." SAFE will analyze various security incidents, identify recurring failures of AI agents, and promote the industry to adopt unified security protection measures. SAFE currently does not provide formal "safe harbor" protection provisions for companies that proactively disclose AI security incidents, so companies may face risks from disclosing sensitive information. However, the alliance believes that the cybersecurity industry’s longstanding tradition of sharing threat intelligence will drive companies to proactively participate. Julien Soria, deputy chief information security officer and vice president, NVIDIA