Image credit: Josh Kirschner/Techlicious via ChatGPT
OpenAI told the New South Wales government on October 1 that one of its AI models collected non-public wildfire statistics from a state service in June, the fifth Australian government system tied to unintended activity by its models. The Guardian reports that OpenAI became aware of it on Tuesday and spent 48 hours reviewing it before briefing the NSW Premier's office – moths after the incident occurred.
According to OpenAI's statement to ABC News, the model queried the National Parks and Wildlife Service's Fire History service in a way that went "beyond its intended use," gathering summary fire statistics the service doesn't make public. OpenAI says its review found no sign the model retrieved personal information. The NSW Department of Climate Change, Energy, the Environment and Water is working with Cyber Security NSW and its technology provider to assess the impact, ABC Australia reports
OpenAI's own account says a review of past training and evaluation activity, begun after July's Hugging Face breach, turned up activity at four other Australian agencies in mid-August. It notified those affected government services back in September.
OpenAI says safeguards added since July, including blocking live internet access in its research environments and notifying a human reviewer when a model breaks out, would now have flagged the Australian activity. The company has also paused training and evaluation involving tool use for its most capable models until it has additional protections in place. The Hugging Face breach, in which OpenAI's agents broke into the AI platform's systems during testing, remains the most severe incident the company has identified.
Australian lawmakers are questioning whether AI companies can be relied on to report these events promptly. Greens MP Abigail Boyd said "we simply cannot trust these companies, They have no respect for the sovereignty of our governments, or our way of life" and criticized the gap between the June incident and this week's notification. Prime Minister Anthony Albanese called the delay in notifying the government over earlier incidents unacceptable.
U.S. regulators are examining the same problem. California Attorney General Rob Bonta served OpenAI with an investigative subpoena on September 30 as part of a formal investigation into the Hugging Face incident. A coalition of 15 state attorneys general led by Iowa's Brenna Bird is also seeking information from OpenAI about that breach.
OpenAI Chief Strategy Officer Jason Kwon is scheduled to appear before Australia's Joint Select Committee on Artificial Intelligence in Sydney on October 6. OpenAI says it will notify any additional agencies its ongoing review identifies.
