The Open Security AI Alliance says the Hugging Face/OpenAI mess proves frontier labs can't be trusted to properly secure sensitive systems
In the wake of OpenAI agents attacking Hugging Face, Nvidia has recruited a new posse of partners to promote open source models as the security solution the industry needs. The AI arms dealer announced the foundation, the Open Secure AI Alliance, in a blog post today, describing the mission of the group being “to ensure defenders everywhere have open, frontier tools they can trust and control.”
Partners in the group are numerous, ranging from established tech giants like Microsoft, Red Hat, HPE, IBM, and Adobe to newer groups like Palantir, SpacexAI, Hugging Face, and The Linux Foundation. What all the founding members have in common, Nvidia said, is that they agree open source AI models are a fundamental part of modern cybersecurity, just like prior open source tech has been for the infosec space.
“The United States and its partners now face a choice in AI security: whether the defenses that protect our infrastructure will sit inside a few opaque systems or be built on open models, harnesses and tools that any defender can study, adapt and deploy,” Nvidia said in the announcement.
The claims in many ways echo the pleadings from tech industry heavyweights made in an open letter to US government regulators last week. That letter, signed by many of the same companies that are part of the founding OSAA cadre, essentially argues that regulators should ensure Anthropic, Google, and OpenAI don’t end up with total control of the US AI market, and that open-weight models should be given a seat at the table, too.
The new alliance is arguing that, not only do open-weight models need to be allowed to proliferate in the US, but they also need to be considered a fundamental part of the security puzzle.
For those unfamiliar with the Hugging Face incident, a group of autonomous OpenAI agents, operating in a sandbox and stripped of guardrails to test their full capability to solve cybersecurity puzzles, exploited a pair of zero-days to escape and gain access to the internet. For some reason, the bots thought the solution to the problems they were posed could be found in Hugging Face systems, so they broke in and accessed a bunch of private information and hijacked some credentials.
When Hugging Face turned to closed-source US frontier AI lab bots to examine the incident and help figure out what happened, those tools declined to help because they thought the data Hugging Face was trying to examine was itself malicious. Hugging Face turned to Chinese-made GLM 5.2, hosted on its own infrastructure, to figure things out.
“That incident showed a practical truth,” said Nvidia. “When defenders cannot inspect, adapt and run advanced AI on their own infrastructure, their ability to respond is constrained at exactly the moment speed matters most.”
Only open-source AI models, which China leads development on, can fill that role, the OSAA argues, and it’s prepared to counter those who say open models are a threat: Just look at what happened last week and it's readily apparent that closed source models are dangerous too.
The Alliance is pooling its efforts to give security pros access to essential open tools. Nvidia said that it’s participating by releasing its Object-Oriented Agent project on GitHub, HPE is contributing its SPIFFE/SPIRE zero-trust AI identity framework, Hugging Face has handed its Safetensors transparent AI model weight formatting to the PyTorch Foundation, and SpaceXAI has open-sourced Grok Build (though the reason behind that doesn’t appear to be entirely benevolent).
In addition, IBM and Red Hat have released Lightwell, an automated open-source vulnerability remediation platform, while Microsoft has come out with MDASH, a multi-model agentic scanning harness to automate bug discovery and remediation. Those efforts, while not open source themselves, are still a sign that Alliance members “are building an open defense stack,” Nvidia said.
MORE CONTEXT
The OSAA ended its announcement with another call for policymakers not simply ban open-source AI models, as doing so “would weaken defensive capacity and risk concentrating power, dependence, and vulnerability in a few closed providers,” the group said.
Many providers, as we saw last week, are more concerned with protecting themselves than helping victims of autonomous cyber attacks respond quickly.
Clement Delangue, cofounder and CEO of Hugging Face, said in a post on X that he spoke to OpenAI over the weekend about last week’s incident and asked the company to provide funding to support the development of better open-source AI cyber defenses. It’s not clear if the company plans to fulfill that request; it’s not a founding member of the Nvidia-led OSAA. Neither is Google or Anthropic, for that matter.
We reached out to all three companies for their take on the new initiative, but didn’t hear back from any of them. ®
