OpenAI has amended its newly signed contract with the US Department of War after chief executive Sam Altman admitted the original announcement “looked opportunistic and sloppy”.
The news triggered fears the company’s technology could be used for domestic mass surveillance.
The San Francisco-based AI firm signed the agreement with the Pentagon on Friday, hours after rival Anthropic was blacklisted by the Trump administration and designated a “supply chain risk” following a dispute over safeguards.
But just days after, the ChatGPT owner was forced into damage control.
In a post on X late Monday, Altman said the company would “explicitly” bar its systems from being “intentionally used for domestic surveillance of U.S. Persons and nationals”.
Intelligence agencies like the National Security Agency would similarly require a “follow-on modification” before being allowed to use OpenAI’s models under the contract.
“We shouldn’t have rushed to obtain this out on Friday”, Altman wrote. “The issues are super complex and demand clear communication. We were genuinely trying to de-escalate things and avoid a much worse outcome, but I think it just looked opportunistic and sloppy.”
The Shifting Sands of AI and Defence
This situation highlights the growing rivalry between OpenAI and Anthropic, and the increasing entanglement of AI with US military strategy. Anthropic’s refusal to allow its technology to be used for “mass domestic surveillance” or “fully autonomous weapons” led to its designation as a “supply chain risk”.
The US defence secretary’s response underscores a willingness to prioritize access to AI capabilities, even if it means sidelining companies with strong ethical objections. This sets a precedent that could influence future collaborations between the government and AI developers.
The Future of AI Safeguards
The debate surrounding OpenAI and Anthropic’s contracts raises critical questions about the appropriate level of safeguards for AI systems deployed by the military. While OpenAI claims its agreement has “more guardrails” than previous deployments, concerns remain about the potential for misuse and the difficulty of verifying compliance.
The open letter signed by nearly 900 employees across OpenAI and Google reflects a broader anxiety within the AI community about the ethical implications of military applications. The call for companies to resist demands for loosening safeguards suggests a growing movement to prioritize responsible AI development.
The incident with Anthropic and OpenAI demonstrates a clear divergence in approaches to AI ethics within the industry. This divergence could lead to a fragmented landscape, with some companies prioritizing innovation and government contracts, while others remain committed to stricter ethical principles.
AI’s Expanding Role in Modern Warfare
AI is already deeply embedded in Western military infrastructure, with systems like Palantir being used for data analysis and decision-making. As AI capabilities continue to advance, its role in warfare is likely to expand, raising concerns about the potential for increased automation and the erosion of human control.
The debate over “human in the loop” systems highlights the necessitate for clear guidelines and oversight mechanisms to ensure that AI is used responsibly and ethically in military contexts. The challenge lies in balancing the potential benefits of AI with the risks of unintended consequences.
The Potential of Open-Source AI
The current situation may accelerate the development and adoption of open-source AI models. If companies are hesitant to work with the Department of Defence due to ethical concerns, open-source alternatives could provide a way to ensure that AI technology remains accessible and transparent.
Open-source AI allows for greater scrutiny and collaboration, potentially leading to more robust safeguards and a more equitable distribution of benefits. However, it also raises challenges related to security and the potential for misuse.