An intelligence report circulated across the US military this spring during the war with Iran set off an alarming chain of events after it claimed that a Chinese ship in the Middle East was carrying components linked to a nuclear weapons programme.According to CNN, the report prompted the US military to prepare an operation to intercept the vessel. Armed US personnel were preparing to board the ship, while military aircraft were already in the air, according to sources familiar with the episode. It was only shortly before the planned operation that officials took a closer look at the intelligence and discovered that it had been produced with the assistance of artificial intelligence.The chatbot used by a US Special Operations Command analyst had incorrectly identified the material listed in the ship’s manifest. The resulting intelligence report, described by one source as “entirely false”, had travelled through official channels and was being treated as credible military intelligence. “It almost started a war,” the source told CNN.
AI error reaches military chain
The incident began when the analyst examined intelligence about the ship’s manifest that originated with US Special Operations Command Pacific, based in Hawaii. The analyst queried a chatbot about the information, but the AI system incorrectly identified the cargo.The chatbot had combined open-source information with classified signals intelligence held by the US government before reaching its conclusion. The analyst then used AI again to turn the findings into a standard intelligence report, giving the information the format and appearance of conventional military intelligence before it was disseminated.CNN reported that it could not establish whether the chatbot was a commercially available system or an AI tool developed for the US government. A former senior US official familiar with the systems used by military and intelligence analysts told CNN that internal tools were “mostly just copies of the commercial stuff wearing lipstick”.The episode highlights the risks of relying on AI-generated analysis in situations where decisions have to be made rapidly and the consequences of an error can extend beyond the battlefield. Any US attempt to board or otherwise confront a Chinese vessel could have risked escalating into a broader confrontation between Washington and Beijing.US military and intelligence agencies are nevertheless rapidly expanding their use of AI. The technology is being explored for analysing huge volumes of intelligence, selecting targets, moving military assets and handling logistics, budgeting and supply chains. Defence officials have argued that faster AI-assisted decision-making could be critical in a future conflict with China or another major adversary.In January, US Defence Secretary Pete Hegseth released the Pentagon’s “Artificial Intelligence Acceleration Strategy”, aimed at speeding up the military’s adoption of AI.
From battlefield intelligence to weapons
The episode comes as governments, militaries and AI companies grapple with another question: how increasingly capable AI systems can be prevented from being misused for military purposes.Anthropic recently reported that a Yemen-based weapons engineering cell had used its Claude AI model to develop guidance, navigation and control software for missiles and a guided rocket. The company said the group used Claude Code in place of human software engineers to integrate an open-source autopilot with a phone-class flight computer, write control software, tune settings, run firmware builds and conduct flight simulations.Anthropic said there was no evidence that the group successfully deployed an operational weapon, although it carried out a test launch of a guided rocket that appeared to fail. The company said safeguards blocked many requests but not all, while the group repeatedly tried to circumvent restrictions.The Yemen case was among several conventional-weapons operations identified by Anthropic, including work involving drone swarms, torpedo interception and targeting systems for electronic warfare and air-defence suppression. The company also identified around 35 potentially concerning biological research efforts over a 30-day period, while cautioning that it could not determine whether those activities were intended for harm or legitimate scientific research.The concerns are now extending to the highest levels of international security discussions. OpenAI CEO Sam Altman is due to brief the UN Security Council on AI and international security next week, according to Reuters. The meeting comes as governments consider how increasingly powerful AI systems should be governed and how common safety standards could be established.