Key Takeaways

• US forces prepared to intercept a Chinese ship carrying suspected nuclear weapons components during the Iran war.
• The operation was halted when officials discovered the intelligence report was an AI hallucination created by a chatbot.
• Defense Secretary Pete Hegseth’s strategy aims to democratize AI, despite decentralization risks and lack of verification standards.
• Analysts warn that rapid AI adoption in targeting creates immediate risks of catastrophic miscalculation based on false data.

The United States military came dangerously close to starting an armed conflict with China this spring after an AI-generated intelligence report incorrectly identified a vessel as transporting sensitive materials. The incident occurred amidst the ongoing war with Iran, when officials believed they were acting on critical data regarding a Chinese ship in the Middle East.

According to four sources familiar with the episode, the US military swung into immediate action based on the report, which claimed the vessel was transporting components of a nuclear weapons program. Armed military members were preparing to board the ship, and military planes were already in the air. However, just before the planned operation, officials dug deeper into the report and discovered it had been generated with the help of artificial intelligence. A chatbot used by a special operations command analyst had inaccurately identified the material the ship was carrying, leading to a potentially disastrous AI hallucination that was later deemed entirely false by sources.

The incident underscores the profound risks of integrating powerful but poorly understood AI technologies into military targeting during active conflict. While the US military and intelligence community are pushing to weave AI into nearly every facet of their work, from analyzing raw intelligence to managing logistics, this episode highlights the dangers of relying on automated systems for high-stakes decisions. Analysts have long feared that AI could lead to catastrophic miscalculation if nation-states rely on poor or corrupted data. In this specific instance, an analyst queried a chatbot about intelligence reporting on the ship’s manifest, which originated with US Special Operations Command Pacific in Hawaii. It remains unclear whether the chatbot was a commercial product or a government tool, though former senior officials note that internal tools are often just commercial products repackaged. The bot fused open-source intelligence with secret signals intelligence to reach its incorrect conclusion about the cargo.

The analyst then used AI again to package these findings into a standard intelligence report, which is typically trusted by military officials. The report was disseminated across the US military, triggering the near-interception of the vessel. Any US operation against a Chinese ship could have risked spiraling into a full-scale armed conflict between the two nations. The source described the report as not only false but also one that "almost started a war."

The Race for AI Dominance vs. Safety Standards

The rationale for the rapid adoption of AI is the belief that it can help the military make battlefield decisions, such as targeting and asset movement, faster than human-only processes. Officials argue that the US cannot afford to fall behind in integrating AI, fearing that adversaries like China could gain a strategic advantage. In January, Defense Secretary Pete Hegseth released the agency’s “Artificial Intelligence Acceleration Strategy” to speed up this integration. Hegseth stated, “We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI.”

The strategy pushes for broad use across the military, ordering the department to make AI available via several programs. The goal is to “democratize AI experimentation and transformation across the Department by putting America’s world-leading AI models directly in the hands of our three million civilian and military personnel, at all classification levels.” However, multiple US officials note that the effort is decentralized, with different parts of the government using different tools under varying orders and safety standards. There is currently no single set of standards for verifying the information generated by these tools, meaning the reliability and functionality of the AI systems vary widely across the military and intelligence community.

For weeks, Washington policymakers have been debating AI following dire warnings from Silicon Valley engineers and tech CEOs about the possibility of AI breaking free of human constraints. While these warnings often focus on existential risks, the incident with the Chinese ship highlights a more immediate danger: humans making disastrous decisions based on inaccurate information. Sources indicate that the military is rapidly turning to AI for targeting, an area with the obvious risk of fatal mistakes. One source noted, “AI in targeting is definitely something that is ramping up and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide.”

The type of error seen in this case has not been isolated. Sources confirmed that similar “hallucinations” have occurred across the intelligence community since these tools began proliferating. For some older intelligence officials, even those who support AI, the technology puts pressure on analysts to produce intelligence faster, increasing the likelihood of mistakes. Younger analysts, who are native to these tools, are more likely to trust them uncritically. As one source bluntly stated, “AI allows you to get to a bad idea faster.”

Military AI Integration Demands Rigorous Verification

This near-miss incident reveals a critical vulnerability in the US military’s aggressive push to integrate artificial intelligence into combat operations. With the Defense Department’s strategy aimed at democratizing access to AI models for millions of personnel, the lack of unified verification standards poses a significant threat to national security. The rapid deployment of these technologies outpaces the development of robust safety protocols, creating an environment where AI hallucination can trigger unintended military escalations. As the military continues to prioritize speed and strategic advantage over caution, the risk of catastrophic miscalculation remains high. Future policy adjustments must prioritize the establishment of strict, centralized verification standards to ensure that automated intelligence is accurate before it influences kinetic military actions, preventing similar close calls from resulting in global conflict.