U.S. Air Force didn’t conduct AI simulation in which military drone killed operator, official says

Are artificial intelligence-enabled drones operated by the U.S. Air Force going rogue in simulations and turning on their operators? An Instagram video posted June 2 claimed as much, saying one drone in training "had a mind of its own."

"U.S. Air Force is working with an AI-enabled drone that'll take out enemy air defenses," said a man in a video. The speaker was identified in the post as Philip DeFranco, a YouTuber who comments on current events.

"There is just one problem, though," DeFranco warned: It was "crazy."

In simulations, the drone would identify targets, he said. The operator would then direct the drone whether or not to kill the threat. 

"Because the AI knew that it got points if it destroyed threats, when an operator would prevent it from doing so, it would kill its operators," he said. "So then they trained the AI not to do that. So, instead, it started taking out communication towers to prevent the operator from telling it to not kill the target." 

The Instagram post was flagged as part of Meta’s efforts to combat false news and misinformation on its News Feed. (Read more about our partnership with Meta, which owns Facebook and Instagram.)

It’s unclear when DeFranco made the video — we didn’t find it on his social media accounts and he did not respond to our email.

We found that the claim stemmed from a scenario described by an Air Force official. But that official later said he misspoke — the Air Force did not conduct any such "simulation."

During a Royal Aeronautical Society conference held May 23-24 in London, U.S. Air Force Col. Tucker "Cinco" Hamilton, chief of AI test and operations, spoke about the benefits and hazards of autonomous weapons systems.

Hamilton relayed a story that mirrored the one described in the video, according to highlights of the conference that were published on Royal Aeronautical Society’s website. He described a simulated test with an AI-enabled drone tasked to identify and destroy surface-to-air missile sites, with a human giving the final instruction. 

He said that because training reinforced that destroying the surface-to-air missile was preferred, the AI responded to human "no-go" decisions by attacking the operator. 

Featured Fact-check

X posts

stated on September 24, 2024 in a post

Image of people boarding up a window shows Michigan election officials are “preparing to steal this election”

After the system was trained not to kill the operator, he said, it started destroying the communication tower that the operator used.

The Royal Aeronautical Society updated the highlights page June 2, saying that Hamilton said he misspoke in his presentation and the "rogue AI drone simulation" was a "hypothetical ‘thought experiment’ from outside the military, based on plausible scenarios and likely outcomes rather than an actual USAF real-world simulation."

According to that note, Hamilton said, "We've never run that experiment, nor would we need to in order to realise that this is a plausible outcome." He also clarified that the Air Force has not tested any weaponized AI in this way. 

Air Force spokesperson Ann Stefanek told Insider that the department "has not conducted any such AI-drone simulations and remains committed to ethical and responsible use of AI technology." PolitiFact reached out to Stefanek but did not hear back.

We rate the claim that an AI-led drone killed its operator and took out a communication tower False.