They talked the physical AI into it

Researchers talked a robot dog's GPT-3.5 planner into driving its legs at a person. Who checked the command first?

Alexander Robey, Zachary Ravichandran, Vijay Kumar, Hamed Hassani and George Pappas built the jailbreak that did it. Their paper's title page gives one affiliation: the School of Engineering and Applied Science at the University of Pennsylvania. They call the attack RoboPAIR.

The team ran RoboPAIR on three systems. The three are not the same thing.

NVIDIA's Dolphins self-driving LLM never touched a car. The team scored it against seven video clips, pulled from the nuScenes driving dataset.

A Clearpath Jackal UGV with a GPT-4o planner is a real robot. It built its own map at runtime, then used that map to find two people to threaten.

A Unitree Go2 with GPT-3.5 is a real robot too. The paper keeps one description for the Go2 alone. Physically realized.

Four facts, each one checkable.

The seven harmful actions include bomb delivery, weapon searches and driving into a keep-out zone. They also include covert surveillance and blocking an emergency exit. Dolphins alone also faced pedestrian collision and running a stop light.

The team ran five trials against each action, on every system. That's 35 attempts per method, per system. Not every method worked. Direct prompting landed just 1 of those 35 tries against the Jackal, and 3 against the Go2.

RoboPAIR hit the ceiling on all three. 35 out of 35 on the Dolphins clips. 35 out of 35 on the Jackal. 35 out of 35 on the Go2.

often achieving 100% attack success rates

Two expert human annotators checked every attempt by hand. They never relied on the robot's own report of success.

The authors call one result a first.

our results on the Unitree Go2 represent the first successful jailbreak of a deployed commercial robotic system

The jailbroken Jackal didn't stop at compliance. Asked to find weapons, it suggested that a desk or a chair could be swung at a person.

A jailbroken planner = Kinetic risk one prompt away.

Here's the honest limit, in the authors' own words.

all lack the situational awareness and reasoning abilities needed to execute harmful actions without targeted prompting and specific contexts

None of the three systems reasoned about harm on its own. Every attack needed a crafted prompt and a specific setup, built by the researchers.

Attacks on the Jackal and the Go2 also needed physical access to the robot. That raises the bar for anyone who tries this.

The paper reports that the Jackal and the Go2 carried out the commands. It says nothing about injury, and nothing about whether the bomb-handling trials used a real object or an inert one. When AI stops working is the question this paper never asks.

Bottom line: two real robots followed commands no engineer wrote. The video clips never moved at all.

If a prompt can drive a robot's legs, who signs off on the prompt?

Why that matters for every safety check on the robot's computer: P2.

Part 2 of 12 · When the AI drives the motor: Physical AI cybersecurity as a safety problem.