What is embodied AI? How robots turn camera images into movements

Embodied AI is artificial intelligence that controls a physical body. It takes in what a robot’s cameras and sensors report, chooses a movement, drives the motors, then looks again to see what changed, in a loop that repeats many times a second. A chatbot runs a different loop: text in, text out, and a person reads the answer before anything happens. The newest robot models are built from the same family of models as chatbots, but their output is a stream of motor commands, so a mistake can knock over a glass instead of producing a wrong sentence.

The term covers robot vacuums, robot arms and humanoid robots alike. This guide focuses on the approach labs and robot makers have published since 2023: one large model that reads images and an instruction and outputs movements.

How does a robot go from seeing to moving?

Every robot runs some version of a perceive, decide, act cycle. Perception comes from cameras, depth sensors, lidar, an IMU for balance and encoders that report joint angles. Combining these readings into one estimate of where the robot is and what surrounds it is sensor fusion. The decision step picks the next motion; the action step sends commands to actuators, the motors that move joints and wheels. Then a fresh observation starts the cycle again, which is how a robot can notice that a cup slipped.

For decades the decision step was a chain of hand-written modules: recognize the object, plan a path, compute a grasp. Embodied AI in the current sense replaces much of that chain with a model trained on recordings of robots doing tasks. Not all of it: Google DeepMind’s technical report on Gemini Robotics says models like it need to be connected to safety-critical lower-level controllers, the conventional software that keeps motors within safe limits.

The perceive, decide, act loop with a VLA model in the decision step and safety controllers before the motors; below, where four published systems run the model, from the cloud (RT-2) to entirely on the robot (Helix, Gemini Robotics On-Device).

What is a vision-language-action model?

A vision-language-action model (VLA) takes camera images and an instruction in plain language, such as “put the apple in the bowl”, and outputs robot actions, a short sequence of arm, gripper or joint positions at a time. Most start from a vision-language model, the kind that describes images after training on web data, and are trained further on robot recordings, so that knowledge from the web, such as what an apple looks like, carries over to control.

Google DeepMind named the category with RT-2 in July 2023, a model that writes robot actions as text tokens. Others followed, and the most useful difference between them for a reader is the source: a peer-reviewed paper was checked by other researchers before publication, a preprint was not, and a company announcement is the company’s own description.

Model	Who	First public	Source	Status
RT-2	Google DeepMind	28 July 2023	arXiv paper	Peer-reviewed, CoRL 2023
OpenVLA	Stanford-led team	13 June 2024	arXiv paper, open weights	Peer-reviewed, CoRL 2024
π0	Physical Intelligence	31 October 2024	arXiv paper, company blog	Peer-reviewed, RSS 2025
Helix	Figure	20 February 2025	Company announcement	No paper
Gemini Robotics	Google DeepMind	12 March 2025	Company announcement, arXiv report	Preprint
GR00T N1	NVIDIA	18 March 2025	Company announcement, arXiv paper	Preprint

The table shows the first generation of each model; every one has newer versions. Google DeepMind announced Gemini Robotics 2 on 30 July 2026 with a safety technical report, the VLA open only to early-access partners. Physical Intelligence followed π0 with π0.5 (22 April 2025, an arXiv preprint), π*0.6 (17 November 2025) and π0.7 (16 April 2026), the last two also as arXiv preprints (π*0.6, π0.7). Figure announced Helix 02 (27 January 2026), adding whole-body control and a balance layer that runs 1,000 times a second, and Helix 2.5 (17 September 2026), neither with a paper. NVIDIA lists GR00T N1.7 as a general availability release on its GitHub page, without a separate paper. In between, Gemini Robotics 1.5 (25 September 2025) added a reasoning model that plans the steps of a task and passes instructions to the VLA.

Where does the computing happen: on the robot or in the cloud?

An answer that takes a second is fine for a chatbot and too slow for a hand closing on a cup. That argues for running inference on the robot, as edge AI does for cameras (see Edge AI explained), but the largest models do not fit. The published systems differ:

Cloud. The RT-2 authors write that their 55-billion-parameter model could not run on the GPUs usually mounted on robots, so they queried it on a cloud service over the network, at 1 to 3 decisions per second. In October 2026 Physical Intelligence opened a beta API whose showcased applications mostly ran with cloud-hosted inference; the company says some early deployments hit network connectivity problems.
Desktop GPU. OpenVLA, with 7 billion parameters, runs at about 6 decisions per second on one NVIDIA RTX 4090 graphics card, according to its paper.
Split. In the Gemini Robotics report, the main model is hosted in the cloud and a local action decoder runs on the robot, with about 250 milliseconds from observation to action. Gemini Robotics On-Device, announced on 24 June 2025, runs on the robot without a data network, Google DeepMind says, initially for selected testers.
On board, two speeds. Figure says Helix runs entirely on two embedded GPUs: a 7-billion-parameter model reads scene and instruction 7 to 9 times a second, and an 80-million-parameter policy outputs movements 200 times a second. The GR00T N1 paper describes a similar slow and fast pair, but its reported speeds are for an NVIDIA L40, a data-center GPU, rather than a computer on the robot. For GR00T N1.7, NVIDIA’s GitHub page lists about 28 decisions per second on its Jetson AGX Thor embedded module.

A cloud model can be larger, but it needs a working connection, adds network delay, and camera images have to leave the house to reach it. A model on board works offline and keeps images local, but it is limited by the power, heat and battery a robot can carry, which is why makers turn to embedded GPUs and NPUs. What a home robot sends out, and who sees it, is covered in the guide to home robot privacy and teleoperation.

How do robots learn new tasks?

Mostly from demonstrations. In teleoperation, a person drives the robot remotely and the recorded images and motions become training examples. Figure says Helix was trained on about 500 hours of teleoperated data. OpenVLA used 970,000 robot demonstrations from the Open X-Embodiment collection, pooled from 21 institutions and 22 robot types. Once a general model exists, a new task needs fewer examples: the Gemini Robotics report describes new short tasks learned from as few as 100 demonstrations.

Simulation adds synthetic examples: NVIDIA says its tools generated 780,000 synthetic trajectories in 11 hours, and that mixing them with real data improved GR00T N1. Videos of people are another source: the GR00T N1 paper lists human videos among its training data.

Data is the bottleneck because there is no web of robot motions. Text and images exist in enormous quantities; recordings of a robot folding a towel exist only if someone made them. The RT-2 paper states the consequence: web pretraining helped the robot handle new objects and instructions, but its physical skills stayed limited to those in its robot data.

What can’t embodied AI do yet?

The clearest limits come from the authors themselves:

RT-2: no new motions beyond the robot data, and a computing cost that may become a bottleneck for fast control.
OpenVLA: one camera image at a time, too slow for robots that need 50 commands a second, and success rates on the tested tasks typically below 90%.
π0: Physical Intelligence calls generalist robot policies “still in their infancy” and lists robustness and safety among open problems.
Gemini Robotics: the most dexterous tasks remain hard to learn in its multi-task setup, and real-world evaluations are affected by changing environments and deteriorating hardware.
Helix: Figure calls its results early.

These success rates come from tasks and setups the authors chose. Makers have started to publish tests in real homes. Figure reports blind trials of Helix 2.5 in 30 Bay Area homes where it had collected no data, on three chores it picked (tidying toys, folding towels, making a bed): the robot completed the whole task in 56% of trials, against 9% for a version trained without Figure’s large pretraining dataset, and Figure writes that “the point is not that general humanoid robotics is solved”. Physical Intelligence says π0.5 can clean up homes it never saw in training but “often makes mistakes”. These are the makers’ own tests, short trials rather than months of daily use. Demonstration videos show what worked on a given day, not how often it fails, and whether a robot was remote-controlled is known only if the maker says so. None of this means the robot understands a kitchen: it outputs movements learned from its training examples.

Where to go next

Can you buy a humanoid robot for your home? covers prices and deliveries.
Home robots, cameras and remote operators covers who sees inside your home.
What is AIoT? places robots among AI on connected devices.
On-device computer vision explains how cameras turn pixels into decisions locally.

Topics: robotics, embodied AI, VLA, edge AI