The Epicenter of Spatial Intelligence

The unifying idea: perceiving, understanding, reasoning about, and acting in 3D space. XR applies it for people; physical AI applies it for machines.

Both bring computing into the physical world. Spatial computing centers on how people interact with it; physical AI centers on machines that perceive and act in it.

Software that interprets images and video. XR uses it to place content and read gestures; robots use it to decide what to do next.

Identifying objects, surfaces, people, and how they relate. XR needs it to place content believably; robots need it to act safely.

A map that carries meaning as well as geometry: not just "something is here," but "this is a door, a table, a person."

XR world understanding describes what is in the environment right now. An AI world model also predicts how the environment behaves and changes, especially after an action.

Systems that adapt their behavior to what the person, the machine, or the environment is doing at the moment.

Measuring distance with stereo cameras, lidar, time-of-flight, or structured light. XR uses it to place content on real surfaces; robots use it to navigate and grasp.

A set of 3D points captured from a scene; the raw input for reconstruction, mapping, and perception in both fields.

Combining cameras, motion sensors (IMUs), depth, and other inputs into one estimate that holds up when any single sensor struggles.

Inside-out tracking is an XR design choice: the sensors sit on the device, not in the room. VIO, which fuses camera images with motion sensors, is the technique that often powers it, and the same technique keeps drones and robots oriented.

Position (x, y, z) and orientation (pitch, yaw, roll) in 3D space. XR tracks the head, hands, and controllers; robots track themselves, their arms, and the objects they handle.

Both answer "Where am I?" XR tends to say tracking; robotics says localization.

Simultaneous localization and mapping: building a map while tracking your position in it. One of the clearest technical overlaps; the same algorithms run in headsets, robots, drones, and vehicles.

Finding your position on a known map at startup or after losing track. Robotics calls the hardest version the "kidnapped robot" problem.

Finding a precise position by matching what the camera sees against a prebuilt map, useful outdoors where GPS is not precise enough.

Identifying an object and following its position and motion over time.

Measuring each sensor's properties and how the sensors sit relative to each other. Without it, virtual content drifts and robot arms miss their targets.

The same geometry. In XR it usually means what the person sees through the display; in robotics, what a sensor can perceive.

The same kind of sensor stream with a different audience: XR shows it to a person, while a robot feeds it into perception models.

Creating a digital representation of a physical space, fundamental to both fields.

A surface model of the surroundings. XR uses it for occlusion and physics; robots use it for path planning and collision checks.

Capturing a real place or object as a machine-readable 3D model.

Photogrammetry is one method of 3D reconstruction: rebuilding geometry from many overlapping photos.

Learned representations of real places that render photorealistically. XR uses them for viewing and sharing; physical AI uses them as realistic training scenes.

A persistent reference point that ties information to a physical location.

Remembering a space across sessions and over time instead of rediscovering it each time.

One persistent map that many devices or robots share and keep up to date.

The same phenomenon for different purposes: XR must hide virtual objects behind real ones, while robots must reason about what they cannot see.

The physical area a user or a robot can safely move and work in.

A defined boundary that a person or machine must stay inside, or outside.

A virtual replica of a real asset, facility, or process, connected to real-world data. XR lets people walk through and review it at human scale; physical AI uses it to simulate, monitor, and control.

Often the same technical artifact seen through different use cases: an experience for people, a training ground for machines.

XR simulates environments for people to experience; physical AI relies on simulation to develop, train, and test machines before they touch the real world.

Data generated from simulated worlds to train perception and AI systems without capturing everything in reality.

The same simulated world can train both people and machines, one of the strongest convergence stories.

First-person video and sensor data. Smart glasses capture it naturally, and robotics teams increasingly use it to teach robots how people do tasks.

Both record real human movement in 3D. XR replays it for viewers; robotics uses it as examples for robots to learn from.

XR asks how a person interacts with the environment; robotics asks how a machine acts on it and manipulates it.

The same vision problem: locating hands and joints. XR turns it into input; robotics also retargets captured human motion onto a robot's hands.

Both work out where a person is looking. XR uses it for input and rendering; robots use it to anticipate what a person will do next.

XR delivers touch to people; robots sense touch themselves. In teleoperation they meet: the robot senses force, and the operator feels it.

XR input devices can become intuitive robot controls; precision teleoperation often uses a matching "leader" arm.

Telepresence puts a person virtually somewhere else; teleoperation lets them act there through a machine.

A person stepping in remotely when a worker or a machine needs judgment.

XR starts with the human as the user. Physical AI increasingly needs ways for people to supervise, guide, and intervene.

Both design how people interact with information and physical systems. XR can become the interface through which people understand and direct physical AI.

AR can show what a machine perceives, predicts, or plans, right where it will happen.

XR represents a person in a digital space and asks whether they feel there. Embodied AI describes intelligence acting through a physical body. Related ideas, running in opposite directions.

Several people or machines sharing and coordinating within one space.

XR places sounds in space for people to hear; robots locate and interpret the sources of sounds.

Smart glasses give people machine perception and intelligence, the parallel path to giving machines those abilities.

XR makes people more capable; physical AI makes machines more capable. Where they meet is human-machine collaboration.

Not equivalents, but they play the same role: shared layers that reduce fragmentation and let developers build across hardware.

The software layer between applications and hardware. Different stacks, similar architectural role.

Running AI models on the device rather than in the cloud, for speed and privacy on glasses and for real-time decisions on robots.

The time from sensing a change to responding to it. Too slow in XR causes motion sickness; too slow in a robot causes unstable or unsafe motion.

XR developers design experiences; robotics teams define tasks and train the policies that determine how a machine behaves.