Whole-Body Conditioned Egocentric Video Prediction

modal { display: none; position: fixed; z-index: 9999; padding-top: 50px; left: 0; top: 0; width: 100%; height: 100%; overflow: auto; background-color: rgba(0,0,0,0. 9); }. modal-content { margin: auto; display: block; max-width: 90%; max-height: 90%; }. close { position: absolute; top: 15px; right: 35px; color: #f1f1f1; font-size: 40px; font-weight: bold; transition: 0. 3s; cursor: pointer; }. close:hover,. close:focus { color: #bbb; text-decoration: none; cursor: pointer; }.

clickable-img { cursor: zoom-in; transition: opacity 0. 3s; }. clickable-img:hover { opacity: 0. 9; } @media only screen and (max-width: 700px){. modal-content { width: 100%; } } × document. addEventListener('DOMContentLoaded', function() { var modal = document. getElementById('imageModal'); var modalImg = document. getElementById('modalImg'); var span = document. getElementsByClassName('close')[0]; // Add click handler to all images in the post var images = document. querySelectorAll('.

post-content img, article img'); images. forEach(function(img) { // Make all images clickable img. classList. add('clickable-img'); img. title = 'Click to enlarge'; img. onclick = function() { modal. style. display = 'block'; // Use the original high-res version if it exists var highResSrc = this. src. replace('_web. png', '. png'); modalImg. src = highResSrc; modalImg. onerror = function() { // Fall back to the web version if high-res doesn't exist modalImg. src = img.

src; }; } }); // Close modal when clicking the X span. onclick = function() { modal. style. display = 'none'; } // Close modal when clicking outside the image modal. onclick = function(event) { if (event. target == modal) { modal. style. display = 'none'; } } // Close modal with ESC key document. addEventListener('keydown', function(event) { if (event. key === 'Escape') { modal. style. display = 'none'; } }); }); Predicting Ego-centric Video from human Actions (PEVA).

Given past video frames and an action specifying a desired change in 3D pose, PEVA predicts the next video frame. Our results show that, given the first frame and a sequence of actions, our model can generate videos of atomic actions (a), simulate counterfactuals (b), and support long video generation (c). Recent years have brought significant advances in world models that learn to simulate future outcomes for planning and control.

From intuitive physics to multi-step video prediction, these models have grown increasingly powerful and expressive. But few are designed for truly embodied agents. In order to create a World Model for Embodied Agents, we need a real embodied agent that acts in the real world. A real embodied agent has a physically grounded complex action space as opposed to abstract control signals.

They also must act in diverse real-life scenarios and feature an egocentric view as opposed to aesthetic scenes and stationary cameras. 💡 Tip: Click on any image to view it in full resolution. Why It’s Hard Action and vision are heavily context-dependent. The same view can lead to different movements and vice versa. This is because humans act in complex, embodied, goal-directed environments. Human control is high-dimensional and structured.

Full-body motion spans 48+ degrees of freedom with hierarchical, time-dependent dynamics. Egocentric view reveals intention but hides the body. First-person vision reflects goals, but not motion execution, models must infer consequences from invisible physical actions. Perception lags behind action. Visual feedback often comes seconds later, requiring long-horizon prediction and temporal reasoning. To develop a World Model for Embodied Agents, we must ground our approach in agents that meet these criteria.

Humans routinely look first and act second—our eyes lock onto a goal, the brain runs a brief visual “simulation” of the outcome, and only then does the body move. At every moment, our egocentric view both serves as input from the environment and reflects the intention/goal behind the next movement. When we consider our body…