The news. On July 8, 2026, Mistral released Robostral Navigate, an 8B model that navigates from a single RGB camera plus a natural-language instruction — no depth sensor, no LiDAR. Trained entirely in simulation on 400,000 trajectories across 6,000 scenes, it drives wheeled, legged, and flying robots and reports 76.6% success on R2R-CE unseen routes. Read the release →
Picture a passenger in a car who never reads out GPS coordinates. They just look through the windshield and point: pull up by that blue awning. Robostral Navigate drives a robot the same way. It looks at one camera frame and marks a single pixel in that image as its next waypoint — a spot on the floor to head for — plus which way to face when it arrives. No laser scan, no depth channel; its next move is a point inside the picture it already sees, one perceive-decide-act tick at a time. The action is a place in the image, not a command to the motors.
The catch with pointing is that you can only point at what you can see. When the goal sits behind the robot or around a corner — off the current frame — Robostral falls back to a local-frame displacement: a small move defined relative to the robot's own body, pausing to look again until the target swings back into view and it can point once more. That two-mode action space is what lets one 8B model steer wheeled, legged, and flying robots from the same policy: a pixel-in-the-image and a rough heading mean the same thing whether you roll, walk, or fly, while raw wheel or rotor commands would not. Choosing a point in the picture is the part that transfers across bodies.
So where does a model learn to point at the right pixel? Robostral starts from a grounding-focused vision-language model — one already good at lining up what a phrase asks for with what the camera shows — and repurposes that pointing skill as an action. It then trains entirely in simulation on 400,000 rendered trajectories, and finally sharpens itself with online reinforcement learning, which the paper credits with a 3.2-point bump in success rate. Grounding gives it the eye, simulation gives it the miles, and reinforcement learning files down the mistakes.
| Navigation approach | What the model outputs | Sensors it typically needs | One policy across robot bodies? |
|---|---|---|---|
| Metric-map planning (classic VLN) | waypoints inside a metric 3D map | usually RGB-D or LiDAR + a mapping / SLAM stack | tied to the mapping stack |
| Low-level control policy | raw velocity or motor commands | usually robot-specific | often retuned per robot |
| Pixel-waypoint (Robostral Navigate) | a point in the RGB image + an orientation | one ordinary RGB camera | yes — wheeled, legged, and flying |
Why packing the episode saves 22× the training tokens
Training a navigation policy the naive way treats every step of an episode as its own sample, and each sample re-reads the whole history so far. Walk that out for one illustrative 43-step episode: step 1 encodes 1 observation, step 2 re-encodes 2, step 3 re-encodes 3, all the way to step 43 — a total of 1 + 2 + … + 43 ≈ 946 observation-encodes for a single episode. Robostral instead packs the whole episode into one sequence and reuses the shared history with prefix caching and tree-based attention masking, so each of the 43 observations is encoded once — 43 encodes, not 946. That ratio is about 22×, exactly the training-token saving Mistral reports. (The 22× is the paper's figure; the 43-step episode and the arithmetic are an illustrative walk-through of why packing yields that scale — turning a quadratic re-encode into a linear one — not published per-episode numbers.) Cheap training is what makes 400,000 simulated trajectories across 6,000 scenes affordable in the first place.
Goes deeper in: AI Agents → Agent Loop & State → Inside a Tick — the step that frames each agent action as one perceive-decide-act cycle, exactly what a single navigation move is here.
Related explainers
- S-Agent — spatio-temporal evidence accumulation — another agent that turns looking around a scene into a reliable action, rather than answering from a single glance
- SpatialClaw — code-as-action vs structured tool-calls — the same core question in a different domain: what should an agent's action actually be?
- ViQ — text-aligned visual tokens — how an image becomes tokens a language model can ground and act on — the machinery under Robostral's "point at a pixel"