Policy optimization
The world model predicts visual outcomes for candidate actions. Comparative feedback guides policy optimization with GRPO.
RIWANavRecursive World–Action Models with Self-Improvement
for Urban Navigation
1 MoE Key Lab of Artificial Intelligence, Institute of AI, School of Computer Science,
Shanghai Jiao Tong University, Shanghai, China
2 Beihang University, Beijing, China3 The Hong Kong University of Science and Technology, Hong Kong, China
4 Tsinghua University, Beijing, China5 The University of Texas at Austin, Austin, USA6 Intelligent Game and Decision Lab (IGDL), Beijing, China
Overview
RIWANav jointly improves a world model and a navigation policy. The world model predicts the outcomes of candidate actions to guide policy optimization. The updated policy then selects reliable training examples for world-model adaptation.
RIWANav is a post-training framework for language-guided urban navigation. It alternates policy optimization with world-model adaptation, keeping the two aligned as the agent learns. World-model updates use recorded future frames from the offline dataset.
Demo
UrbanNav replays and four physical-robot trials. Physical trials are shown at 1.5× speed.
Method
World-model adaptation
The improved policy proposes expert-consistent action–video pairs. Behavioral novelty and prediction error determine which examples refine the world model.
Expert-consistency filtering keeps proposed actions close to recorded trajectories. The selected actions are paired with recorded future frames, providing supervision for the next world-model update.
Results on UrbanNav
We evaluate RIWANav on the official test-seen and test-unseen splits, comparing training frameworks and representative navigation methods.
Test-unseen environments
88.25%
Success rate ± 0.43
82.59%
SPL ± 0.25
2.04m
Navigation error ± 0.08
+7.94 percentage points in success rate over GRPO with a frozen world model.
Swipe to view all metrics →
| Training framework | SR (%) ↑ | SPL (%) ↑ | NE (m) ↓ |
|---|---|---|---|
| Imitation learning | 67.88 | 66.03 | 4.4537 |
| PPO | 67.94 ± 0.09 | 65.65 ± 0.16 | 4.3768 ± 0.0434 |
| GRPO | 70.63 ± 1.10 | 68.49 ± 1.06 | 4.5795 ± 0.0894 |
| PPO + frozen WM | 75.05 ± 0.21 | 72.44 ± 0.19 | 3.5912 ± 0.0462 |
| GRPO + frozen WM | 80.31 ± 0.94 | 76.09 ± 1.05 | 3.3314 ± 0.1014 |
| RIWANav Ours | 88.25 ± 0.43 | 82.59 ± 0.25 | 2.0429 ± 0.0815 |
Mean ± standard deviation over three seeds for post-training methods; imitation learning is the reference initialization. SR: success rate. SPL: success weighted by path length. NE: navigation error. Success threshold: 3 m. These are offline UrbanNav results; the physical-robot demonstrations are qualitative examples.
Curriculum ablation
On test-unseen, success rate rises from 84.83% with uniformly sampled policy-induced updates to 88.25% with the grounded self-curriculum. Updating on static expert data alone reaches 78.23%.
World-model prediction quality
After adaptation, LPIPS decreases from 0.4539 to 0.4217 on a fixed set of 128 held-out policy-induced action–video pairs, evaluated over an 8-second prediction horizon.
Citation
arXiv:2610.08640 · 2026
@misc{riwanav,
title = {RIWANav: Recursive World--Action Models with
Self-Improvement for Urban Navigation},
author = {Xie, Jing and Ruan, Shouwei and Wang, Yubin and
Zhang, Yuxiang and Yang, Haitao and Jin, Songchang and Shi, Dianxi},
year = {2026},
eprint = {2610.08640},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.08640}
}