LaLa-World: Scaling Latent World Models
with Natural Language Actions

TL;DR

A Language-conditioned Latent World model (LaLa-World) uses natural language as a shared action interface to learn from simulation, robotics, images, and videos. It predicts future visual representations for efficient planning across domains and can be fine-tuned into LaLa-Policy, which jointly predicts robot actions and future representations.