New domestic interactive world model, text and pictures can be used to generate a 3D world with one click

Zhidongxi Author | Yang Jingli Editor | Li Shuiqing Zhidongxi reported on August 18 that yesterday, HiDream.ai, a large multi-modal model company, released the native full-modal interactive world model HiDream-O1-World. This model supports multi-modal input such as text, image and interaction, and has three major functions: roaming, editing and interaction. Users can generate a complete world that is consistent in space and time, conforms to physical laws, and can be deduced for a long time with one click. In the third-party interactive world model evaluation benchmark WBench, HiDream-O1-World topped the Navi score list with an average score of 80.9. Among them, the model scored 73.3 points in the physical dimension, ranking first, and scored 88.0 in the consistency dimension. HiDream-O1 is equipped with the native all-modal (UiT) architecture developed by Zhixiang, which has achieved key breakthroughs in spatio-temporal consistency and physical consistency: when the camera is pushed, pulled and panned, the geometric structure of the scene space can still remain highly stable, and objects will not disappear or deform; collisions, occlusions, and gravity responses between objects are highly consistent with the causal logic of the real world, rather than randomly "guessed" picture splicing. In the actual experience, users can start from a piece of text or a picture, explore the generated scene from a first-person or third-person perspective, and control character actions, weather and object status. In the future, Zhixiang plans to use this model for AI interactive video games, embodied intelligent simulation and 3D scene production. In response to problems such as scene drift, object disappearance, and physical distortion that are prone to occur in interactive world models, this model writes 3D spatial information into the Memory context and continuously adjusts the model during the inference process through Test-Time Training (TTT) to maintain spatial and physical consistency under long-term interactions. It is worth noting that around spatial consistency, the research paper "DreamWorld: Towards a 3D Consistent World" jointly carried out by Zhixiang Future and Fudan University