SOTA · vision

3D & World Models

3D asset generation, scene reconstruction, long-horizon world prediction

3D and world models sit closest to AGI inside visual generation, because the output is structure rather than pixels: meshes, voxels, radiance fields, and predictions of the form "if I take this action in this state, the world becomes that". 3D asset generation turns flat creative work into objects that go into an engine, take lighting and hold up from any angle - the entry point for games, industrial design and digital twins. World models let an agent rehearse the consequences of action in its head, which is the precondition for embodied intelligence and long-horizon planning. Both rest on the same geometric understanding, which is why they share one domain: an asset side (single objects, topology, materials, editability) and a scene-and-prediction side (reconstruction, long-horizon consistency, action-conditioned generation). Progress here is judged by geometric correctness, not by how good the still render looks - a beauty shot can lie, a turntable cannot.

RULERGeometric consistency, scene reconstruction, long-horizon prediction

The ladder

3D & World ModelsGame MakingTopC

Meshy 7: the 3D asset platform that finishes what comes after generation, from topology and PBR to rigging, printability, and MCP

Meshy headlines Meshy 7 (2026) and its "true-to-reference" claim, meaning alignment between the generated model and your input image, with image-to-3D stated at under a minute. The real differentiator is not generation but finishing the engineering constraints that follow it: Smart Topology returns clean topology with a controllable poly count in about ten seconds, one-click PBR delivers the full albedo, normal, metallic, and roughness set at HD 4K with no shader setup, auto-rigging covers humanoid and quadruped with 600+ motion presets, and on the print side meshes come out watertight with auto-repair before export. It exports GLB, FBX, OBJ, STL, USDZ, and 3MF straight into OrcaSlicer and Bambu Studio. The agent surface is complete too: a REST endpoint per generation capability, an MCP server that connects into Claude or Cursor, and a CLI that turns one command into a finished model. Stated scale is 100M+ models and 12M+ users, with 100 free credits a month. Boundaries: no public quantitative benchmark (the quality stars are self-assigned), credit pricing per tier undisclosed, and "under a minute" excludes post-processing.

<1minVendor claim: single image → 3D modelVendor Claim · 2026

ProductMeshySite

3D & World ModelsGame MakingTopC

Tripo: image-to-engine-ready-asset compressed into a ten-second step

VAST's Tripo is the most productionised commercial text/image-to-3D service tracked here: meshes with UVs and PBR materials, quad remeshing, an API plus Blender/Unity plugins that drop straight into game and e-commerce asset pipelines. Its weakness is equally clear - strong on single objects, weak on complex assemblies and exact dimensions - and the sub-10-second figure is the vendor's own claim, which has not been re-measured.

<10sVendor claim: single image → usable meshVendor Claim · 2025-06

ProductVASTSite

3D & World ModelsDesign & FrontendTopC

Hunyuan3D 2.1: the open-weight 3D generation baseline and default starting point for self-hosting

Tencent released weights for both the shape and texture stages plus PBR material output, which gives 3D generation its first reference implementation you can run on your own machine, fine-tune, and wire into an internal pipeline. Its value is not a leading score but reproducibility: any claim about 3D generation quality can now be measured against it.

ProductTencentSite Repo

Image GenerationDesign & FrontendTopC

FLUX: the line that moved image generation from "produce a picture" to controllable editing with open weights

Black Forest Labs' FLUX family generates images with a rectified-flow transformer and ships Apache-2.0 dev weights, making it the most important open baseline on the image side; the Kontext tier turned "edit this part of the image on instruction without disturbing the rest" into a usable capability. The collected record also covers the FLUX 3 launch, which the vendor positions as a world model unifying image, video and audio - a claim log as unverified.

Apache-2.0Open-weight tier (dev)Vendor Claim · 2024-08

ProductBlack Forest LabsSite Repo

Image GenerationGame MakingC

Scenario: style-consistent asset volume for game art, not one beautiful picture at a time

Scenario packages image generation as an asset line for game studios: train a stylised generator on your own art, then produce icons, props, characters and set pieces in volume, wiring generation into tools like Blender through MCP and plugins. Its claim is consistency and throughput - precisely the two things general text-to-image models are worst at.

ProductScenarioSite

3D & World ModelsDesign & FrontendD

TRELLIS: a research baseline that unifies 3D asset representations through a sparse structured latent

Microsoft Research's TRELLIS introduces a sparse structured latent (SLAT) that lets one generated result decode into a radiance field, 3D Gaussians or a mesh, splitting generation into "first predict the sparse voxel structure, then generate features on active voxels". It is a paper-grade asset that matters for understanding the current 3D generation route, not a tool for shipping product.

ResearchMicrosoft ResearchRepo

3D & World Models

MCP for Blender: 32 tools that make Blender a 3D workspace an LLM can drive

Formerly blender-mcp, a third-party community plugin (the README states on its first screen that it is not made by Blender), MIT licensed, Python, and the highest-starred entry among 3D MCP servers. The PyPI package is now mcp-for-blender, yet uvx blender-mcp still runs and existing configs need no change. Two components only: an addon inside the Blender process runs a socket server that actually executes commands (the only place with bpy, the current scene and the UI thread), while a separate MCP server process translates protocol, joined by JSON over TCP on localhost:9876 by default. The source defines 32 @mcp.tool() decorators across four layers - perception (get_scene_info, get_object_info, get_viewport_screenshot returning an MCP Image, closing an act-then-look loop), reference lookup (bpy_api_lookup and describe_node_type so the model queries instead of guessing socket order and enum names), execution (execute_blender_code as the escape hatch, set_texture, export_scene to GLB/FBX), and asset retrieval and generation (Poly Haven with semantic search, importing the .blend source and writing attribution properties; Poly Pizza with about 10,600 low-poly models, size normalisation and automatic CC-BY credit; Sketchfab; Hyper3D Rodin text-to-3D and image-to-3D; Tencent Hunyuan3D distinguishing the mainland ai3d endpoint from the international hunyuan one). On security, arbitrary code execution is the design rather than a bug: an optional safe mode (BLENDER_MCP_SAFE_MODE=1) statically validates scripts before running, blocking file access, subprocesses, network and persistent code while modelling, rendering, saving, import and export pass normally, and blocked scripts return to the model with the reason so it can retry. Content telemetry is off by default, and a dedicated disable_telemetry tool can only turn collection off. 29.2k stars. Not run with Blender locally yet, so this is graded as pending reproduction.

29kGitHub stars

Site Repo

3D & World Models

Text-to-CAD: the open-source plugin and skill suite that gives coding agents CAD superpowers

text-to-cad turns parametric CAD into a local agent skill: plain language or an image becomes a STEP model with STL/3MF/GLB exports, plus DFM review, dimensioned drawings, URDF/SRDF/SDF robot descriptions and 3D printing, sheet metal and CNC integrations. Kernel is build123d on OCP; the shape is twelve skills plus the cadgen engine and a local CAD Viewer. MIT, about 20.6k stars.

21kGitHub stars

Site Repo

3D & World Models

img2threejs: Rebuilding Reference Images as Code-Only Procedural Three.js Models

An agent skill for Claude Code, Codex and OpenCode that rebuilds a reference image as a code-only procedural Three.js model: an eight-pass sculpting pipeline with deterministic gates, shipping a diffable TypeScript factory instead of a mesh.

15kGitHub stars

Site Repo

3D & World Models

Modly: Local, Open-Source Image-to-3D Mesh Generation Desktop App

Modly is an open-source desktop app from Lightning Pixel (Windows, Linux, Apple Silicon macOS; v0.4.3; MIT plus an attribution clause) that turns a photo into a usable 3D mesh with open models running on your own GPU, so inference never leaves the machine. An Electron shell supervises a Python FastAPI backend bound to 127.0.0.1:8765. The default model is TripoSR (about 2.4 GB); Hunyuan3D 2 Mini, TripoSG and Trellis2 GGUF arrive as extensions, where an extension is a GitHub repo with a manifest.json that can declare multi-repo model_sources, shared weight_groups and separately installable weight_variants carrying size_gb and vram_gb. Generation is expressed as a node-graph workflow (Image to Generate Mesh to Add to Scene), scenes follow the modly.scene-manifest.v1 contract, and export covers glb/stl/obj/ply with a dedicated stl/obj printing path that scales to millimetres for OrcaSlicer. Three agent surfaces ship in the repo: a stdlib-only JSON-first CLI with its own SKILL.md, an MCP server exposing nine modly_* tools, and an in-app tool-calling agent that can run on a local llama.cpp GGUF pool or an external model endpoint. AMD ROCm auto-detection, memory-budgeted Apple Silicon workflows and headless Jetson runs are documented with measured numbers.

8.1kGitHub stars

Site Repo

3D & World Models

image-blaster: blast one image into a collidable 3D world

A set of Claude Code skills that turns one image into an explorable Gaussian-splat environment with a collider mesh and metric scale, plus interactable object meshes and sound effects, in under five minutes.

5.1kGitHub stars

Repo

3D & World Models

lidar_localization_ros2: map-based 3D LiDAR localization for ROS 2 that finds its own start pose

Give it a point-cloud map and a LiDAR stream and it publishes the robot pose in the map frame: NDT/GICP backends, guarded IMU preintegration, continuous-time deskew and Nav2 integration; branch-and-bound global initialization at cold start and a guarded supervisor that re-seeds after lost tracking, with public-dataset benchmarks documented.

580GitHub stars

Repo

3D & World Models

image-to-3d-pipeline: A Fair Benchmark and Browser Explorer for Image-to-3D Models

A fixed-protocol harness for comparing image-to-3D models. One set of AI renders of a fictional submersible goes into TRELLIS V1/V2 and TripoSR; each mesh is inspected through an identical Blender 4.3 EEVEE orbit and scored on a 0-2 five-axis rubric, then the winner ships into a Three.js + three-mesh-bvh explorer with real collision. TRELLIS V2 stochastic ranks first at 1.56 MB / 11,866 faces.

258GitHub stars

Repo

Embodied AI

Genesis World: The Unified Multi-Physics Simulation Platform for Physical AI

Genesis AI's open simulation platform for physical AI: one Pythonic API over a multi-physics engine (rigid, FEM, MPM, PBD, SPH, Stable Fluid, IPC), the Nyx path tracer, and Quadrants, a cross-platform Python-to-GPU compiler. It is built to be the evaluation engine for robot foundation models rather than a data factory: 30,000 parallel environments at 43M FPS, and under a zero-shot real-to-sim protocol, sim-vs-real evaluation agreement of Pearson 0.8996 with mean maximum rank violation 0.0166. Apache 2.0, 29.9k stars.

30kGitHub stars

Site Repo

Latest demos

261

0:52

Papers & deep reads

2026-11-01Building a Stage-Ready 3D Anime Music Video with GPT, Muse, and Opus 5.5Hailey combines GPT for design, Muse for dance footage, and Opus 5.5 for Three.js keying and stage VFX, turning generated clips into editable, beat-synced music-video assets.

2026-10-29L3G-World: Real-Time Infinite World Generation with Latent 3D Gaussian MemoryL3G-World maintains a compact latent 3D Gaussian memory that adds primitives only in unseen regions and refines existing ones. Rendering the memory directly into diffusion latent space avoids VAE re-encoding and sustains long-horizon consistency at 9.02 FPS on one H100.

2026-10-29NanoDreamer: A 750k-Parameter World Model That Learns to Act Through ImaginationBuilt from a custom autograd engine to a 750k-parameter Dreamer, NanoDreamer trains its actor entirely in imagined rollouts and beats random play across all five MinAtar games. The write-up keeps the failed first attempt, reward-timing diagnostics, critic optimism, CNN/multi-step rebuild, and ablations.

2026-10-26From One Prompt to a Rigged 3D Performance: Inside prompt2rig's Video-to-Mixamo Pipelineprompt2rig turns a text prompt into a generated performer clip, then reconstructs full-body and finger motion with browser MediaPipe and local GVHMR/WiLoR before retargeting to Mixamo and exporting GLB.

2026-10-23UniAction: Unified Action Conditioning for World ModelsActions that condition world models are highly heterogeneous, and thus existing action-conditioned world models are typically built for individual action types, fragmenting already limited action-annotated data across separate models. We introduce UniAction, a unified action-conditioning framework that represents any action as a sparse set of action elements, each with a discrete semantic identity and a continuous spatiotemporal grounding coordinate. By mapping these coordinates directly into the pretrained video model's native positional space, UniAction allows heterogeneous actions to condition a shared world model within the same spatiotemporal structure. We further introduce cardinality-calibrated attention to address the bias from differing action representation granularities. We evaluate UniAction on four representative action forms: force, camera control, hand pose, and robot pose. UniAction outperforms conditioning alternatives in action following, and a single jointly trained model surpasses the action-specific experts, showing that heterogeneous action data benefit from unified model and joint training.

2026-10-23RynnUnify: Towards Unified Embodied BrainWe introduce RynnUnify, a 16B unified embodied model that combines strong embodied understanding with long-horizon visual planning through interleaved video--language reasoning. With capabilities spanning spatial reasoning, physical prediction, and grounding, RynnUnify connects an understanding of the physical world with visual anticipation of how it evolves across successive subtasks. Building on these capabilities, we further develop an agentic extension through reinforcement learning to selectively use video predictions for reasoning about unobserved states and interaction outcomes. We also develop a world-action extension of the full RynnUnify model, translating its understanding and prediction capabilities into robot control. To evaluate these capabilities, we introduce Rynn-VP and Rynn-GEU for long-horizon visual planning and anticipatory reasoning. Experiments show that RynnUnify achieves the highest average embodied understanding score among the evaluated unified multimodal models and improves the Rynn-VP overall score by 9.75 points over Cosmos 3 Nano. Its agentic extension improves the Rynn-GEU average by 6.0 points over direct reasoning, while the world-action extension achieves 51.5\% success on 20 selected composite RoboCasa365 tasks and 46.7\% average success across three real-robot tasks. Together, these results establish RynnUnify as a shared foundation for embodied understanding, visual planning, and robot control, taking a step toward a unified embodied brain.

2026-10-21Agentic World Action ModelsRecent world action models provide visual foresight and future prediction for robot manipulation, but lack explicit mechanisms for compositional skill reuse. Agentic robotics combines code-based planning and reasoning with reusable skills, but often lacks predictive grounding in physical dynamics. We introduce AW-0, an agentic world action model framework that integrates code-based planning with joint video and action prediction. We jointly model code, future video, and robot actions as three coupled generative streams within a unified mixture-of-transformers architecture. Within this architecture a code expert generates parameterized skill routines and control logic while video and action experts generate future visual states and low-level motor commands. Code serves two complementary roles: it structures multi-step prediction and control within the world model and represents skills as reusable programs for subsequent agentic execution. We further introduce an automated curation pipeline that labels robot and human video trajectories with aligned programs and consolidates recurring routines into reusable skill libraries. Evaluations on RoboTwin 2.0 and physical bimanual manipulation tasks show improved task performance over VLA and WAM baselines, alongside gains in language following, semantic reasoning and robustness, demonstrating that agentic code enables connection between planning, prediction and execution within WAMs.

2026-10-2010 Professional Resin 3D Printers for Commercial and Light ManufacturingA buyer's guide to ten professional resin printers from $4,500 to ~$42,000: build volumes, LCD/DLP/SLA optics, open-material ecosystems and wash/cure workflows that decide real throughput.

2026-10-15Can World and Action Modeling Benefit Each Other under Unified Causal Training?World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action-conditioned world modeling. We use LIBERO, LIBERO-Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real-world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi-view world-prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.

2026-10-13Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied IntelligenceWorld models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics, geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules rather than being jointly represented and evolved within a shared dynamics model. In this work, we present Thinker-WM, a unified multimodal world model that jointly models and generates RGB, metric depth, and robot action through a shared intermediate Unified Dynamics Transformer with modality-specific encoders and decoders. The model performs multimodal unification at the representation level: appearance, geometry, and action are transformed into a common dynamic representation and jointly evolved by shared parameters, while their distinct output structures are preserved. The same parameterization supports multiple functions, including joint video-action generation, world prediction, policy inference, action-conditioned generation, video-to-action generation, and inverse dynamics. To examine whether the shared representation captures action-related information, we introduce a lightweight motion probe that shows the frozen visual representation contains recoverable motion information. Thinker-WM is trained on a heterogeneous corpus comprising approximately 20K hours of egocentric video, 10K hours of real-robot data, and 1K hours of simulation data, including approximately 4K hours of real-robot data with aligned depth supervision. Extensive experiments demonstrate strong performance across embodied world-model evaluations. Thinker-WM ranks first on EWMBench, WorldOlympiad and WorldArena, while controlled studies further demonstrate the benefits of action conditioning, geometric interaction, and multi-function training within the shared representation.

2026-10-13Data-Free Continual Learning for World Action Model with Rectified Generative ReplayWorld action models (WAMs) built on pretrained video generation models adapt quickly to new robotic tasks, but they forget previously learned tasks under sequential fine-tuning. Experience replay (ER) mitigates such forgetting by rehearsing data of previous tasks, yet this data is often unavailable in robotics, where a policy is deployed as weights only and its training data is not shared along with it. In this work, we ask whether a WAM can achieve data-free continual learning by regenerating its own past. Our key observation is that a WAM can indeed autoregressively regenerate complete rollouts of previously learned tasks and replay them to mitigate forgetting. However, replaying such raw rollouts still lags far behind replaying real data, which we attribute to two issues. First, the WAM is unable to recognize task completion and keeps generating purposeless motion after the task is done. Second, under open-loop generation, the generated actions gradually drift away from the generated observations. To address both issues, we present rectified generative replay (R$^2$-WAM), which truncates generated rollouts at task completion with an agentic evaluator and filters out rollouts whose actions are physically inconsistent with the observations. On the four LIBERO suites and five real-world tasks, R$^2$-WAM reduces forgetting by over 90% relative to sequential fine-tuning and nearly matches ER without accessing any data from previous tasks. Our analysis further reveals that forgetting in WAMs concentrates in inverse dynamics modeling: a forgotten WAM still produces visually plausible observations of previous tasks while its generated actions fail. Project page: https://kaipengfang.github.io/r2-wam/

2026-10-13Into the LidarVerse: A Real-Time Unified LiDAR World Model Across Tasks and SensorsWe present LidarVerse, a real-time unified LiDAR world model for sensor simulation across tasks and sensors. Existing LiDAR simulators largely follow one of two complementary paradigms: reconstruction-based methods preserve the geometry of a captured scene but require lengthy per-scene optimization, whereas generative methods amortize learning across data but lack the consistency needed for long-horizon rollouts. LidarVerse brings both paradigms into a single, unified conditional generative framework. A universal tokenizer maps LiDAR range sequences into a shared latent space, where a bidirectional teacher learns a common interface for conditioning on a LiDAR prefix, an HD-Map, a persistent spatial cache, and optionally, text. Varying the available scene evidence turns the same model into a generator of new worlds, a point-cloud forecaster, or a generalizable resimulator of observed scenes without per-scene optimization. We distill the teacher into a four-step autoregressive causal student and maintain both a rolling temporal K/V cache and a world-aligned spatial cache, enabling persistent long-horizon rollouts with real-time inference at 11 FPS on a single H100 GPU. LidarVerse achieves state-of-the-art performance on most reported metrics across all three evaluation protocols, while its tokenizer supports 32-, 64-, and 128-beam LiDAR and generalizes to an unseen scanning pattern. These results demonstrate that generative and reconstruction-grounded LiDAR simulation can be realized within one model, providing the core capabilities for future closed-loop sensor simulation.

2026-10-12Ten Checkers for Agentic CAD: The Verification Loop to Run After Every Model UpdateDavid Bar's ten checkers for agentic CAD - spec, motion, geometry, edge, manufacturability, fastener, hole, picture, accepted-problems and test-the-checkers - written in Python on build123d and run after every model update.

2026-10-08AgentGarten: Code Worlds for Evolving AgentsInteractive virtual worlds allow agents to learn through exploration and interaction. What agents can learn is bounded by the environments they practice in, which must be faithful, with consistent state, rules, and dynamics, and realistic, with observations that follow the real-world visual distributions. Achieving both across diverse worlds remains a bottleneck. We introduce AgentGarten, a framework that couples simulators and game engines with a shared neural renderer to build real-time interactive environments. Its simulation backends maintain persistent world state and execute program-defined interaction rules, while the renderer generates visual observations from structured conditions exported through a common interface. To build the neural renderer, we adapt a pretrained video model to geometry conditions, distill it with our proposed Adversarial Forcing, and optimize inference for real-time interaction. Adversarial Forcing makes history prefilling differentiable through exact replay, so that losses on later predictions update how the renderer encodes prior observations, and adds real-data adversarial supervision to improve its visual quality. In AgentGarten, agents perceive the world through visual observations, interact with it in real time, and improve by distilling each round of experience into playbooks that subsequent agents inherit and refine. Our empirical study demonstrates a substantial gain in learning efficiency, with agents learning from just 4 rounds compared with millions for a conventional reinforcement learning counterpart. As new worlds can be written as code and rendered through the same interface, environments can scale in both number and difficulty alongside their agents, a step toward agents that keep evolving through interactive experience.

2026-10-06RIWANav: Recursive World-Action Models with Self-Improvement for Urban NavigationLong-horizon urban navigation compounds single-step errors, imitation learning only supervises expert successes, and a frozen action-conditioned world model drifts out of alignment as the policy evolves. RIWANav casts the coupled adaptation of a world model and a navigation policy as task-specific recursive self-improvement: Cosmos imagines 17 future frames for four sampled waypoint chunks per state, the resulting scores drive GRPO with a KL penalty and an expert anchor, and the improved policy then proposes actions whose expert-consistent action-video pairs (weighted ADE/MADE/AOE/MAOE consistency Q >= 0.35) enter a policy-induced replay pool prioritized by behavioral novelty and current prediction error to refine the world model over K=19 rounds. On UrbanNav it reaches 89.92% SR / 86.87% SPL on test-seen and 88.25% / 82.59% on test-unseen, beating frozen-world-model GRPO-WM by 7.94 / 6.50 points and cutting navigation error from 3.3314 to 2.0429, with world-model L1/LPIPS on policy-induced actions down 3.2% / 7.1%. Real-world trials on a custom mobile robot confirm practical transfer.

2026-10-01Introducing PROWL-2: Fidelity-Gated Dual Curricula — Jointly Learning Simulation and Decision-MakingOdyssey (with UCL AI Centre and the University of Basel) released PROWL-2 on Oct 1: the first framework coupling an agent curriculum and a world-model repair curriculum inside one continual training loop. Core insight: in imagination training, a high-learning-signal trajectory is ambiguous — a real policy weakness or just a wrong world-model prediction; prioritizing it indiscriminately reinforces the model's own hallucinations. A fidelity gate separates 'useful' from 'trustworthy': reliable high-potential imagined experience feeds the policy curriculum, unreliable rollouts — with their already-stored real continuations — go to a repair pool, fixed by a randomly-initialized, KL-anchored developer policy exploring the real environment, re-audited and readmitted once repaired. First place on all nine SMACv2 scenarios (+4-18% at 5v5, +20-91% at 10v10/10v11 over the backbone); big margins on hard MQE tasks — Gate-3 70.4% vs 29.8%, Shepherd-Hard 28.2% vs 7.3%, all other baselines at 0. Ablations show the curricula are not additive: repair alone is marginal, the ungated curriculum falls below the backbone in all nine scenarios, and the gate turns the same curriculum into the largest single gain. Architecture- and algorithm-agnostic; next steps point at humanoid coordination and long-horizon multi-agent games.

2026-09-30Social-WM: Safety-Aware Latent World Models for Robot Social NavigationSafe social navigation requires a robot to anticipate not only the future consequences of its actions, but also whether a nominal action can actually be executed under surrounding physical and social constraints. We present Social-WM, an efficient latent world-model planning framework trained from egocentric RGB video sequences. Our key observation is that social-navigation experience contains a systematic discrepancy between the nominal action and the realizable action: a nominal forward action may be fully executed in free space, but needs to be constrained when heading towards a pedestrian or obstacle. Social-WM learns these safety-relevant consequences directly through action-conditioned future prediction, where the target is the actual observed future following each command. We further introduce a realizable inverse-dynamics objective that associates observed latent transitions with the action actually realized rather than the nominal one. At deployment, candidate actions are imagined through the latent world model, and the inverse dynamics model estimates their realizability; nominal--realizable discrepancy then provides a safety signal before execution. The learned dynamics and realizability model remain goal-independent and support both position- and image-goal navigation. On Social-HM3D, Social-WM achieves 63.77% success while reducing human collisions to 21.67%, and maintains strong performance under zero-shot transfer to Social-MP3D, without explicit pedestrian tracking, privileged human state, or online reinforcement learning.

2026-09-28WB-WAM: Heterogeneous Body-Hand Pre-training for Humanoid Loco-ManipulationWB-WAM from Tsinghua IIIS, Xiong'an Institute of AI and University of Melbourne (Hang Zhao group) injects explicit whole-body action supervision into generative video pre-training: a shared 72-D physical action space (body 29 + root 3 + two hands 20+20) unifies partial annotations from 1,880.2 hours of nine-source heterogeneous video and motion data via masked flow matching, followed by PICO egocentric mid-training (22 h / 73 tasks, GMR retargeting + constrained IK + MINT) and real-robot post-training with a forward-kinematics loss on Unitree G1 with Wuji hands. It wins all seven HumanoidArena tasks (81.9% mean), reaches 84.0% on five real tasks vs OpenWAM's 80.0% (ACT 20%, Fast-WAM 6%), and PICO mid-training lets 30 real demos hit 73.8% — beating 100-demo direct training at 65.0%, a 70% cut in robot data.

2026-09-26World SLAM Model: Joint World Modeling for SLAM and NavigationWorld SLAM Model (WSM) embeds the mechanisms of SLAM, including persistent memory, incremental state updates, and local backend refinement, directly into a navigation world model. A unified Mixture-of-Transformers jointly predicts future RGB observations, dense depth, and camera poses from RGB input, enabling closed-loop navigation with accurate localization and mapping.

2026-09-24Rolling-WAM: World Action Models with Rolling ImaginationWorld Action Models (WAMs) couple action generation with future visual prediction for robotic manipulation. However, completing the joint video-action denoising process at each replanning cycle incurs substantial latency, delaying action updates and limiting closed-loop responsiveness. We present Rolling-WAM, a formulation that distributes joint denoising across successive replanning cycles. Our method maintains a sliding window of video-action chunks at staggered noise levels. At each step, a rolling noise schedule fully denoises the imminent action chunk for execution, while partially refining farther-future chunks. As the window advances with new camera observations, the retained future chunks continue their denoising process. This distributes the computational cost over time while carrying an evolving visual-action context across chunk boundaries. Evaluations on LIBERO, RoboTwin, and a real-world Unitree G1 humanoid show that Rolling-WAM achieves competitive manipulation performance. By removing the need to denoise the entire prediction horizon from scratch, it delivers a 4.5x steady-state replanning speedup over standard joint WAMs.

Open source

rsasaki0109/lidar_localization_ros2BSD-2-Clause

lidar_localization_ros2: map-based 3D LiDAR localization for ROS 2 that finds its own start pose

LiDAR LocalizationNDT

Python5801326

earthtojake/text-to-cadMIT

Text-to-CAD: the open-source plugin and skill suite that gives coding agents CAD superpowers

CADAgent Skills

Python21k2.0k100

gentlefress/Omega-0NOASSERTION

Omega-0 Official Code: Training, Inference and Unitree G1 Deployment for a Latent Predictive World-Action Model

Official implementation (MIT) of the omega-0 paper: a world-action model with Qwen3-VL conditioning, a frozen ViT-g frame encoder and Wan2.2 VAE latent supervision, whose 6-block three-stream predictor drives a flow-matching diffusion head over 66-dim whole-body actions (30-step chunks, real-time chunking). The repo also ships the Unitree G1 robot stack: versioned msgpack protocol, module runtime, SONIC whole-body adapter, Pico teleoperation collection and HDF5 recording. The 7.42 GB pretrained checkpoint landed on 2026-10-12, completing a release whose 4,828-episode omega-HOME dataset had been public since September.

World-action modelsHumanoid

Python5632

lightningpixel/modlyNOASSERTION

Modly: Local, Open-Source Image-to-3D Mesh Generation Desktop App

Image-to-3D3D Generation

TypeScript8.1k75755

neilsonnn/image-blasterMIT

image-blaster: blast one image into a collidable 3D world

3DGenerationWorldModel

TypeScript5.1k52255

ahujasid/mcp-for-blenderMIT

MCP for Blender: 32 tools that make Blender a 3D workspace an LLM can drive

MCPBlender

Python29k2.7k202

Genesis-Embodied-AI/genesis-worldApache-2.0

Genesis World: The Unified Multi-Physics Simulation Platform for Physical AI

SimulationPhysics Engine

Python30k2.9k224

img2threejs/img2threejsApache-2.0

img2threejs: Rebuilding Reference Images as Code-Only Procedural Three.js Models

Image-to-3DThree.js

Python15k1.3k59

dreamers-laboratory/image-to-3d-pipelineApache-2.0

image-to-3d-pipeline: A Fair Benchmark and Browser Explorer for Image-to-3D Models

TrellisTripoSR

JavaScript258200

seedleap/zing-world-modelNOASSERTION

Zing-0.5: An Efficient Real-Time Interactive World Model

Open-source causal world model from Seedleap.ai: text prompts and keyboard actions alter world evolution in real time. Causal KV cache + four-step DMD sampling enable long-horizon real-time rollouts on a single GPU; No.1 real-time world model on WBench.

World ModelsReal-time interaction

Python123162

Assets you can use now

No installable skill wired to this domain yet.

Verticals using it

Boundaries & failure modes

This domain carries the most marketing inflation, so the boundaries need detail. Asset side: topology is unusable - feed-forward 3D generation returns dense meshes or isosurfaces with high triangle counts and chaotic edge flow, which cannot go straight into an animation pipeline; retopology and UV unwrapping are still manual, and the "production-ready asset" in vendor demos is usually artist-cleaned. Single objects dominate: complex assemblies with joints, moving parts or internal structure are essentially out of reach, and thin walls, transparent and highly reflective materials break geometrically. Scale and units are inconsistent, so exported meshes need manual calibration before they enter an engine. Textures are low-resolution with incomplete PBR channels - most pipelines emit albedo only, metalness and roughness are guessed, and the moment the lighting changes it shows. Scene and world-model side: long-horizon drift - generative world models start forgetting earlier scene content after tens of seconds, and looking back shows objects moved or vanished. Action conditioning is coarse: precise interaction (grasping, opening a door, pressing a button) is far less reliable than the video suggests. And there is no agreed ruler for geometry: F-score and Chamfer distance depend on reconstruction quality and registration, so numbers across papers are not comparable - which is why this domain uses needs-reproduction heavily. Cost and latency: one 3D asset takes tens of seconds to minutes (meshing and texture baking each take a share). Producing a thousand assets already costs one to two orders of magnitude less than manual modelling, but that refers to first drafts; once retopology, UVs and LODs are added, the advantage shrinks to a few times rather than tens of times. World-model inference is priced like video generation, and interactive frame rates currently hold only in low-resolution, short-horizon demos.

As an Amazon Associate, we earn from qualifying purchases.

The ladder

Meshy 7: the 3D asset platform that finishes what comes after generation, from topology and PBR to rigging, printability, and MCP

Tripo: image-to-engine-ready-asset compressed into a ten-second step

Hunyuan3D 2.1: the open-weight 3D generation baseline and default starting point for self-hosting

FLUX: the line that moved image generation from "produce a picture" to controllable editing with open weights

Scenario: style-consistent asset volume for game art, not one beautiful picture at a time

TRELLIS: a research baseline that unifies 3D asset representations through a sparse structured latent

MCP for Blender: 32 tools that make Blender a 3D workspace an LLM can drive

Text-to-CAD: the open-source plugin and skill suite that gives coding agents CAD superpowers

img2threejs: Rebuilding Reference Images as Code-Only Procedural Three.js Models

Modly: Local, Open-Source Image-to-3D Mesh Generation Desktop App

image-blaster: blast one image into a collidable 3D world

lidar_localization_ros2: map-based 3D LiDAR localization for ROS 2 that finds its own start pose

image-to-3d-pipeline: A Fair Benchmark and Browser Explorer for Image-to-3D Models

Genesis World: The Unified Multi-Physics Simulation Platform for Physical AI

Latest demos

Sand3D Open-Sources 100K Code-Native 3D Objects

Terrain-Grounded Imagination Keeps Gaps and Edges Exact

ID-WM Plans Thousands of Steps with Hierarchical World Models

3D World Simulator Runs Interactive Robot Demos in Browser

Pascal Maps 47 Building Scans in Two Hours

LaMAria Surpasses Aria SLAM in Three of Five Regimes

SplatKit World Turns Prompts Into Explorable 3D Scenes

Building Photorealistic Scenes With Blender MCP

DynaSolver Simulates Fluid Around Moving Robots

3D World Simulator Predicts Geometry and Touch

Solv3R: A Continuous Multi-View Geometry Solver

LongCat-2.5: Native Multimodal Agents

VoxLution Generates 3D Scenes From Sparse Real Scans

SKY-001 Plans a Jaguar From Thousands of Parts

Robot Simulation Needs Accurate Physics, Not Just Visuals

1.5M-Splat Cherry Blossom Street Reconstruction

Faltwerk Streams Dynamic Gaussian Splats to Quest 3

PhysiCalWorld Calibrates Hidden Physics In Context

HairDeck turns one image into rigged 3D hair

Turning one illustration into a VTuber model

LibreYOLO adds one-image Gaussian splatting

A warehouse digital twin compresses 225 GB to 3.8 GB

HYPER3D Turns CAD Drawings Into Moving Machines

AnyScene Builds a Finished Blender Set From a Reference

Papers & deep reads

Open source

lidar_localization_ros2: map-based 3D LiDAR localization for ROS 2 that finds its own start pose

Text-to-CAD: the open-source plugin and skill suite that gives coding agents CAD superpowers

Omega-0 Official Code: Training, Inference and Unitree G1 Deployment for a Latent Predictive World-Action Model

Modly: Local, Open-Source Image-to-3D Mesh Generation Desktop App

image-blaster: blast one image into a collidable 3D world

MCP for Blender: 32 tools that make Blender a 3D workspace an LLM can drive

Genesis World: The Unified Multi-Physics Simulation Platform for Physical AI

img2threejs: Rebuilding Reference Images as Code-Only Procedural Three.js Models

image-to-3d-pipeline: A Fair Benchmark and Browser Explorer for Image-to-3D Models

Zing-0.5: An Efficient Real-Time Interactive World Model

Assets you can use now

Verticals using it

Boundaries & failure modes