Vibe3DScene

Create Your Own 3D Scene with Words Anywhere.

GitHub Web Demo
  • From the perspective of algoirthm, at the core of Vibe3DScene is a vision-aware single agent system which follows the render-and-verify strategy to build scenes. (1) It's built with LangGraph, borrowing some best practices of coding agents like tool call/planning/todos/rollback/memory management. (2) It unifies MCP tools like multimodal understanding, camera control, 3D asset retrieval/AIGC-Generation/PCG and scene management. The architecture is scalable and you can easily add your own tools/tool servers.
  • From the perspective of engineering, Vibe3DScene runs Blender in headless backend mode over network communication, so users can build scenes via chat from web/mobile/Blender-builtin clients, without relying on a local Blender GUI or CC/Cursor IDE. Beyond that, an owner-proxy + NGINX architecture enable multi-process scaling on a single server.
  • The design is modular and you can implement your own 3D agentic workflow.
Use Case 1

Headless Backend + Web Builder

Deploy Vibe3DScene as a backend service and generate complete 3D scenes directly from the web interface.

Use case 1 web demo GIF 1

Create a low poly scene in a dungeon, with a dragon guarding a pot of gold

Use case 1 web demo GIF 2

Create a small modern living room with a sofa, coffee table, floor lamp, rug, and TV stand. Keep realistic scale and clean layout.

Use case 1 image 1

Example1

Use case 1 image 2

Example2

Use case 1 image 3

Example3

Use case 1 image 4

PCG Example

Use Case 2

Blender Built-in Chat Generation

Create and refine scenes from inside Blender through integrated chat, while tool orchestration remains automatic in the background.

Use case 2 blender demo GIF
Architecture

Core System Topology

Runtime Orchestration

flowchart LR
    subgraph ENTRY["Entry Layer"]
        WEB["Web UI"]
        CLI["CLI"]
        BCHAT["Blender Chat"]
    end

    subgraph APIAPP["API + Agent Layer"]
        API["FastAPI Interface"]
        AGENT["LangGraph Agent Runtime"]
        SC["Session Coordinator"]
        OWN["Owner Proxy"]
        MEM["Session Memory / Persistence"]
        REDIS["Redis Registry (Optional)"]
    end

    subgraph EXEC["Execution Layer"]
        MODE{"BLENDER_MODE"}
        HM["Headless Manager"]
        HBL["Headless Blender Instances"]
        LMCP["Per-session MCP Runtime"]
        LCL["Local Addon Socket"]
    end

    subgraph OUTL["Output Layer"]
        OUT["Scene / Render / Asset Outputs"]
    end

    WEB --> API
    CLI --> API
    BCHAT --> API

    API --> AGENT
    API --> SC
    SC --> OWN
    AGENT --> MEM
    SC -.-> REDIS

    SC --> MODE
    MODE -->|headless| HM
    HM --> HBL
    HM --> LMCP
    MODE -->|local-client| LCL

    AGENT --> LMCP
    AGENT --> LCL
    LMCP --> HBL

    HBL --> OUT["Scene / Render / Asset Outputs"]
    LCL --> OUT
    OUT --> API

Execution flow from entry points to session routing and final outputs.

Tool and Service Topology

flowchart TB
    subgraph MCP["MCP Tool Runtime"]
        STRAT["Tool Strategy Router"]
        CORE["Scene / Camera / Render Tools"]
        RET["Retrieval Tools"]
        GEN["3D Generation Tools"]
        PCG["PCG Tools"]
        MEMT["Memory Tools"]
    end

    subgraph EXEC["Blender Targets"]
        HBL["Headless Blender Instances"]
        LCL["Local Blender Addon Socket"]
    end

    subgraph SERVICES["External Services"]
        TR["TRELLIS2"]
        AR["AssetRetrieval3D API"]
        PH["PolyHaven"]
        SK["Sketchfab API"]
        RDN["Hyper3D Rodin API"]
        HY["Hunyuan3D API"]
        INF["PCGIntegrator3D"]
        PG["PostgreSQL"]
    end

    STRAT --> CORE
    STRAT --> RET
    STRAT --> GEN
    STRAT --> PCG
    STRAT --> MEMT

    CORE --> HBL
    CORE --> LCL
    MEMT --> HBL
    MEMT --> LCL

    RET --> AR
    RET --> PH
    RET --> SK
    GEN --> TR
    GEN --> RDN
    GEN --> HY
    PCG --> INF

    AR --> PG

    AR --> HBL
    AR --> LCL
    TR --> HBL
    RDN --> HBL
    RDN --> LCL
    HY --> HBL
    INF --> HBL

Tool categories mapped to concrete service dependencies and Blender execution targets.

Tooling

Supported Tools

Category Concrete Implementations
Scene Inspection and Control get_scene_info, get_object_info, get_viewport_screenshot, execute_blender_code, import_glb_model
Camera and Rendering render_from_objects, render_from_camera, camera_observe, camera_act, camera_set_pose
Session Persistence undo_last_snapshot, get_session_persistence_status
Asset Retrieval search_polyhaven_assets, download_polyhaven_asset, set_texture, search_3d_assets_by_text, import_retrieved_asset, search_sketchfab_models, get_sketchfab_model_preview, download_sketchfab_model
3D Generation generate_trellis2_model, generate_hyper3d_model_via_text, generate_hyper3d_model_via_images, poll_rodin_job_status, import_generated_asset, generate_hunyuan3d_model
PCG get_infinigen_available_assets, generate_infinigen_assets