🎮 VAST just released EdenLive, an open-source AI system that can turn rough 3D game environments into photorealistic video in real time at 24 FPS!

Imagine building a simple graybox game with basic geometry, placeholder textures, and rough animations, then letting AI transform everything into a visually detailed world while you're actually playing.

That's exactly what EdenLive is designed to do.

And the most impressive part? Its 5B model can achieve real-time 720p-class rendering on a single NVIDIA RTX 5090.

How does EdenLive work?

The system separates game logic from visual rendering.

▪️ Game engine: Handles physics, player controls, interactions, object positions, and world state.

▪️ Proxy video: A simplified 2D or 3D representation provides the scene layout, camera movement, and motion.

▪️ AI renderer: Takes the proxy stream, a reference first frame, and a text prompt, then generates detailed video matching the underlying scene.

▪️ DINOv3 encoder: Extracts visual features from proxy frames and feeds them into a diffusion transformer.

This means the AI doesn't need to simulate the entire world. It only needs to render what the game engine is already doing.

Two model architectures are available:

▪️ EdenLive 14B: Built on Wan2.1 I2V, targeting 480p generation.

▪️ EdenLive 5B: Built on Wan2.2, targeting 720p generation.

Both support two inference modes:

▪️ Bidirectional: Processes video frames jointly for improved consistency.

▪️ Streaming: Uses chunk-autoregressive generation with just 4 denoising steps per chunk.

A lightweight TAEHV decoder enables fast video decoding.

The performance is impressive.

With LightX2V optimizations, VAST reports approximately 24.7 FPS at 1280×704 on a single RTX 5090.

The researchers have also demonstrated continuous generation lasting several minutes.

And there's something extra for game developers.

The release includes EdenGolem, an AI agent skill that can turn a text description into a basic playable game, plus a frontend connecting that game to EdenLive's rendering pipeline.

In principle, the workflow becomes:

Describe a game → Generate a playable prototype → Render its world with AI in real time.

What's available?

▪️ Open weights for both 5B and 14B models.

▪️ Bidirectional and streaming checkpoints.

▪️ Real-time inference and streaming code.

▪️ Approximately 210 GB for the complete model bundle.

▪️ A FLUX.2 Klein component for generating the initial reference frame in the full workbench. This component has a non-commercial license.

The reference installation is demanding, although optimized single-GPU inference is supported for the 5B model.

For game developers, this is a fascinating direction.

Instead of spending years creating high-quality textures, lighting, and environments, developers could potentially build simple interactive worlds and let generative AI handle their visual appearance.

It's still early, but the idea of a neural rendering engine replacing parts of the traditional graphics pipeline is becoming increasingly tangible.

Project and demos:

https://vast-eden.ai/?p=edenlive

GitHub:

https://github.com/VAST-AI-Research/EdenLive

Model weights:

https://huggingface.co/VAST-AI/EdenLive

#GameDev #GenerativeAI #OpenSourceAI