Microsoft Unveils Code Video Model for Precise Scene Control

Microsoft has unveiled the Code Video Model, a video generation approach that uses executable code to precisely control scene structure, object motion and camera paths, according to aitimes.com.

The model converts prompts into executable programs through a coding agent that acts as the video generation engine. The agent writes code for object positions, movements, timing and camera paths, applying physics equations. It produces a proxy video and reference images in under one minute.

A pre-trained Reference-to-Video (R2V) model then generates the final detailed video from the proxy video and reference images. An 'early-step conditioning' technique applies the proxy video only during early sampling stages, then removes the condition.

The team also introduced the CodeVideoBench benchmark, covering games, scene editing, physics-based motion, bullet time and 3D/4D reconstruction. The Code Video Model combined with the R2V model MiniMax-H3 scored 85.2, 82.1 and 85.8 points on QwenVL-3.8-27B, GPT-5.6 and GPT-6 respectively. The highest-performing prior learning-based model, VACE, scored 74.3, 73.1 and 80.6 on the same benchmarks.

Weights and related code are available via Hugging Face and GitHub.

Existing video diffusion models can produce realistic videos but determine object placement, movement and camera trajectories internally, making detailed user adjustment difficult.