Training-Free Interactive
World Models

Introduction

$50k Grand Prize Winner @ Inference Time Compute Hackathon by Etched x Mercor x Cognition x Anthropic. 06/20/2026. Inspired by a recent release of Thinking Machine’s full duplex interactive models, we wanted to bring real-interactive capabilities to video generation. We found video generation to be an interesting medium, not only due to its inherently fun demonstrability, but also since there was an interesting problem to be solved.

Once a model can produce frames about as fast as you watch them, the interesting question stops being which clip you got back and becomes whether you can change what happens while it runs. A model you can steer mid-generation behaves less like a render job and more like a simulator you talk to, and had potential for further exploration.

Today’s video gen models treat a clip as one closed computation. The prompt is encoded once, and an autoregressive model denoises chunk after chunk while a KV cache accumulates every frame generated so far, which is what keeps each new chunk consistent with the last. The same cache is what makes the model stubborn: the old scene is baked into its context, so a new prompt injected midway is contradicted by everything the model remembers. The honest fix would be a full-duplex model trained to listen while it generates, but that takes interaction data and training compute we did not have, so the problem becomes getting interactivity purely at inference time from a model that was never trained for it.

Two things have to work at once: you have to inject a new prompt mid-stream without breaking the video, and generation has to be fast enough that “mid-stream” means anything at all.

1. Prompt injection. During a transition, each chunk is conditioned on an interpolation between the old and new prompt embeddings. That is the entire method, with no fine-tuning, no adapters, and no data, so it works on any prompt-conditioned autoregressive video model.
2. Real-time inference. Spectral resolution scheduling, FlashAttention 3, fused kernels, and a two-GPU DiT/VAE pipeline: 700 ms of compute per 750 ms chunk of video, and 0.69 s from voice to pixels.

Training-Free Prompt Injection

Setup

Our base model is Causal Forcing++, an autoregressive video diffusion model that generates video in chunks of 3 latent frames (0.75 s of video each).

A simple idea: interpolate the prompt embeddings

The naive way to change the scene mid-generation is to hard-swap the prompt embedding between chunk $t$ and chunk $t+1$. This fails visibly: the model’s KV cache still encodes the old scene, the new cross-attention conditioning contradicts it, and the video jump-cuts or collapses.

Instead, during a transition each chunk is conditioned on an interpolation between the old and new prompt embeddings. The shipped version is a per-token spherical interpolation over the prompt’s token embeddings, with direction and magnitude interpolated separately and a plain lerp fallback where a token pair is nearly collinear, eased by a min-jerk schedule over a ramp of 4 chunks, about 3 seconds of video:

ẽ_t = slerp(e_old, e_new; g_t),   g_t = r³(10−15r+6r²),   r: 0 → 1 over the ramp

Each chunk in the ramp is generated, and cached, under its own interpolated embedding, so the scene morphs rather than cuts: the meadow gradually frosts over instead of teleporting to winter. The KV cache is never invalidated. Past frames remain valid context, and the model resolves the changing conditioning the same way it resolves any other temporal change.

flowchart LR
  P1["initial prompt e_old"] --> C1["chunk t (g=0)"]
  C1 --"KV"--> C2["chunk t+1 (g≈0.1)"]
  P2["injected prompt e_new"] -.-> C2 & C3 & C4 & C5
  C2 --"KV"--> C3["chunk t+2 (g=0.5)"]
  C3 --"KV"--> C4["chunk t+3 (g≈0.9)"]
  C4 --"KV"--> C5["chunk t+4 (g=1)"]
  P1 -.-> C1 & C2 & C3 & C4

What didn’t work

Almost everything we tried before the ramp was a fight with the KV cache, so it is worth being precise about what that cache is doing. Each new chunk is generated by attending over two things: the text prompt, through cross-attention, and a cache of keys and values from the recently generated (clean) frames, through self-attention. The prompt says what the scene should be; the cached frames say what it has been. When you swap the prompt, the cross-attention conditioning flips immediately, but the cached keys and values still encode the old scene, and self-attention keeps pulling generation back toward it. LongLive (Yang & others, 2025) calls this semantic inertia, and it is exactly what a naive hard swap runs into: for a few chunks the old scene wins, then the new conditioning takes over all at once and the video jump-cuts or collapses.

The direct fix, and LongLive’s actual mechanism, is to re-cache: at the switch, recompute the cached keys and values for the recent frames under the new prompt, so the history reads as if it had always been heading toward the new scene. That removes the inertia, but LongLive only gets away with it because it is paired with streaming long tuning, fine-tuning the model on long sequences that contain prompt switches so it learns to make a recomputed cache consistent with the frames generated after it. We were training-free, so we got the re-caching without the training that makes it behave. On our short chunks the recomputed history and the post-switch frames disagreed at the boundary, and the seam was visible. We reproduced the mechanism and inherited none of the tuning that hides its artifacts.

Shrinking the context window around the switch is the blunt version of the same idea: throw away most of the cached frames and there is less old-scene history left to argue with the new prompt, so the transition lands sooner. It also lands as a cut. With little history to attend over, the model loses temporal continuity across the boundary and the two halves stop looking like one continuous shot.

Between those extremes we tried a run of cache surgeries, each aimed at the inertia from a different angle: keeping the self-attention keys but swapping the values, re-caching under a null prompt to wash out the old conditioning without committing to the new one, and projecting the old prompt’s direction out of the cached representations. We also tried leaving the cache alone and instead reweighting the two attention pathways for a few chunks after a switch, scaling the cross-attention (text) term up and the self-attention (past-frame) term down and sweeping the two coefficients. All of it was tunable and none of it generalized: a setting that gave a clean morph on one transition destabilized the next.

What finally worked was to stop editing the cache at all. Nothing in it is ever recomputed or discarded, so it can never be inconsistent; the prompt embedding is ramped over a few chunks instead, as described above, and each chunk is generated under a prompt only slightly different from the one that produced the frames it attends over. The model reads the change as ordinary temporal drift rather than a contradiction, and the scene morphs on its own. Growing the attention window slightly after a switch helped in a couple of runs, but the ramp alone was enough that the shipped recipe leaves it out.

SPEED: Spectral Progressive Diffusion

Interactivity is only worth having if the video keeps up with you. Our first problem: denoising a full grid of video tokens costs $O(n^2)$ in self-attention, and every denoising step pays it at full resolution.

SPEED (Spectral Progressive Diffusion) (Xiao et al., 2026) makes denoising spectrally autoregressive: early, high-noise steps run at reduced resolution, since structure, layout, and motion live in the low frequencies, and later steps restore full resolution for texture and detail. We ported it into the causal, streaming setting, which took two fixes: RoPE indices are strided so low-resolution tokens land on the full-resolution position grid, and the cache-commit pass stays at full resolution so the KV cache never sees low-res context.

On the frame-wise Causal Forcing++ model this is worth about 10%: 22.2 → 24.3 FPS at half spatial scale with one low-resolution step, no visible quality loss. Modest, and honestly so, since this model is already a few-step sampler, so there is little high-noise work left to make cheap. The paper’s larger wins (2× and up) come from 50-step, full-resolution sampling.

Kernel-Level Inference Optimizations

To know where the milliseconds were going, we profiled a full generation step. The op breakdown:

Two observations drove the kernel work. First, attention dominates, so switching to FlashAttention 3 captured significant savings there for free. Second, nearly a quarter of the step is not math at all. It is memory movement and small elementwise ops, each launching its own kernel and each making a round trip to HBM.

We wrote fused kernels that overlap compute with memory traffic and collapse chains of small ops into single launches:

• Fused RMSNorm: 7.71× speedup over the unfused baseline.
• Fused SiLU gate: 1.68× speedup, attacking the elementwise share directly.

Pipeline Parallelism

Even with fast kernels, one GPU doing everything was not real-time. Generating 750 ms of video took 976 ms when a single GPU ran both the DiT denoising and the VAE decode, slower than playback, which means choppy video no matter how you buffer it.

The fix is classic systems work: the two stages have no reason to share a device.

flowchart LR
  subgraph GPU0["GPU 0: DiT generation"]
    A["noise → 4 denoise steps → latent (~700 ms)"]
  end
  subgraph GPU1["GPU 1: VAE decode"]
    B["latent → RGB frames (~276 ms)"]
  end
  A --"FIFO queue over NVLink"--> B
  B --> C["display"]

• GPU 0 runs the DiT: raw noise in, four denoising steps, latent tensor out, about 700 ms per chunk.
• GPU 1 runs the VAE decode: latent tensor in, RGB pixel frames out, about 276 ms per chunk.
• A FIFO queue over NVLink connects them: while GPU 0 generates chunk $N$, it ships chunk $N-1$ to GPU 1 for decoding.

The VAE decode is now fully hidden behind the next chunk’s generation. End-to-end latency drops by 28%, and the critical path per chunk becomes just the 700 ms DiT pass, comfortably under the 750 ms of video each chunk contains.

Results

What the live system does:

• Generation stays ahead of playback, at 700 ms of compute per chunk, and each chunk is 750 ms of video.
• 0.69 s from the end of speech to the first affected pixels (transcribe → embed → inject → next chunk).
• Transitions morph instead of cutting (meadow to winter, beach to storm, dog to wolf) with the KV cache intact the whole way through.
• Nothing is trained, so nothing is model-specific.

Relation to prior work

Training-free semantic control of diffusion models is a rich area; our contribution is doing it mid-stream in an autoregressive video model, at real-time speeds:

• Prompt-to-Prompt (Hertz et al., 2023) edits images by manipulating cross-attention maps, while we modify no attention maps, only the conditioning embedding.
• Attend-and-Excite (Chefer et al., 2023) optimizes latents at sampling time to strengthen semantics, while we require no latent optimization.
• SEGA (Brack et al., 2023) steers generation along semantic directions found in noise-estimate space, while we need no search for steering vectors.

The interpolation scheme is strictly simpler than all three, which is what makes it fast enough to run inside a real-time loop.

The Infinite-Compute Regime

Diffusion models do best in the high-compute, low-data regime, and that is the regime that keeps getting cheaper. Everything in this post spends compute instead of data: the interpolation is free, SPEED and the kernels buy latency back, and the pipeline spends a second GPU. None of it touches the weights.

A video model that is real-time and steerable mid-stream stops behaving like a media tool and starts behaving like a learned simulator. It is full-duplex, but for video.

The uses we actually want it for:

• games where the world is generated rather than authored, and reacts to what players say;
• explanations that can be transformed on request, like “now show it in winter” or “now slow it down”;
• persistent generated environments that respond to the people in them.

All of it scales with inference-time compute, and none of it with data collection.

Built at the 2026 Inference-Time Compute Hackathon (Etched × Mercor × Cognition × Anthropic) by Ray Kasichainula, Subha Vadlamannati, Bryan Dong, and Gino Chiaranaipanich, where it won the $50,000 grand prize. Code: github.com/bryandong24/SPED.