Skip to main content

Interactive Artistic Environments

A player is deep in a narrative game, approaching a choice the story will remember. The score reads how she moves — how long she hesitates, where her camera rests. When she commits, the resolution is composed live. No two playthroughs sound the same.

Adaptive scoring has been a working craft for over two decades. The constraint is not whether it works; it is who can afford to build it. A flagship AAA title can support a full adaptive-audio team. A niche title cannot, and falls back on fixed loops. Installation and XR work shipped outside the AAA envelope hit the same ceiling. The broader argument for adaptive sound as the consequential market is in the journal piece The Wrong Debate; this page covers the integration specifics.

Beyond middleware​

Wwise, FMOD, and the engine-native audio systems orchestrate finite libraries. Everything they play back was preproduced; the total musical surface of a title is the cartesian product of its stems and its rules. CORPUS-trained models change which side of that product is the constraint:

  • Variation budget. A model under a stylistic envelope does not exhaust the way a stem matrix does.
  • Latency of state response. Generative material can respond at finer granularity than bars and transition windows.
  • Authorial control surface. What a composer hands off is an envelope, a palette, a state vocabulary — not a stem matrix. A different craft, closer to scoring for a system than for a scene.

The middleware layer stays. The model is a new asset class it hosts: a generative voice the runtime treats as a track, alongside the preproduced material that handles scripted moments where authorial control has to be exact.

First try, on-device​

The player is already at the turning point. The resolution lands on the first attempt, in the right key for the scene, at the latency the engine surfaces state at. That rules out cloud-scale generalist song generators regardless of how well they generate standalone tracks. The model has to be small enough for the runtime budget, steerable enough for engine-state resolution, trained on annotation dense enough to make the steering musically meaningful. See the semantic pipeline and scoring system.

Integration paths​

  • Wwise / FMOD plugin as a generative voice with conditioning wired to game state.
  • Engine-native in Unity or Unreal where no external middleware is used.
  • Installation and XR runtimes driven by audience sensing, gaze, environmental input.

Licensing shape — on-device vs. on-server, royalty flow — is worked out per engagement; see Access Models.

Why integrators need a licensed foundation​

  • Storefront and platform AI-content policies. The Audit Trail produces the record without per-title engineering.
  • Content-ID claims on streamed gameplay. A scraped model exposes integrators and their players' streams to claims that did not exist for preproduced cleared scores.
  • Personality rights on vocals. See the relevant section.

A system that has wanted living scores since the first dynamic mixer can have them now — on a foundation it can defend, at budgets below the AAA ceiling. The deeper argument is in Why CORPUS.