Microsoft Xbox Unveils an Overly Ambitious AI Model for ‘Gameplay Ideation’

Yesterday, Microsoft Xbox introduced Muse, a generative AI model designed to spark ideas for gameplay, along with an open-access article on Nature.com and a blog post featuring a YouTube video. If “gameplay ideation” sounds a bit vague, Microsoft clarifies it as the ability to create “game visuals, controller actions, or both.” However, in reality, Muse’s functionality is quite restricted and doesn’t revolutionize the game development process as one might hope.

Despite these limitations, there are some intriguing aspects. For instance, the training was done on a large scale utilizing H100 GPUs, with around 1 million training updates needed to transform a single second of actual gameplay into an additional nine seconds of simulated, accurate gameplay generated by an engine. Most of the training data originated from current multiplayer gaming sessions.

Training Muse wasn’t as simple as running it on a single PC. Instead, Microsoft had to employ a cluster of 100 Nvidia H100 GPUs. This setup is significantly more expensive and power-intensive, yet it could only yield a 300×180 pixel resolution output for nine seconds of gameplay extrapolation.

One of the most compelling demonstrations of Muse was its ability to replicate existing props and enemies within the game’s environment, mirroring their functionality. But with all the resources spent on hardware, electricity, and AI training, wouldn’t it make more sense to use traditional development tools to spawn these elements?

Muse impressively managed to maintain object permanence and mimic the original game’s behavior. Yet, compared to the efficiency of the standard video game development pipeline, these achievements seem rather excessive and perhaps even wasteful.

Looking ahead, Muse might eventually achieve more remarkable feats, but as it stands, it doesn’t offer much beyond the long list of other projects trying to simulate gameplay purely through AI. While its ability to maintain some engine precision and object permanence is commendable, the current approach is rather inefficient for developing, testing, or playing video games. After delving deeply into the available information, it’s difficult to understand why anyone would choose to adopt Muse in its current form.