NVIDIA Presents Covert Protocol Demo Powered by Inworld AI

During today’s GTC 2024 keynote, NVIDIA CEO Jensen Huang showed a new AI NPC demo called Covert Protocol. Unlike the Kairos demo demonstrated at CES 2024, which was based on Convai technology, this one is powered by Inworld AI, the other major AI NPC platform geared towards game development. As you may recall, Inworld AI was first added by modder ‘Bloc’ in games like Mount & Blade II: Bannerlord, The Elder Scrolls V: Skyrim, and Grand Theft Auto V, though Rockstar quickly took that down to the dismay of the creator.

In November, Microsoft announced a partnership with Inworld AI to develop a multiplatform toolset featuring an AI design copilot and a character runtime engine. Now, Inworld’s tech has been integrated with NVIDIA ACE for Games, the Avatar Cloud Engine platform unveiled last year at Computex.

NVIDIA ACE aims to provide a complete ‘digital human pipeline’. It all begins with Riva, which enables automatic speech recognition (ASR) and text-to-speech (TTS) with the customizable Nemotron large language model (LLM). Here, developers can create specific character backstories and personalities that fit a developer’s game world. They can also better align player interactions within the context of a scene through programmable rules for NPCs with NeMo Guardrails. Lastly, Audio2Face automatically creates expressive facial animation for game characters from just an audio source.

On the Inworld end, developers can tweak various parameters for each specific NPC: personality, knowledge and cognition, goals, scenes, safety, long-term memory, and relationships. Kylan Gibbs, CEO of Inworld AI, also said:

The combination of NVIDIA ACE microservices and the Inworld Engine enables developers to create digital characters that can drive dynamic narratives, opening new possibilities for how gamers can decipher, deduce, and play.

Interestingly, NVIDIA intends its ‘Digital Human Platform Partners’ to take ACE to market, whether to game developers or enterprises for customer service purposes. Here’s the current list shared at GTC 2024: