Nvidia has just shown off DLSS 5 coming this fall... and currently it looks a lot like an AI filter

"Computer graphics comes to life... now what did we do?" Asks Nvidia CEO, Jen-Hsun Huang after showing off just how kinda stunning/kinda AI-filter-y DLSS 5 could look in games. It's honestly a weird, almost entirely contextless teaser. Though more context has come since, with Nvidia announcing it as a "real-time neural rendering model that infuses pixels with photoreal lighting and materials."

Anyways... back to Jen-Hsun.

"We fused controllable 3D graphics, the ground truth of virtual worlds, the structured data of virtual worlds, of generated worlds. We combined 3D graphics with generative AI, probabilistic computing.

"One of them is completely predictive, the other one, probabilistic yet highly realistic. The content is beautiful as well as controllable. This concept of fusing structured information and generative AI will repeat itself in one industry after another. Structured data is the foundation of trustworthy AI."

There are live demos over at GTC, which Digital Foundry has had access to ahead of the keynote today, and notes that in the demo, where the RTX 5090 is being used to render the path tracing version of Resident Evil Requiem and the standard Oblivion Remastered, a whole other RTX 5090 is being used to power the DLSS 5 component.

That's quite the hardware requirement, but hopefully when DLSS 5 comes out in the Autumn/Fall of this year it won't be quite so intimidating. Apparently, in the labs, Nvidia already has DLSS 5 running on a single GPU, but obviously the targets are going to have to be that it will run on single GPUs that aren't RTX 5090s. After all, those $4,000 cards are pretty hard to come by...

Nvidia is talking about this as just a snapshot of current development, so not exactly what DLSS 5 is going to look or perform like in release.

But how does it work?

According to Nvidia, "DLSS 5 takes a game's color and motion vectors for each frame as input, and uses an AI model to infuse the scene with photoreal lighting and materials that are anchored to source 3D content and consistent from frame to frame. DLSS 5 runs in real time at up to 4K resolution for smooth, interactive gameplay."

The model in use is apparently trained "end-to-end" to understand the fine details of a scene, from the characters, to the hair and fabric in the clothing, and the slightly translucent nature of human skin, as well as a host of different environmental factors, too.

And all this from analysing a single frame and extrapolating it to the rest. Sounds pretty wild, if you ask me.

To me, it kinda just looks like a pretty AI lighting filter, which I guess is fine, though a lot of care is going to have to be taken to ensure our game characters don't just end up all looking like utterly homogenous AI faces, just because that's what the DLSS 5 model thinks humans ought to look like that.

Thankfully, this seems to be something that's at the forefront of Nvidia's thinking on this, with a lot of mention of retain the original game artists' original vision.

Jen-Hsun is quoted on the Nvidia blog as saying: "DLSS 5 is the GPT moment for graphics—blending hand-crafted rendering with generative AI to deliver a dramatic leap in visual realism while preserving the control artists need for creative expression."

The same post goes on to explain that there are a host of different controls baked into DLSS 5 for game developers to tweak the knobs and pull the levers of the technology, in order to have the game world looking how they want:

"DLSS 5 provides game developers with detailed controls for intensity, color grading and masking, so artists can determine where and how enhancements are applied to maintain each game’s unique aesthetic. Integration is seamless, using the same NVIDIA Streamline framework used by existing DLSS and NVIDIA Reflex technologies."

And if you want an example of DLSS 5 games sticking to the original creative design of the developers you only have to look at the Starfield example. I mean, it still looks very, very, very Starfield. And no, that is not a compliment.

Still, you can be sure that there will be a lot of issues, from within and without game development, about the level of AI artifice is being placed on top of the games. There will be conversations to be had about artistic intent on the part of the designers and devs, but also what the end-user actually wants to see in their games.

It's going to be interesting to see how this initial take on DLSS 5 is going to be received, because I'm expecting it not to be pretty.

Anyways, in terms of confirmed games signed up for it...

"DLSS 5 will arrive this fall and be supported by the industry’s biggest publishers and game developers, including Bethesda, CAPCOM, Hotta Studio, NetEase, NCSOFT, S-GAME, Tencent, Ubisoft and Warner Bros. Games. DLSS 5 will come to games including AION 2, Assassin’s Creed Shadows, Black State, CINDER CITY, Delta Force, Hogwarts Legacy, Justice, NARAKA: BLADEPOINT, NTE: Neverness to Everness, Phantom Blade Zero, Resident Evil Requiem, Sea of Remnants, Starfield, The Elder Scrolls IV: Oblivion Remastered, Where Winds Meet and more."