A very interesting video focused on Marvel’s Spider-Man was just uploaded on Digital Foundry’s YouTube channel. Technology Editor Richard Leadbetter managed to capture raw footage of the game running on PlayStation 4 Pro, stripped out every other frame and then put it through Adobe After Effects’ Pixel Motion Frame Blending technique to increase its frame rate from the default 30 frames per second to 60 frames per second. That took many hours of offline rendering, powered by a Titan RTX GPU and an Intel i9 10900K CPU, and despite a few artifacts here and there it is a testament of how smooth the game would look and feel at 60 frames per second.
Despite that, the evidence points to Marvel’s Spider-Man: Miles Morales, the PlayStation 5 launch title revealed recently, being focused on delivering 30 frames per second action. Even with more and more gamers clamoring for high frame rate options, particularly with next-gen hardware on the horizon, it seems like console developers are still opting to target 30 FPS.
That decision also has positives, by the way, as seen in the debut Marvel’s Spider-Man: Miles Morales footage not just in terms of increased visual fidelity and additional graphics effects but also increased population density and a deeper, more accurate city simulation overall.
The Digital Foundry video then goes beyond this Marvel’s Spider-Man experiment, though, pointing out that frame rate upscaling techniques could indeed be achieved in real time. LucasArts worked on a prototype for a two-frame based interpolation that would have been implemented in Star Wars: The Force Unleashed II, which launched almost ten years ago. Since this was integrated into the game engine, the result worked fine in real time, but the final release didn’t include the interpolation option.
Leadbetter notes that today, reprojection and reconstruction techniques are commonly used to achieve Temporal Anti-Aliasing and even NVIDIA’s DLSS 2.0, which is possibly one of the most shining examples of using AI in games yet.
In fact, the Marvel’s Spider-Man Digital Foundry video actually ends with footage of PS4’s Killzone: Shadow Fall ‘upscaled’ to 60 frames per second via DAIN (Depth-Aware Video Frame Interpolation model, downloadable on GitHub, though it requires an NVIDIA graphics card). Again, this was done via offline rendering, but it could happen in real time as well, albeit probably not on the PlayStation 5.
Sony has given no indication that its next-gen console features any hardware dedicated to AI processing. However, that’s not the case with the Xbox Series X, which features DirectML support and special hardware support for ML algorithms, with performance estimated in 49 TOPS (trillion operations per second) for 8-bit integer operations and 97 TOPS for 4-bit integer operations. Whether that’ll be used just for regular image upscaling or for something like the frame rate upscaling proposed by Digital Foundry remains to be seen.
