As announced in late June, AMD unveiled an AI-powered method to compress textures called NTBC (Neural Texture Block Compression). Following its presentation at last week’s 35th Eurographics Symposium on Rendering, AMD has now released the paper penned by researchers Shin Fujieda and Takahiro Harada on the GPU Open website, revealing details on this new technique.
The objective of NTBC is to reduce game file sizes, which have been on a constant increase over the past few years. It’s not at all rare anymore for games to require around 150GB of free disk space to be installed. The main culprit for the increasingly large game file sizes are, of course, high-resolution textures for 4K displays and the like.
With the neural-based approach, NTBC learns how to map uncompressed textures to block-compressed textures (the paper focuses on the BC1 and BC4 formats), achieving up to 70% smaller storage footprint while maintaining reasonable texture quality and avoiding any change in the shaders. Here’s an excerpt from the paper on how it all works:
NTBC employs multi-layer perceptions (MLPs) to simultaneously encode block-compressed data of all textures in one material, achieving lower bit rates than the standard BC while producing the same block-compressed data format.
NTBC is meant to reduce the texture footprint on the disk. Network weights are stored in the disk which are loaded into the memory. Then inference is executed to reconstruct block-compressed texture data which are copied to VRAM. Therefore, it does not require any change in the shaders, which makes our method easier to adopt in the existing graphics pipelines. We also utilize multi-resolution feature grids to encourage model optimization and compress them through quantization-aware training to reduce storage costs.
The AMD researchers note that NTBC predicts block-compressed data instead of loading block-compressed textures from the disk; then, texel values are decoded from the compressed data using the existing BC decompression method at runtime. As such, the technique only introduces a modest computational overhead at the texture loading phase in the graphics pipeline.
This was estimated to be between 27.31 ms and 49.84 ms, depending on the choice between conservative (which runs two distinct models for RGB and single-channel textures) and aggressive approaches. It would seem relatively high, but the researchers clarify it’s not going to be a problem because the inference is only executed once per material instead of loading textures from the disk.
Of course, NTBC isn’t perfect yet. Its creators recognize that it occasionally creates blurred details or block artifacts due to its usage of lower-resolution grids compared to the original texture. Possible solutions include different encoding methods, like texture-focusing encoding or local positional encoding. Regarding formats, the AMD researchers believe NTBC could be extended to BC6H and BC7, too.
Right now, it’s anyone’s guess when this might be implemented by games. The sooner, the better, though.
