Sandwiched Image Compression: Wrapping Neural Networks Around a Standard Codec

We sandwich a standard image codec between two neural networks: a preprocessor that outputs neural codes, and a postprocessor that reconstructs the image. The neural codes are compressed as ordinary images by the standard codec. Using differentiable proxies for rate and distortion we develop a rate-distortion optimization framework that trains the networks to generate neural codes that are efficiently compressible as images. This architecture not only improves ratedistortion performance for ordinary RGB images, but also enables efficient compression of alternative image types (such as normal maps of computer graphics) using standard image codecs. Results demonstrate the effectiveness and flexibility of neural processing in mapping a variety of input data modalities to the rigid structure of standard codecs. This flexibility is so much so that rate-distortion-optimized neural processing even manages to competently transport color images using a single-channel (grayscale) codec.

Publications

teaser image of Sandwiched Image Compression: Wrapping Neural Networks Around a Standard Codec

Sandwiched Image Compression: Wrapping Neural Networks Around a Standard Codec

2021 IEEE International Conference on Image Processing (ICIP), 2021.
Keywords: deep learning, image compression

Videos

Talks

Stay In Touch