Image-GS: Content-Adaptive Image Representation via 2D Gaussians

Yunxiang Zhang^1*, Bingxuan Li^1*, Alexandr Kuznetsov^3†, Akshay Jindal², Stavros Diolatzis², Kenneth Chen¹, Anton Sochenov², Anton Kaplanyan², Qi Sun¹

* Equal contribution   † Work done while at Intel

^{1   2   3}

Neural image representations have emerged as a promising approach for encoding and rendering visual data. Combined with learning-based workflows, they demonstrate impressive trade-offs between visual fidelity and memory footprint. Existing methods in this domain, however, often rely on fixed data structures that suboptimally allocate memory or compute-intensive implicit models, hindering their practicality for real-time graphics applications.

Inspired by recent advancements in radiance field rendering, we introduce Image-GS, a content-adaptive image representation based on 2D Gaussians. Leveraging a custom differentiable renderer, Image-GS reconstructs images by adaptively allocating and progressively optimizing a group of anisotropic, colored 2D Gaussians. It achieves a favorable balance between visual fidelity and memory efficiency across a variety of stylized images frequently seen in graphics workflows, especially for those showing non-uniformly distributed features and in low-bitrate regimes. Moreover, it supports hardware-friendly rapid random access for real-time usage, requiring only 0.3K MACs to decode a pixel. Through error-guided progressive optimization, Image-GS naturally constructs a smooth level-of-detail hierarchy. We demonstrate its versatility with several applications, including texture compression, semantics-aware compression, and joint image compression and restoration.

_{Figure 1: Image-GS reconstructs an image by adaptively allocating and progressively optimizing a set of colored 2D Gaussians. It achieves favorable rate-distortion trade-offs, hardware-friendly random access, and flexible quality control through a smooth level-of-detail stack. (a) visualizes the optimized spatial distribution of Gaussians (20% randomly sampled for clarity). (b) Image-GS’s explicit content-adaptive design effectively captures non-uniformly distributed image features and better preserves fine details under constrained memory budgets. In the inset error maps, brighter colors indicate larger errors.}

Setup

1. Create a dedicated Python environment and install the dependencies

   git clone https://github.com/NYU-ICL/image-gs.git
   cd image-gs
   conda env create -f environment.yml
   conda activate image-gs
   pip install git+https://github.com/rahul-goel/fused-ssim/ --no-build-isolation
   cd gsplat
   pip install -e ".[dev]"
   cd ..

2. Download the image and texture datasets from OneDrive and organize the folder structure as follows

   image-gs
   └── media
       ├── images
       └── textures
   

3. (Optional) To run saliency-guided Gaussian position initialization, download the pre-trained EML-Net models (res_imagenet.pth, res_places.pth, res_decoder.pth) and place them under the models/emlnet/ folder

   image-gs
   └── models
       └── emlnet
           ├── res_decoder.pth
           ├── res_imagenet.pth
           └── res_places.pth
   

Quick Start

Image Compression

• Optimize an Image-GS representation for an input image anime-1_2k.png using 10000 Gaussians with half-precision parameters

python main.py --input_path="images/anime-1_2k.png" --exp_name="test/anime-1_2k" --num_gaussians=10000 --quantize

• Render the corresponding optimized Image-GS representation at a new resolution with height 4000 (aspect ratio is maintained)

python main.py --input_path="images/anime-1_2k.png" --exp_name="test/anime-1_2k" --num_gaussians=10000 --quantize --eval --render_height=4000

Texture Stack Compression

• Optimize an Image-GS representation for an input texture stack alarm-clock_2k using 30000 Gaussians with half-precision parameters

python main.py --input_path="textures/alarm-clock_2k" --exp_name="test/alarm-clock_2k" --num_gaussians=30000 --quantize

• Render the corresponding optimized Image-GS representation at a new resolution with height 3000 (aspect ratio is maintained)

python main.py --input_path="textures/alarm-clock_2k" --exp_name="test/alarm-clock_2k" --num_gaussians=30000 --quantize  --eval --render_height=3000

Control bit precision of Gaussian parameters

• Optimize an Image-GS representation for an input image anime-1_2k.png using 10000 Gaussians with 12-bit-precision parameters

python main.py --input_path="images/anime-1_2k.png" --exp_name="test/anime-1_2k" --num_gaussians=10000 --quantize --pos_bits=12 --scale_bits 12 --rot_bits 12 --feat_bits 12

Switch to saliency-guided Gaussian position initialization

• Optimize an Image-GS representation for an input image anime-1_2k.png using 10000 Gaussians with half-precision parameters and saliency-guided initialization

python main.py --input_path="images/anime-1_2k.png" --exp_name="test/anime-1_2k" --num_gaussians=10000 --quantize --init_mode="saliency"

Command Line Arguments

Please refer to cfgs/default.yaml for the full list of arguments and their default values.

Post-optimization rendering

• --eval render the optimized Image-GS representation.
• --render_height image height for rendering (aspect ratio is maintained).

Bit precision control: 32 bits (float32) per dimension by default

• --quantize enable bit precision control of Gaussian parameters.
• --pos_bits bit precision of individual coordinate dimension.
• --scale_bits bit precision of individual scale dimension.
• --rot_bits bit precision of Gaussian orientation angle.
• --feat_bits bit precision of individual feature dimension.

Logging

• --exp_name path to the logging directory.
• --vis_gaussians: visualize Gaussians during optimization.
• --save_image_steps frequency of rendering intermediate results during optimization.
• --save_ckpt_steps frequency of checkpointing during optimization.

Input image

• --input_path path to an image file or a directory containing a texture stack.
• --downsample load a downsampled version of the input image or texture stack as the optimization target to evaluate image upsampling performance.
• --downsample_ratio downsampling ratio.
• --gamma optimize in a gamma-corrected space, modify with caution.

Gaussian

• --num_gaussians number of Gaussians (for compression rate control).
• --init_scale initial Gaussian scale in number of pixels.
• --disable_topk_norm disable top-K normalization.
• --disable_inverse_scale disable inverse Gaussian scale optimization.
• --init_mode Gaussian position initialization mode, valid values include "gradient", "saliency", and "random".
• --init_random_ratio ratio of Gaussians with randomly initialized position.

Optimization

• --disable_tiles disable tile-based rendering (warning: optimization and rendering without tiles will be way slower).
• --max_steps maximum number of optimization steps.
• --pos_lr Gaussian position learning rate.
• --scale_lr Gaussian scale learning rate.
• --rot_lr Gaussian orientation angle learning rate.
• --feat_lr Gaussian feature learning rate.
• --disable_lr_schedule disable learning rate decay and early stopping schedule.
• --disable_prog_optim disable error-guided progressive optimization.

Acknowledgements

We would like to thank the gsplat team, as well as the authors of 3D Gaussian Splatting, Fused SSIM, and EML-Net for their outstanding work, which provided an important foundation for the development of Image-GS. We also thank Károly Zsolnai-Fehér for featuring our work on Two Minute Papers and introducing it to a broader audience, and Shubham Anand for creating a clear, thorough, and well-explained tutorial on our work on LearnOpenCV.

License

This project is licensed under the terms of the MIT license.

Citation

If you find this project helpful to your research, please consider citing BibTeX:

@inproceedings{zhang2025image,
  title={Image-gs: Content-adaptive image representation via 2d gaussians},
  author={Zhang, Yunxiang and Li, Bingxuan and Kuznetsov, Alexandr and Jindal, Akshay and Diolatzis, Stavros and Chen, Kenneth and Sochenov, Anton and Kaplanyan, Anton and Sun, Qi},
  booktitle={Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers},
  pages={1--11},
  year={2025}
}