Zero-Shot Image Restoration with Diffusion Models
12.01.2023
This project was developed for the Neural Networks course at university, and focused on replicating the results from the paper "Zero Shot Image Restoration Using Denoising Diffusion Null-Space Model" by Wang et al. (2022).
The paper introduces a zero-shot approach to solving Image Restoration (IR) tasks using a pre-trained diffusion model. The key innovation is the DDNM algorithm (and its enhanced version DDNM+), which allows the model to perform different IR tasks such as colorization, inpainting, and super resolution without requiring task-specific training. Instead, it only needs to define a task-dependent matrix A.
To better understand the background concepts, we first implemented the classic DDPM sampling process described by Ho et al. (2020), even though it was only briefly mentioned by Wang et al. This step helped us grasp the internal mechanisms of diffusion models before diving into the specifics of DDNM.
Afterward, we implemented:
- Standard DDNM, following the methodology described in the paper
- An intermediate version that supports noisy IR but without the time-travel trick
- DDNM+, including the time-travel step for improved results
Using code snippets provided by the authors for matrices A and A†, we tested our implementations across three major IR tasks: colorization, inpainting, and super resolution, both individually and in combination.
The project was extremely valuable for studying advanced image generation techniques, understanding the balance between computing time and output quality, and gaining hands-on experience with diffusion models.
You can find the complete PDF report and the Colab notebook with all the experiments on the GitHub repo.