Natural Images, Gaussian Mixtures, and Dead Leaves
Annotated Paper Link: Google Drive
These are the notes for the paper Zoran & Weiss (2012).
Unconstrained Gaussian Mixture Models (GMMs) with a small number of mixture components learned from patches perform exteremly good
Valuable because
They are simple
Many of the current models are GMMs with exponential or infinite number of components with constrained covariance matrix
Paper Overview: A study of the nature of GMMs
Proof GMMs are excellent at modeling natural images patches
Analysis of properties of natural images captured by GMMs and relation to number of components \(K\)
Proof of strong connection between natural images statistics and a simple variant of dead leaves models
GMMs are Really Goos for Natural Images Patches
Compared GMM with 200 components to 8 other models
Trained on patches from the Berkley Segementation Database and used its test set
3 Experiments
Log-Likelihood: GMM outperformed all models and similar performance to Karklin and Lewicki
Denoising:
Added independent white Gaussian noise to the test set
Calculated MAP for each model given noisy patch
Evaluated performance using Peak Signal to Noise Ratio (PSNR)
GMMs performed exceptionally well
Sample Quality: Generated samples from the models. No quantitative analysis, but the GMMs samples looked capturing the structure of natural images like edges and textures
GMMs are really good
No claims of being the best
Analysis of the results
Adding more components to the GMM increases the performance; although, they seem to be converging at an upper bound (this is shown experimentally)
GMM as a Generative Process
You can generate a new sample from a GMM by choosing one of the \(K\) components and sampling \(N\) independent Gaussian variables with mean 0 and variance 1. Put these values in a vector and call it \(z\).
To compute the sample, use \(\mathbf{x} = \mathbf{V}_k \mathbf{D}_k^{0.5}\mathbf{z}\)
\(\mathbf{V}_k\) is the eigenvector matrix of \(\Sigma_k\)
\(\mathbf{D}_k\) is the eigenvalues matrix of \(\Sigma_k\)
It can be shown that this operation make the random vector \(\mathbf{z}\) follow the correaltion structure of \(\Sigma_k\)
Conclusion: To understand GMM, we need to understand their eigenvalues and eigenvectors
Eigenvalues and Eigenvectors of the First Components
Have very eigenvectors
Eigenvalues spectrum has very similar structure but differs with a multiplicative constant
This is equivalent to using a Gaussian Scale Mixture model
These components capture the contrast variability of the image patches
The Upcoming Components
Adding complements shows more specialized components capturing different properties of the natural images
They capture textures and boundaries on various scales and orientations
Mini Dead Leaves
The mini dead leaves model is a variant of the dead leaves model proposed by the authors that works on batches
It divides patches into two types (flat and edges)
Flat patches are sampled from a texture producing model (They use a GSM trained on natural images)
Edge patches randomly select an angle and distance from the center that divide the patch into two parts. Each part is sample separately as a flat patch
They model contrast and occlusions in images
They created images using mini dead leaves and trained GMM and the other previously used models on the new images.
The GMM performed incredibly well proving that its great performance on natural images is probably due to its ability to model occulsions and contrast
The mini dead leaves is weaker than GMM and expected to perform worse
- Primiarily due to weakness of GSM in modeling textures and the weak occlusion creation strategy (a random straight line)