17.3 Representation Learning: Autoencoders

An autoencoder is a neural network trained to reconstruct its input. It learns a compressed latent representation (the bottleneck) that ideally captures essential structure and discards noise. Below we explore architecture, interpolation, metrics, and latent structure with lightweight synthetic examples (no external data required).

1. Architecture Visualization

Adjust the parameters to see a symmetric encoder–decoder design. Encoder widths decay toward a bottleneck; decoder mirrors them.

Input Dim Encoder Stages Latent Dim Decay Factor

Geometric width decay enforced until latent dimension; mirrored expansion reconstructs output dimension.

2. Latent Space Interpolation

Select random latent endpoints A & B and interpolate. The procedural image generator maps latent vectors to grayscale glyph-like patterns to illustrate continuity.

Latent Dim t {{interp.t | number:2}}

Interp

Interpolation Strip

3. Latent Space Exploration (Synthetic Scatter)

A 2D projection of randomly generated latent points colored by a simple factor (cluster id). Hover to preview a reconstruction. Points are produced from Gaussian mixture components.

Clusters Points

4. Reconstruction Quality Metrics (Simulated)

We approximate MSE and a pseudo-SSIM-like score by comparing a noisy input to its denoised reconstruction (simple blur). Adjust latent size proxy and noise to see trade-offs.

Latent Dim Proxy {{metrics.latent}} Noise Level {{metrics.noise | number:2}}

Input / Noisy

Reconstruction

Metrics

MSE	{{metrics.values.mse \| number:5}}
PS-SIM	{{metrics.values.pssim \| number:5}}
Compression Ratio	{{metrics.values.compression \| number:2}}:1

5. Feature Disentanglement (Toy)

Independent latent sliders control orthogonal factors (rotation, scale, intensity). Changes combine multiplicatively to form the final rendered glyph.

Rotation {{factors.rot}}° Scale {{factors.scale | number:2}} Intensity {{factors.intensity | number:2}}

Composite

6. VAE KL Divergence Illustration (Preview)

Coming soon: a dynamic comparison of posterior \(q(z|x)=\mathcal{N}(\mu,\sigma^2)\) vs prior \(p(z)=\mathcal{N}(0,1)\) with area-based KL intuition and annealing schedule.

7. Reconstruction Loss Landscape (Preview)

Planned: heatmap of reconstruction error as a function of (latent dimension, noise level) to visualize the compression vs fidelity trade-off.