GAN Architectures for Vision Tasks
From conditional generation to high-resolution synthesis: exploring the evolution of GANs for computer vision.
"One architecture to rule them all? Not quite—but each brings its own magic!"
Introduction
Since the introduction of GANs, researchers have developed numerous specialized architectures to tackle specific vision tasks. These innovations address challenges like generating high-resolution images, conditioning on labels or semantic maps, and achieving photorealistic results.
In this chapter, we'll explore prominent GAN architectures designed for vision applications—from conditional GANs that give you control, to progressive training that scales to high resolutions, and specialized models for image-to-image translation.
Conditional GAN (cGAN)
Key Idea: Guide the data generation process by conditioning both the generator and discriminator on additional information (labels, text, images, etc.).
How cGANs Work
cGAN Architecture: Both generator and discriminator receive the conditioning label y, enabling controlled generation.
⚙️ Generator G(z, y)
Takes both noise vector z and condition y (e.g., class label "cat") to produce samples that match the condition.
🔍 Discriminator D(x, y)
Judges both authenticity AND relevance to condition y, ensuring generated samples match the specified condition.
Example: Training on MNIST with labels: passing noise + label "7" generates handwritten digit 7. Without cGAN, the generator produces random digits with no control!
🌟 Applications of cGANs
📸 Image-to-Image Translation
Sketches → Photos, Day → Night scenes
✨ Photo Enhancement
Super-resolution, denoising, restoration
📊 Data Augmentation
Generate samples for rare classes
🎨 Style Transfer
Apply artistic styles to images
💬 Text-to-Image
Generate images from descriptions
🏥 Medical Imaging
Synthetic data for privacy-sensitive domains
✅ Advantages
- ✓ Controlled Generation: Specify what you want to generate
- ✓ Higher Quality: Conditioning guides meaningful outputs
- ✓ Versatile: Works with labels, images, text, etc.
- ✓ Cross-Domain: Art, science, medical, entertainment
⚠️ Challenges
- ⚠ Training Stability: Still inherits GAN training difficulties
- ⚠ Mode Collapse: Can collapse despite conditioning
- ⚠ Condition Quality: Effectiveness depends on condition relevance
- ⚠ Architecture Tuning: Requires careful design
🧠 Quick Quiz 1 Test your understanding
What is the main difference between GAN and cGAN?
Progressive Growing of GANs
Key Innovation: Instead of training on high-resolution images from the start, progressively add layers to both generator and discriminator, beginning with low resolutions and gradually increasing detail.
Progressive Training Stages
Stage 1: 4×4
Train on low-res
Stage 2: 4×4 → 64×64
Add layers, refine details
Stage 3: 4×4 → 1024×1024
High-res synthesis!
Training Progresses
Faster Training
Start with coarse structures, gradually add fine details
Better Stability
Gradual complexity increase prevents training collapse
Higher Quality
Produces unprecedented quality images at high resolutions
Specialized GAN Architectures
Pix2Pix
🖼️Image-to-Image Translation with Conditional Adversarial Networks
Learns mapping from input images to output images using cGAN with combined adversarial + L1 loss for pixel-wise accuracy.
StyleGAN
🎨Style-Based Generator Architecture for High-Quality Images
Uses Adaptive Instance Normalization (AdaIN) to control styles at each layer, enabling fine-grained control over texture, color, and structure.
GigaGAN
🚀Scaling Up GANs for Text-to-Image Synthesis
Multi-stage architecture handling complex text-to-image synthesis at high resolutions, with stages focusing from coarse shapes to fine details.
GauGAN (SPADE)
🌄Semantic Image Synthesis with Spatially-Adaptive Normalization
SPADE (Spatially-Adaptive Normalization) modulates features based on semantic layout, producing photorealistic images from semantic maps.
🧠 Quick Quiz 2 Challenge
What is the main advantage of Progressive GAN?
Architecture Comparison
| Architecture | Key Innovation | Best For | Conditioning |
|---|---|---|---|
| cGAN | Conditional inputs | Controlled generation | ✓ Yes (labels, text, images) |
| Progressive GAN | Growing architecture | High-res synthesis | ✗ No |
| Pix2Pix | Paired image translation | Image-to-image tasks | ✓ Input image |
| StyleGAN | AdaIN style control | Disentangled generation | ✓ Style vectors |
| GigaGAN | Text-to-image scaling | Text descriptions | ✓ Text |
| GauGAN | SPADE normalization | Semantic synthesis | ✓ Semantic maps |
Key Takeaways
🎯 Control
Conditional GANs enable precise control over generation through labels, text, or images
📈 Scaling
Progressive training enables stable, high-quality synthesis at unprecedented resolutions
🎨 Specialization
Task-specific architectures (Pix2Pix, StyleGAN, etc.) excel at their designed applications
From Control to Quality...
"Each GAN architecture is a tool in the creative toolkit—
choose wisely based on your task!"