Master modern CNN architectures from VGG to EfficientNet. Learn residual connections, inception modules, and transfer learning for state-of-the-art computer vision.
In Lesson 7, you learned the fundamentals of Convolutional Neural Networks. Now it's time to explore the architectures that revolutionized computer vision and enabled deep learning to surpass human-level performance on image tasks.
From 2012 to 2025, CNN architecture design has been a story of continuous innovation. Each breakthrough architecture introduced novel concepts that pushed the boundaries of what's possible:
In this lesson, we'll deep-dive into VGG, ResNet, and Inception—the three architectures that form the foundation of modern computer vision. You'll understand not just *what* they do, but *why* they work, and *when* to use each architecture.
By the end, you'll be able to choose the right architecture for your computer vision task and leverage transfer learning to achieve state-of-the-art results with minimal training data.
Understanding how we got here helps you choose the right architecture for your task.
The ImageNet Large Scale Visual Recognition Challenge (ILSVRC) drove CNN innovation:
2010-2011: Traditional CV Methods
2012: AlexNet - The Deep Learning Revolution
1# AlexNet architecture (simplified)
2import tensorflow as tf
3
4def alexnet_architecture():
5 """AlexNet: The first modern deep CNN"""
6 model = tf.keras.Sequential([
7 # Conv layers with large filters
8 tf.keras.layers.Conv2D(96, 11, strides=4, activation='relu'), # 11x11!
9 tf.keras.layers.MaxPooling2D(3, strides=2),
10
11 tf.keras.layers.Conv2D(256, 5, padding='same', activation='relu'),
12 tf.keras.layers.MaxPooling2D(3, strides=2),
13
14 tf.keras.layers.Conv2D(384, 3, padding='same', activation='relu'),
15 tf.keras.layers.Conv2D(384, 3, padding='same', activation='relu'),
16 tf.keras.layers.Conv2D(256, 3, padding='same', activation='relu'),
17 tf.keras.layers.MaxPooling2D(3, strides=2),
18
19 # Dense layers
20 tf.keras.layers.Flatten(),
21 tf.keras.layers.Dense(4096, activation='relu'),
22 tf.keras.layers.Dropout(0.5),
23 tf.keras.layers.Dense(4096, activation='relu'),
24 tf.keras.layers.Dropout(0.5),
25 tf.keras.layers.Dense(1000, activation='softmax')
26 ])
27 return model
28
29# Key innovations:
30# ✓ ReLU instead of sigmoid/tanh
31# ✓ Dropout for regularization
32# ✓ Data augmentation
33# ✓ GPU acceleration2014: VGG - Go Deeper with Small Filters
2014: GoogLeNet/Inception - Multi-Scale Features
2015: ResNet - Residual Learning
2017: DenseNet - Dense Connections
2019+: EfficientNet, Vision Transformers
1┌──────────────┬────────┬────────────┬────────────┬────────────┐
2│ Architecture │ Layers │ Parameters │ Top-1 Acc │ Year │
3├──────────────┼────────┼────────────┼────────────┼────────────┤
4│ AlexNet │ 8 │ 61M │ 63.3% │ 2012 │
5│ VGG-16 │ 16 │ 138M │ 71.3% │ 2014 │
6│ VGG-19 │ 19 │ 144M │ 71.1% │ 2014 │
7│ GoogLeNet │ 22 │ 7M │ 69.8% │ 2014 │
8│ ResNet-50 │ 50 │ 26M │ 76.1% │ 2015 │
9│ ResNet-152 │ 152 │ 60M │ 77.6% │ 2015 │
10│ DenseNet-201 │ 201 │ 20M │ 77.3% │ 2017 │
11│ EfficientNet │ - │ 5M │ 84.3% │ 2019 │
12└──────────────┴────────┴────────────┴────────────┴────────────┘Each architecture teaches us something fundamental:
Modern Practice (2025): Use pre-trained ResNet or EfficientNet for transfer learning unless you have 100M+ training images!
VGG (Visual Geometry Group, Oxford) proved that simple, uniform architecture design leads to strong performance.
Design Principles:
Why 3×3 filters?
Two 3×3 convolutions have the same receptive field as one 5×5 convolution, but with fewer parameters and more non-linearity:
1Receptive field of two 3×3 = 5×5
2Parameters: 2 × (3×3×C×C) = 18C²
3vs. one 5×5: 5×5×C×C = 25C²
4
5Saving: 28% fewer parameters!
6Bonus: Extra ReLU layer = more non-linearity1import tensorflow as tf
2
3def vgg16(input_shape=(224, 224, 3), num_classes=1000):
4 """
5 VGG-16 architecture
6
7 Pattern: [Conv-Conv-Pool] × 2, [Conv-Conv-Conv-Pool] × 3
8 Channels: 64 → 128 → 256 → 512 → 512
9 """
10
11 inputs = tf.keras.Input(shape=input_shape)
12
13 # Block 1 (64 channels)
14 x = tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu', name='block1_conv1')(inputs)
15 x = tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu', name='block1_conv2')(x)
16 x = tf.keras.layers.MaxPooling2D(2, strides=2, name='block1_pool')(x)
17
18 # Block 2 (128 channels)
19 x = tf.keras.layers.Conv2D(128, 3, padding='same', activation='relu', name='block2_conv1')(x)
20 x = tf.keras.layers.Conv2D(128, 3, padding='same', activation='relu', name='block2_conv2')(x)
21 x = tf.keras.layers.MaxPooling2D(2, strides=2, name='block2_pool')(x)
22
23 # Block 3 (256 channels)
24 x = tf.keras.layers.Conv2D(256, 3, padding='same', activation='relu', name='block3_conv1')(x)
25 x = tf.keras.layers.Conv2D(256, 3, padding='same', activation='relu', name='block3_conv2')(x)
26 x = tf.keras.layers.Conv2D(256, 3, padding='same', activation='relu', name='block3_conv3')(x)
27 x = tf.keras.layers.MaxPooling2D(2, strides=2, name='block3_pool')(x)
28
29 # Block 4 (512 channels)
30 x = tf.keras.layers.Conv2D(512, 3, padding='same', activation='relu', name='block4_conv1')(x)
31 x = tf.keras.layers.Conv2D(512, 3, padding='same', activation='relu', name='block4_conv2')(x)
32 x = tf.keras.layers.Conv2D(512, 3, padding='same', activation='relu', name='block4_conv3')(x)
33 x = tf.keras.layers.MaxPooling2D(2, strides=2, name='block4_pool')(x)
34
35 # Block 5 (512 channels)
36 x = tf.keras.layers.Conv2D(512, 3, padding='same', activation='relu', name='block5_conv1')(x)
37 x = tf.keras.layers.Conv2D(512, 3, padding='same', activation='relu', name='block5_conv2')(x)
38 x = tf.keras.layers.Conv2D(512, 3, padding='same', activation='relu', name='block5_conv3')(x)
39 x = tf.keras.layers.MaxPooling2D(2, strides=2, name='block5_pool')(x)
40
41 # Classification head
42 x = tf.keras.layers.Flatten(name='flatten')(x)
43 x = tf.keras.layers.Dense(4096, activation='relu', name='fc1')(x)
44 x = tf.keras.layers.Dropout(0.5, name='dropout1')(x)
45 x = tf.keras.layers.Dense(4096, activation='relu', name='fc2')(x)
46 x = tf.keras.layers.Dropout(0.5, name='dropout2')(x)
47 outputs = tf.keras.layers.Dense(num_classes, activation='softmax', name='predictions')(x)
48
49 model = tf.keras.Model(inputs=inputs, outputs=outputs, name='vgg16')
50 return model
51
52# Create and inspect
53model = vgg16()
54print(f"Total parameters: {model.count_params():,}")
55# Output: Total parameters: 138,357,544 (138M!)
56
57# Most parameters are in FC layers!
58# Conv layers: ~15M parameters
59# FC layers: ~123M parameters1Input: 224×224×3
2Block1: 224×224×64 (after pool: 112×112×64)
3Block2: 112×112×128 (after pool: 56×56×128)
4Block3: 56×56×256 (after pool: 28×28×256)
5Block4: 28×28×512 (after pool: 14×14×512)
6Block5: 14×14×512 (after pool: 7×7×512)
7Flatten: 7×7×512 = 25,088 features
8FC: 4096 → 4096 → 1000Strengths:
✓ Simple, uniform architecture (easy to understand and implement)
✓ Works well for transfer learning (pre-trained features are rich)
✓ Strong baseline for many computer vision tasks
✓ Generalizes well to new datasets
Weaknesses:
✗ 138M parameters (huge model size)
✗ Slow training and inference
✗ Most parameters in FC layers (inefficient)
✗ Vanishing gradients with deeper variants (VGG-19+)
When to Use VGG:
Modern Alternative: ResNet achieves better accuracy with fewer parameters and faster training!
ResNet (Residual Networks) solved the degradation problem: deeper networks performed worse than shallower ones! The solution? Skip connections.
Before ResNet, a puzzling observation:
120-layer network: 91.2% test accuracy
256-layer network: 89.6% test accuracy ← WORSE!This wasn't overfitting (training error was also higher). Deeper networks were harder to optimize.
Hypothesis: Maybe learning the identity function is hard?
Key Insight: Instead of learning H(x) directly, learn the residual F(x) = H(x) - x
1Traditional: H(x) = layers(x)
2Residual: H(x) = F(x) + x
3
4Where F(x) = layers(x)Why This Works:
1import tensorflow as tf
2
3def residual_block(x, filters, stride=1, name='res_block'):
4 """
5 Basic Residual Block for ResNet
6
7 Two 3×3 convolutions with a skip connection
8 """
9 # Save input for skip connection
10 shortcut = x
11
12 # Main path: Conv → BN → ReLU → Conv → BN
13 x = tf.keras.layers.Conv2D(
14 filters, 3, strides=stride, padding='same',
15 name=f'{name}_conv1'
16 )(x)
17 x = tf.keras.layers.BatchNormalization(name=f'{name}_bn1')(x)
18 x = tf.keras.layers.ReLU(name=f'{name}_relu1')(x)
19
20 x = tf.keras.layers.Conv2D(
21 filters, 3, padding='same',
22 name=f'{name}_conv2'
23 )(x)
24 x = tf.keras.layers.BatchNormalization(name=f'{name}_bn2')(x)
25
26 # If dimensions changed, project shortcut
27 if stride != 1 or shortcut.shape[-1] != filters:
28 shortcut = tf.keras.layers.Conv2D(
29 filters, 1, strides=stride,
30 name=f'{name}_shortcut_conv'
31 )(shortcut)
32 shortcut = tf.keras.layers.BatchNormalization(
33 name=f'{name}_shortcut_bn'
34 )(shortcut)
35
36 # Add skip connection
37 x = tf.keras.layers.Add(name=f'{name}_add')([x, shortcut])
38 x = tf.keras.layers.ReLU(name=f'{name}_relu2')(x)
39
40 return x
41
42# Visualize skip connectionSkip Connection Diagram:
1Input x
2 |
3 |--------------------→ (skip connection)
4 | |
5 Conv 3×3, BN, ReLU |
6 | |
7 Conv 3×3, BN |
8 | |
9 +---------ADD ←-------+
10 |
11 ReLU
12 |
13OutputFor deeper networks, use 1×1 convolutions to reduce parameters:
1def bottleneck_block(x, filters, stride=1, name='bottleneck'):
2 """
3 Bottleneck block for ResNet-50/101/152
4
5 1×1 (reduce) → 3×3 → 1×1 (expand)
6 """
7 shortcut = x
8
9 # Reduce dimensions
10 x = tf.keras.layers.Conv2D(filters, 1, name=f'{name}_conv1')(x)
11 x = tf.keras.layers.BatchNormalization(name=f'{name}_bn1')(x)
12 x = tf.keras.layers.ReLU(name=f'{name}_relu1')(x)
13
14 # 3×3 convolution
15 x = tf.keras.layers.Conv2D(
16 filters, 3, strides=stride, padding='same',
17 name=f'{name}_conv2'
18 )(x)
19 x = tf.keras.layers.BatchNormalization(name=f'{name}_bn2')(x)
20 x = tf.keras.layers.ReLU(name=f'{name}_relu2')(x)
21
22 # Expand dimensions (4× filters)
23 x = tf.keras.layers.Conv2D(filters * 4, 1, name=f'{name}_conv3')(x)
24 x = tf.keras.layers.BatchNormalization(name=f'{name}_bn3')(x)
25
26 # Project shortcut if needed
27 if stride != 1 or shortcut.shape[-1] != filters * 4:
28 shortcut = tf.keras.layers.Conv2D(
29 filters * 4, 1, strides=stride,
30 name=f'{name}_shortcut_conv'
31 )(shortcut)
32 shortcut = tf.keras.layers.BatchNormalization(
33 name=f'{name}_shortcut_bn'
34 )(shortcut)
35
36 x = tf.keras.layers.Add(name=f'{name}_add')([x, shortcut])
37 x = tf.keras.layers.ReLU(name=f'{name}_relu3')(x)
38
39 return xBottleneck Diagram:
1Input (256 channels)
2 |
3 |--------------------→ (skip connection)
4 | |
5 1×1, 64 (reduce) |
6 | |
7 3×3, 64 |
8 | |
9 1×1, 256 (expand) |
10 | |
11 +---------ADD ←-------+
12 |
13 ReLU
14 |
15Output (256 channels)Why Bottleneck?
1# ResNet-50 architecture (50 layers total)
2def resnet50(input_shape=(224, 224, 3), num_classes=1000):
3 inputs = tf.keras.Input(shape=input_shape)
4
5 # Initial conv
6 x = tf.keras.layers.Conv2D(64, 7, strides=2, padding='same')(inputs)
7 x = tf.keras.layers.BatchNormalization()(x)
8 x = tf.keras.layers.ReLU()(x)
9 x = tf.keras.layers.MaxPooling2D(3, strides=2, padding='same')(x)
10
11 # Residual blocks
12 # Stage 1: 3 blocks, 64 filters
13 for i in range(3):
14 x = bottleneck_block(x, 64, stride=1 if i > 0 else 1, name=f'stage1_block{i+1}')
15
16 # Stage 2: 4 blocks, 128 filters
17 for i in range(4):
18 x = bottleneck_block(x, 128, stride=2 if i == 0 else 1, name=f'stage2_block{i+1}')
19
20 # Stage 3: 6 blocks, 256 filters
21 for i in range(6):
22 x = bottleneck_block(x, 256, stride=2 if i == 0 else 1, name=f'stage3_block{i+1}')
23
24 # Stage 4: 3 blocks, 512 filters
25 for i in range(3):
26 x = bottleneck_block(x, 512, stride=2 if i == 0 else 1, name=f'stage4_block{i+1}')
27
28 # Classification head
29 x = tf.keras.layers.GlobalAveragePooling2D()(x) # No FC layers!
30 outputs = tf.keras.layers.Dense(num_classes, activation='softmax')(x)
31
32 model = tf.keras.Model(inputs=inputs, outputs=outputs, name='resnet50')
33 return model
34
35# ResNet variants
36RESNET_CONFIGS = {
37 'ResNet-18': [2, 2, 2, 2], # 18 layers
38 'ResNet-34': [3, 4, 6, 3], # 34 layers
39 'ResNet-50': [3, 4, 6, 3], # 50 layers (bottleneck)
40 'ResNet-101': [3, 4, 23, 3], # 101 layers
41 'ResNet-152': [3, 8, 36, 3], # 152 layers
42}1Model Layers Params Top-1 Acc Training Time
2ResNet-18 18 11M 69.8% 1×
3ResNet-34 34 21M 73.3% 1.5×
4ResNet-50 50 26M 76.1% 2×
5ResNet-101 101 45M 77.4% 3×
6ResNet-152 152 60M 77.6% 4×Sweet Spot: ResNet-50 offers best accuracy-speed trade-off for most applications!
✓ Default choice for image classification
✓ Transfer learning (best pre-trained models)
✓ Object detection backbone (Faster R-CNN, YOLO)
✓ Semantic segmentation (DeepLab, U-Net)
✓ Any computer vision task requiring deep networks
ResNet is the VGG of the modern era—simple, effective, widely used!
Inception (GoogLeNet) asked: "What if we extract features at multiple scales simultaneously?"
Problem: What's the best kernel size?
Insight: Why choose? Use them all in parallel!
1def inception_module(x, filters_1x1, filters_3x3_reduce, filters_3x3,
2 filters_5x5_reduce, filters_5x5, filters_pool_proj,
3 name='inception'):
4 """
5 Original Inception module (Inception v1)
6
7 Four parallel paths:
8 1. 1×1 convolutions (channel mixing)
9 2. 1×1 → 3×3 convolutions
10 3. 1×1 → 5×5 convolutions
11 4. MaxPool → 1×1 convolutions
12 """
13
14 # Path 1: 1×1 convolutions
15 path1 = tf.keras.layers.Conv2D(
16 filters_1x1, 1, padding='same', activation='relu',
17 name=f'{name}_path1_conv'
18 )(x)
19
20 # Path 2: 1×1 → 3×3 convolutions
21 path2 = tf.keras.layers.Conv2D(
22 filters_3x3_reduce, 1, padding='same', activation='relu',
23 name=f'{name}_path2_reduce'
24 )(x)
25 path2 = tf.keras.layers.Conv2D(
26 filters_3x3, 3, padding='same', activation='relu',
27 name=f'{name}_path2_conv'
28 )(path2)
29
30 # Path 3: 1×1 → 5×5 convolutions
31 path3 = tf.keras.layers.Conv2D(
32 filters_5x5_reduce, 1, padding='same', activation='relu',
33 name=f'{name}_path3_reduce'
34 )(x)
35 path3 = tf.keras.layers.Conv2D(
36 filters_5x5, 5, padding='same', activation='relu',
37 name=f'{name}_path3_conv'
38 )(path3)
39
40 # Path 4: MaxPool → 1×1 convolutions
41 path4 = tf.keras.layers.MaxPooling2D(
42 3, strides=1, padding='same',
43 name=f'{name}_path4_pool'
44 )(x)
45 path4 = tf.keras.layers.Conv2D(
46 filters_pool_proj, 1, padding='same', activation='relu',
47 name=f'{name}_path4_conv'
48 )(path4)
49
50 # Concatenate all paths
51 output = tf.keras.layers.Concatenate(
52 name=f'{name}_concat'
53 )([path1, path2, path3, path4])
54
55 return outputInception Module Diagram:
1 Input
2 |
3 +-------------+--------------+--------------+
4 | | | |
5 1×1 conv 1×1 conv 1×1 conv Max Pool
6 | | | |
7 | 3×3 conv 5×5 conv 1×1 conv
8 | | | |
9 +-------------+--------------+--------------+
10 |
11 Concatenate
12 |
13 OutputDimensionality Reduction (bottleneck layers):
1# Without 1×1 reduction:
2# 5×5 conv on 256 channels → 256 channels
3# Parameters: 256 × 5 × 5 × 256 = 1,638,400
4
5# With 1×1 reduction:
6# 1×1 conv: 256 → 64 channels
7# 5×5 conv: 64 → 64 channels
8# 1×1 conv: 64 → 256 channels
9# Parameters: 256×64 + 64×5×5×64 + 64×256 = 140,800
10
11# Savings: 91% fewer parameters!Benefits:
Factorized Convolutions:
1# Factorize 5×5 → 3×3 + 3×3
2# Parameters: 5×5 = 25 vs 3×3 + 3×3 = 18
3# Savings: 28%
4
5# Factorize 7×7 → 1×7 + 7×1
6# Parameters: 7×7 = 49 vs 1×7 + 7×1 = 14
7# Savings: 71%!
8
9def factorized_inception(x, name='factorized'):
10 # 1×n and n×1 convolutions
11 x = tf.keras.layers.Conv2D(64, (1, 7), padding='same', activation='relu')(x)
12 x = tf.keras.layers.Conv2D(64, (7, 1), padding='same', activation='relu')(x)
13 return xCombine Inception modules with ResNet skip connections:
1def inception_resnet_block(x, name='inception_resnet'):
2 """
3 Inception module with residual connection
4 """
5 shortcut = x
6
7 # Inception module (simplified)
8 path1 = tf.keras.layers.Conv2D(32, 1, activation='relu')(x)
9
10 path2 = tf.keras.layers.Conv2D(32, 1, activation='relu')(x)
11 path2 = tf.keras.layers.Conv2D(32, 3, padding='same', activation='relu')(path2)
12
13 path3 = tf.keras.layers.Conv2D(32, 1, activation='relu')(x)
14 path3 = tf.keras.layers.Conv2D(48, 3, padding='same', activation='relu')(path3)
15 path3 = tf.keras.layers.Conv2D(64, 3, padding='same', activation='relu')(path3)
16
17 # Concatenate
18 x = tf.keras.layers.Concatenate()([path1, path2, path3])
19
20 # 1×1 conv to match shortcut dimensions
21 x = tf.keras.layers.Conv2D(shortcut.shape[-1], 1)(x)
22
23 # Add skip connection
24 x = tf.keras.layers.Add()([x, shortcut])
25 x = tf.keras.layers.ReLU()(x)
26
27 return xDepthwise Separable Convolutions:
1# Standard convolution: spatial + channel mixing together
2# Depthwise separable: separate spatial and channel operations
3
4def depthwise_separable_conv(x, filters, name='dwsep'):
5 """
6 Depthwise separable convolution
7 = Depthwise conv + Pointwise conv (1×1)
8 """
9 # Depthwise: Apply 3×3 conv to each channel separately
10 x = tf.keras.layers.DepthwiseConv2D(
11 3, padding='same', activation='relu',
12 name=f'{name}_depthwise'
13 )(x)
14
15 # Pointwise: 1×1 conv to mix channels
16 x = tf.keras.layers.Conv2D(
17 filters, 1, activation='relu',
18 name=f'{name}_pointwise'
19 )(x)
20
21 return x
22
23# Parameters comparison (256 → 256 channels):
24# Standard 3×3: 256 × 3 × 3 × 256 = 589,824
25# Depthwise Sep: 256 × 3 × 3 + 256 × 256 = 67,840
26# Savings: 88%!✓ Multi-scale features are important (varied object sizes)
✓ Need efficient models (MobileNet uses depthwise separable)
✓ Embedded/mobile deployment (Xception, MobileNet)
✓ When you want to try something different from ResNet
1Feature VGG ResNet Inception
2─────────────────────────────────────────────────
3Depth Medium Deep Deep
4Parameters Very Medium Low
5 High
6Skip Connections No Yes Optional
7Multi-Scale No No Yes
8Training Speed Slow Fast Fast
9Inference Speed Slow Fast Fast
10Transfer Learning Good Best Good
11Mobile Friendly No No Yes
12 (Xception)Modern Recommendation: Start with ResNet-50, try EfficientNet if you need better efficiency!
Training CNNs from scratch requires millions of images and days of GPU time. Transfer learning lets you achieve state-of-the-art results with just hundreds of images!
Key Insight: Features learned on ImageNet (1.2M images, 1000 classes) generalize to other visual tasks!
Why It Works:
Strategy: Use a model pre-trained on ImageNet, adapt it to your task.
1. Feature Extraction (Frozen Backbone)
1# Load pre-trained ResNet50 (without top classification layer)
2base_model = tf.keras.applications.ResNet50(
3 weights='imagenet', # Pre-trained weights
4 include_top=False, # Remove classification head
5 input_shape=(224, 224, 3)
6)
7
8# Freeze all layers
9base_model.trainable = False
10
11# Add custom classification head
12model = tf.keras.Sequential([
13 base_model,
14 tf.keras.layers.GlobalAveragePooling2D(),
15 tf.keras.layers.Dense(256, activation='relu'),
16 tf.keras.layers.Dropout(0.5),
17 tf.keras.layers.Dense(num_classes, activation='softmax')
18])
19
20# Only train the new layers
21model.compile(
22 optimizer='adam',
23 loss='categorical_crossentropy',
24 metrics=['accuracy']
25)
26
27# Fast training (only classification head updates)
28model.fit(train_data, epochs=10)When to use: Small dataset (<1000 images), limited compute
2. Fine-Tuning (Partially Frozen)
1# After feature extraction training converges...
2
3# Unfreeze some layers (top layers are more task-specific)
4base_model.trainable = True
5
6# Freeze only early layers
7for layer in base_model.layers[:100]: # First 100 layers
8 layer.trainable = False
9
10# Use small learning rate to avoid destroying pre-trained weights
11model.compile(
12 optimizer=tf.keras.optimizers.Adam(1e-5), # Small LR!
13 loss='categorical_crossentropy',
14 metrics=['accuracy']
15)
16
17# Fine-tune
18model.fit(train_data, epochs=10)When to use: Medium dataset (1000-10,000 images), want best accuracy
3. Full Training (No Freezing)
1# Use pre-trained weights as initialization
2base_model = tf.keras.applications.ResNet50(
3 weights='imagenet',
4 include_top=False,
5 input_shape=(224, 224, 3)
6)
7
8# All layers trainable
9base_model.trainable = True
10
11# Train entire model
12model = tf.keras.Sequential([base_model, ...])
13model.compile(...)
14model.fit(train_data, epochs=50)When to use: Large dataset (>10,000 images), domain very different from ImageNet
1import tensorflow as tf
2from tensorflow.keras import layers, models
3
4def build_transfer_learning_model(num_classes, base_model_name='ResNet50'):
5 """
6 Complete transfer learning pipeline
7 """
8
9 # Available pre-trained models
10 BASE_MODELS = {
11 'VGG16': tf.keras.applications.VGG16,
12 'VGG19': tf.keras.applications.VGG19,
13 'ResNet50': tf.keras.applications.ResNet50,
14 'ResNet101': tf.keras.applications.ResNet101,
15 'ResNet152': tf.keras.applications.ResNet152,
16 'InceptionV3': tf.keras.applications.InceptionV3,
17 'InceptionResNetV2': tf.keras.applications.InceptionResNetV2,
18 'MobileNet': tf.keras.applications.MobileNet,
19 'MobileNetV2': tf.keras.applications.MobileNetV2,
20 'DenseNet121': tf.keras.applications.DenseNet121,
21 'EfficientNetB0': tf.keras.applications.EfficientNetB0,
22 }
23
24 # Load base model
25 BaseModel = BASE_MODELS[base_model_name]
26 base_model = BaseModel(
27 weights='imagenet',
28 include_top=False,
29 input_shape=(224, 224, 3),
30 pooling='avg' # Global average pooling
31 )
32
33 # Freeze base model
34 base_model.trainable = False
35
36 # Build complete model
37 inputs = layers.Input(shape=(224, 224, 3))
38
39 # Preprocessing (model-specific normalization)
40 x = tf.keras.applications.resnet.preprocess_input(inputs)
41
42 # Base model
43 x = base_model(x, training=False)
44
45 # Classification head
46 x = layers.Dense(512, activation='relu')(x)
47 x = layers.Dropout(0.5)(x)
48 x = layers.Dense(256, activation='relu')(x)
49 x = layers.Dropout(0.3)(x)
50 outputs = layers.Dense(num_classes, activation='softmax')(x)
51
52 model = models.Model(inputs, outputs)
53
54 return model, base_model
55
56# Usage
57model, base_model = build_transfer_learning_model(num_classes=10, base_model_name='ResNet50')
58
59# Phase 1: Train classification head
60model.compile(
61 optimizer=tf.keras.optimizers.Adam(1e-3),
62 loss='categorical_crossentropy',
63 metrics=['accuracy']
64)
65history1 = model.fit(train_data, epochs=10, validation_data=val_data)
66
67# Phase 2: Fine-tune top layers
68base_model.trainable = True
69for layer in base_model.layers[:-30]: # Freeze all but last 30 layers
70 layer.trainable = False
71
72model.compile(
73 optimizer=tf.keras.optimizers.Adam(1e-5), # Lower LR!
74 loss='categorical_crossentropy',
75 metrics=['accuracy']
76)
77history2 = model.fit(train_data, epochs=10, validation_data=val_data)1# Essential for small datasets
2data_augmentation = tf.keras.Sequential([
3 layers.RandomFlip('horizontal'),
4 layers.RandomRotation(0.1),
5 layers.RandomZoom(0.1),
6 layers.RandomContrast(0.1),
7 layers.RandomBrightness(0.1)
8])
9
10# Add to model
11inputs = layers.Input(shape=(224, 224, 3))
12x = data_augmentation(inputs) # Only applied during training
13x = base_model(x)
14...1# Phase 1: Feature extraction (high LR ok)
2lr_schedule_phase1 = tf.keras.optimizers.schedules.ExponentialDecay(
3 initial_learning_rate=1e-3,
4 decay_steps=1000,
5 decay_rate=0.9
6)
7
8# Phase 2: Fine-tuning (low LR essential)
9lr_schedule_phase2 = tf.keras.optimizers.schedules.ExponentialDecay(
10 initial_learning_rate=1e-5, # 100× smaller!
11 decay_steps=1000,
12 decay_rate=0.95
13)Similar Domain (e.g., ImageNet → Cats vs Dogs):
✓ Feature extraction works great
✓ Minimal fine-tuning needed
✓ Can achieve 95%+ accuracy with 1000 images
Different Domain (e.g., ImageNet → Medical Images):
✓ More fine-tuning required
✓ May need to unfreeze more layers
✓ Domain-specific preprocessing helps
Very Different Domain (e.g., ImageNet → Satellite Images):
✓ Consider training from scratch
✓ Or use domain-specific pre-trained models
✓ Transfer learning still provides good initialization
Scenarios where training from scratch might be better:
Best Practices:
Expected Results:
Transfer learning is the secret weapon of modern computer vision!
Build a complete transfer learning pipeline using ResNet-50 pre-trained on ImageNet. Implement both feature extraction and fine-tuning phases to achieve state-of-the-art accuracy on a custom image classification task.
1# Complete Transfer Learning Pipeline with ResNet50
2# Cats vs Dogs Classification
3
4import tensorflow as tf
5from tensorflow import keras
6from tensorflow.keras import layers
7import numpy as np
8import matplotlib.pyplot as plt
9
10print("=" * 70)
11print("TRANSFER LEARNING WITH RESNET50")
12print("=" * 70)
13print()
14
15# ========================================================================
16# STEP 1: Load and Prepare Data
17# ========================================================================
18
19print("Loading Cats vs Dogs dataset...")
20
21# Load dataset (using a subset for demonstration)
22(x_train, y_train), (x_test, y_test) = keras.datasets.cifar10.load_data()
23
24# Use only cats (class 3) and dogs (class 5)
25cat_indices = np.where(y_train == 3)[0][:1000]
26dog_indices = np.where(y_train == 5)[0][:1000]
27train_indices = np.concatenate([cat_indices, dog_indices])
28
29cat_test_indices = np.where(y_test == 3)[0][:200]
30dog_test_indices = np.where(y_test == 5)[0][:200]
31test_indices = np.concatenate([cat_test_indices, dog_test_indices])
32
33x_train = x_train[train_indices]
34y_train = (y_train[train_indices] == 5).astype(int) # 0=cat, 1=dog
35x_test = x_test[test_indices]
36y_test = (y_test[test_indices] == 5).astype(int)
37
38print(f"Training samples: {len(x_train)} (Cats: {np.sum(y_train==0)}, Dogs: {np.sum(y_train==1)})")
39print(f"Test samples: {len(x_test)} (Cats: {np.sum(y_test==0)}, Dogs: {np.sum(y_test==1)})")
40print()
41
42# Resize images to 224×224 (ResNet input size)
43x_train_resized = tf.image.resize(x_train, (224, 224)).numpy()
44x_test_resized = tf.image.resize(x_test, (224, 224)).numpy()
45
46# ========================================================================
47# STEP 2: Build Transfer Learning Model
48# ========================================================================
49
50print("Building transfer learning model with ResNet50...")
51print()
52
53# Load pre-trained ResNet50
54base_model = tf.keras.applications.ResNet50(
55 weights='imagenet',
56 include_top=False,
57 input_shape=(224, 224, 3),
58 pooling='avg'
59)
60
61print(f"ResNet50 loaded with {base_model.count_params():,} parameters")
62print()
63
64# Freeze base model (feature extraction only)
65base_model.trainable = False
66
67# Build complete model
68inputs = layers.Input(shape=(224, 224, 3))
69
70# Preprocessing
71x = tf.keras.applications.resnet.preprocess_input(inputs)
72
73# Base model (frozen)
74x = base_model(x, training=False)
75
76# Classification head
77x = layers.Dense(256, activation='relu', name='fc1')(x)
78x = layers.Dropout(0.5)(x)
79x = layers.Dense(128, activation='relu', name='fc2')(x)
80x = layers.Dropout(0.3)(x)
81outputs = layers.Dense(1, activation='sigmoid', name='output')(x)
82
83model = keras.Model(inputs, outputs, name='transfer_learning_model')
84
85print("Model architecture:")
86print(f"Total parameters: {model.count_params():,}")
87print(f"Trainable parameters: {sum([tf.size(v).numpy() for v in model.trainable_variables]):,}")
88print()
89
90# ========================================================================
91# STEP 3: Phase 1 - Feature Extraction Training
92# ========================================================================
93
94print("PHASE 1: Training classification head (base model frozen)")
95print("-" * 70)
96
97model.compile(
98 optimizer=keras.optimizers.Adam(1e-3),
99 loss='binary_crossentropy',
100 metrics=['accuracy']
101)
102
103# Data augmentation
104data_augmentation = tf.keras.Sequential([
105 layers.RandomFlip('horizontal'),
106 layers.RandomRotation(0.1),
107 layers.RandomZoom(0.1),
108])
109
110# Create augmented dataset
111def augment_and_preprocess(images, labels):
112 augmented = data_augmentation(images, training=True)
113 return augmented, labels
114
115# Train classification head
116history_phase1 = model.fit(
117 x_train_resized, y_train,
118 batch_size=32,
119 epochs=5,
120 validation_split=0.2,
121 verbose=2
122)
123
124# Evaluate after phase 1
125print()
126print("Phase 1 Results:")
127test_loss, test_acc = model.evaluate(x_test_resized, y_test, verbose=0)
128print(f"Test Accuracy: {test_acc*100:.2f}%")
129print()
130
131# ========================================================================
132# STEP 4: Phase 2 - Fine-Tuning
133# ========================================================================
134
135print("PHASE 2: Fine-tuning top layers of ResNet50")
136print("-" * 70)
137
138# Unfreeze top layers of base model
139base_model.trainable = True
140
141# Freeze all except last 10 layers
142for layer in base_model.layers[:-10]:
143 layer.trainable = False
144
145print(f"Trainable layers: {len([l for l in base_model.layers if l.trainable])}")
146print(f"Trainable parameters: {sum([tf.size(v).numpy() for v in model.trainable_variables]):,}")
147print()
148
149# Recompile with lower learning rate
150model.compile(
151 optimizer=keras.optimizers.Adam(1e-5), # 100× lower!
152 loss='binary_crossentropy',
153 metrics=['accuracy']
154)
155
156# Fine-tune
157history_phase2 = model.fit(
158 x_train_resized, y_train,
159 batch_size=32,
160 epochs=5,
161 validation_split=0.2,
162 verbose=2
163)
164
165# Final evaluation
166print()
167print("Final Results:")
168test_loss, test_acc = model.evaluate(x_test_resized, y_test, verbose=0)
169print(f"Test Accuracy: {test_acc*100:.2f}%")
170print()
171
172# ========================================================================
173# STEP 5: Visualize Training Progress
174# ========================================================================
175
176# Combine histories
177combined_history = {
178 'loss': history_phase1.history['loss'] + history_phase2.history['loss'],
179 'val_loss': history_phase1.history['val_loss'] + history_phase2.history['val_loss'],
180 'accuracy': history_phase1.history['accuracy'] + history_phase2.history['accuracy'],
181 'val_accuracy': history_phase1.history['val_accuracy'] + history_phase2.history['val_accuracy']
182}
183
184fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))
185fig.suptitle('Transfer Learning: Feature Extraction → Fine-Tuning', fontsize=14, fontweight='bold')
186
187# Plot loss
188ax1.plot(combined_history['loss'], label='Training Loss', linewidth=2)
189ax1.plot(combined_history['val_loss'], label='Validation Loss', linewidth=2)
190ax1.axvline(x=5, color='red', linestyle='--', label='Fine-tuning starts', linewidth=2)
191ax1.set_xlabel('Epoch', fontweight='bold')
192ax1.set_ylabel('Loss', fontweight='bold')
193ax1.set_title('Loss Over Time', fontweight='bold')
194ax1.legend()
195ax1.grid(True, alpha=0.3)
196
197# Plot accuracy
198ax2.plot(combined_history['accuracy'], label='Training Accuracy', linewidth=2)
199ax2.plot(combined_history['val_accuracy'], label='Validation Accuracy', linewidth=2)
200ax2.axvline(x=5, color='red', linestyle='--', label='Fine-tuning starts', linewidth=2)
201ax2.set_xlabel('Epoch', fontweight='bold')
202ax2.set_ylabel('Accuracy', fontweight='bold')
203ax2.set_title('Accuracy Over Time', fontweight='bold')
204ax2.legend()
205ax2.grid(True, alpha=0.3)
206
207plt.tight_layout()
208plt.show()
209
210# ========================================================================
211# STEP 6: Visualize Predictions
212# ========================================================================
213
214# Make predictions on test set
215predictions = model.predict(x_test_resized[:20], verbose=0)
216
217fig, axes = plt.subplots(4, 5, figsize=(15, 12))
218fig.suptitle('Model Predictions: Cats vs Dogs', fontsize=16, fontweight='bold')
219
220for i, ax in enumerate(axes.flat):
221 img = x_test[i].astype('uint8')
222 true_label = 'Dog' if y_test[i] == 1 else 'Cat'
223 pred_prob = predictions[i][0]
224 pred_label = 'Dog' if pred_prob > 0.5 else 'Cat'
225 confidence = pred_prob if pred_prob > 0.5 else 1 - pred_prob
226
227 color = 'green' if pred_label == true_label else 'red'
228
229 ax.imshow(img)
230 ax.set_title(f'True: {true_label}\nPred: {pred_label} ({confidence*100:.1f}%)',
231 color=color, fontweight='bold', fontsize=10)
232 ax.axis('off')
233
234plt.tight_layout()
235plt.show()
236
237# ========================================================================
238# KEY INSIGHTS
239# ========================================================================
240
241print("=" * 70)
242print("KEY INSIGHTS FROM TRANSFER LEARNING")
243print("=" * 70)
244print()
245print("1️⃣ PRE-TRAINED FEATURES")
246print(" → ResNet50 learned powerful features on ImageNet")
247print(" → These features generalize to cats vs dogs!")
248print()
249print("2️⃣ TWO-PHASE TRAINING")
250print(" → Phase 1: Train only classification head (fast)")
251print(" → Phase 2: Fine-tune top layers (better accuracy)")
252print()
253print("3️⃣ LEARNING RATE STRATEGY")
254print(" → Phase 1: High LR (1e-3) for new layers")
255print(" → Phase 2: Low LR (1e-5) to preserve pre-trained weights")
256print()
257print("4️⃣ SMALL DATASET SUCCESS")
258print(f" → Achieved {test_acc*100:.1f}% accuracy with only 2000 training images!")
259print(" → From scratch would need 10,000+ images")
260print()
261print("5️⃣ TRANSFER LEARNING BENEFITS")
262print(" → Faster training (minutes vs days)")
263print(" → Better accuracy with less data")
264print(" → Reduced computational cost")
265print()
266print("=" * 70)
267print("Transfer learning is the standard approach for computer vision!")
268print("=" * 70)This comprehensive example demonstrates the complete transfer learning workflow using ResNet-50, the most popular architecture for transfer learning in 2025.
1. Two-Phase Training Strategy
Phase 1: Feature Extraction (5 epochs)
Phase 2: Fine-Tuning (5 epochs)
2. Architecture Design
The classification head is carefully designed:
1ResNet-50 (frozen/partial) → GlobalAvgPool
2 ↓
3Dense(256) + ReLU + Dropout(0.5)
4 ↓
5Dense(128) + ReLU + Dropout(0.3)
6 ↓
7Dense(1) + SigmoidWhy this works:
3. Learning Rate is Critical
1Phase 1: LR = 1e-3 (high)
2 → Only new layers, no risk of destroying pre-trained weights
3
4Phase 2: LR = 1e-5 (low)
5 → Fine-tuning pre-trained layers requires caution
6 → Too high LR will erase useful features!4. Data Efficiency
With only 2,000 training images, transfer learning achieves ~93% accuracy.
Training from scratch would require:
5. Generalization Power
ResNet-50 was trained on ImageNet (1.2M images, 1000 classes):
Even though our task is different (binary cats vs dogs), the features learned on ImageNet transfer remarkably well because:
Training Curves:
Predictions:
This exact approach is used in production for:
Expected Results:
This is why ResNet-50 is the default choice for transfer learning in 2025!
Excellent work! You now understand the architectures that power modern computer vision. You've learned how ResNet's skip connections, VGG's simplicity, and Inception's multi-scale features revolutionized the field.
Next: In Lesson 9: Recurrent Neural Networks (RNNs), we'll shift from spatial data (images) to sequential data (text, time series, audio). You'll learn:
From images to sequences—let's master temporal modeling! 🚀
Your progress is saved in this browser, on this device only.