Master Google's powerful deep learning framework. Learn tensors, automatic differentiation, and build neural networks with the Keras API. Train models on MNIST and leverage GPU acceleration.
In Lesson 2, you built a neural network from scratch using NumPy, manually computing gradients and updating weights. It was powerful—but imagine doing that for a network with millions of parameters! Enter TensorFlow.
TensorFlow is Google's open-source deep learning framework that automates everything we coded by hand: gradient computation, GPU acceleration, model training, and production deployment. It's the engine behind YouTube recommendations, Google Translate, and countless AI applications.
Released in 2015 and revolutionized in 2019 with TensorFlow 2.0, it now features:
By the end of this lesson, you'll build, train, and evaluate neural networks on real datasets—all in under 20 lines of code. Let's dive into the framework that powers modern AI!
TensorFlow is more than a library—it's a complete ecosystem for machine learning from research to production.
Key Improvements Over TensorFlow 1.x:
tf.Session())@tf.function decorator for performance1# TensorFlow 1.x (Old Way - Complex!)
2import tensorflow.compat.v1 as tf
3tf.disable_v2_behavior()
4
5# Build computation graph
6x = tf.placeholder(tf.float32, shape=[None, 784])
7W = tf.Variable(tf.zeros([784, 10]))
8y = tf.matmul(x, W)
9
10# Create session and run
11sess = tf.Session()
12sess.run(tf.global_variables_initializer())
13result = sess.run(y, feed_dict={x: data})
14
15# TensorFlow 2.x (New Way - Simple!)
16import tensorflow as tf
17
18# Just use it like NumPy!
19x = tf.constant(data, dtype=tf.float32)
20W = tf.Variable(tf.zeros([784, 10]))
21y = tf.matmul(x, W)
22# Result is immediately available, no session needed!Core Components:
1. TensorFlow Core
tf.GradientTape)2. Keras (tf.keras)
3. TensorFlow Datasets (TFDS)
4. TensorBoard
5. TensorFlow Serving
6. TensorFlow Lite
1# Install TensorFlow
2# CPU version (sufficient for learning)
3pip install tensorflow
4
5# GPU version (requires CUDA drivers)
6pip install tensorflow-gpu
7
8# Verify installation
9import tensorflow as tf
10print(f"TensorFlow version: {tf.__version__}")
11print(f"GPU available: {tf.config.list_physical_devices('GPU')}")
12print(f"Built with CUDA: {tf.test.is_built_with_cuda()}")
13
14# Output example:
15# TensorFlow version: 2.15.0
16# GPU available: [PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]
17# Built with CUDA: TrueAdvantages:
When to Use TensorFlow vs PyTorch:
Key Insight: TensorFlow excels at taking models from research to production. Its ecosystem handles the entire ML lifecycle.
Everything in TensorFlow is a tensor—a multi-dimensional array similar to NumPy arrays but with GPU acceleration and automatic differentiation.
Tensor: A generalization of scalars, vectors, and matrices to arbitrary dimensions.
1import tensorflow as tf
2import numpy as np
3
4# Rank 0 tensor: Scalar
5scalar = tf.constant(42)
6print(f"Scalar: {scalar}")
7print(f"Shape: {scalar.shape}, Rank: {tf.rank(scalar).numpy()}")
8# Output: Scalar: tf.Tensor(42, shape=(), dtype=int32)
9# Shape: (), Rank: 0
10
11# Rank 1 tensor: Vector
12vector = tf.constant([1, 2, 3, 4, 5])
13print(f"\nVector: {vector}")
14print(f"Shape: {vector.shape}, Rank: {tf.rank(vector).numpy()}")
15# Output: Vector: tf.Tensor([1 2 3 4 5], shape=(5,), dtype=int32)
16# Shape: (5,), Rank: 1
17
18# Rank 2 tensor: Matrix
19matrix = tf.constant([[1, 2], [3, 4], [5, 6]])
20print(f"\nMatrix:\n{matrix}")
21print(f"Shape: {matrix.shape}, Rank: {tf.rank(matrix).numpy()}")
22# Output: Matrix:
23# [[1 2]
24# [3 4]
25# [5 6]]
26# Shape: (3, 2), Rank: 2
27
28# Rank 3 tensor: 3D array (e.g., RGB image)
29tensor_3d = tf.constant([
30 [[1, 2], [3, 4]],
31 [[5, 6], [7, 8]]
32])
33print(f"\n3D Tensor shape: {tensor_3d.shape}, Rank: {tf.rank(tensor_3d).numpy()}")
34# Output: 3D Tensor shape: (2, 2, 2), Rank: 3
35
36# Rank 4 tensor: Batch of images (batch, height, width, channels)
37batch_images = tf.random.normal([32, 224, 224, 3])
38print(f"\nBatch of images shape: {batch_images.shape}")
39# Output: Batch of images shape: (32, 224, 224, 3)
40# 32 images, 224×224 pixels, 3 color channels (RGB)1# From Python lists
2tensor_list = tf.constant([1, 2, 3, 4])
3
4# From NumPy arrays
5numpy_array = np.array([1, 2, 3, 4])
6tensor_numpy = tf.constant(numpy_array)
7
8# Zeros and ones
9zeros = tf.zeros([3, 4]) # 3×4 matrix of zeros
10ones = tf.ones([2, 3, 5]) # 2×3×5 tensor of ones
11
12# Random tensors
13random_normal = tf.random.normal([100, 100], mean=0.0, stddev=1.0)
14random_uniform = tf.random.uniform([50, 50], minval=0, maxval=10)
15
16# Range
17range_tensor = tf.range(start=0, limit=10, delta=2)
18print(range_tensor) # [0, 2, 4, 6, 8]
19
20# Identity matrix
21identity = tf.eye(5) # 5×5 identity matrix
22
23# Variables (trainable tensors for neural network weights)
24weight = tf.Variable(tf.random.normal([784, 10]))
25print(f"Weight shape: {weight.shape}")
26print(f"Trainable: {weight.trainable}")1# Element-wise operations
2a = tf.constant([1, 2, 3, 4])
3b = tf.constant([5, 6, 7, 8])
4
5print(f"Addition: {a + b}") # [6, 8, 10, 12]
6print(f"Multiplication: {a * b}") # [5, 12, 21, 32]
7print(f"Power: {a ** 2}") # [1, 4, 9, 16]
8
9# Matrix operations
10A = tf.constant([[1, 2], [3, 4]], dtype=tf.float32)
11B = tf.constant([[5, 6], [7, 8]], dtype=tf.float32)
12
13# Matrix multiplication
14C = tf.matmul(A, B)
15print(f"Matrix multiplication:\n{C}")
16# [[19, 22],
17# [43, 50]]
18
19# Transpose
20A_T = tf.transpose(A)
21print(f"Transpose:\n{A_T}")
22
23# Reshaping
24tensor = tf.constant([1, 2, 3, 4, 5, 6])
25reshaped = tf.reshape(tensor, [2, 3])
26print(f"Reshaped:\n{reshaped}")
27# [[1, 2, 3],
28# [4, 5, 6]]
29
30# Reduction operations
31tensor = tf.constant([[1, 2, 3], [4, 5, 6]], dtype=tf.float32)
32print(f"Mean: {tf.reduce_mean(tensor)}") # 3.5
33print(f"Sum: {tf.reduce_sum(tensor)}") # 21
34print(f"Max: {tf.reduce_max(tensor)}") # 6
35print(f"Mean axis 0: {tf.reduce_mean(tensor, axis=0)}") # [2.5, 3.5, 4.5]
36print(f"Sum axis 1: {tf.reduce_sum(tensor, axis=1)}") # [6, 15]1# TensorFlow tensors work seamlessly with NumPy
2import numpy as np
3
4# Convert TensorFlow tensor to NumPy
5tf_tensor = tf.constant([1, 2, 3])
6np_array = tf_tensor.numpy()
7print(f"NumPy array: {np_array}, type: {type(np_array)}")
8
9# Convert NumPy array to TensorFlow tensor
10np_array = np.array([4, 5, 6])
11tf_tensor = tf.constant(np_array)
12print(f"TensorFlow tensor: {tf_tensor}")
13
14# Operations work with both
15tf_tensor = tf.constant([1, 2, 3])
16np_array = np.array([4, 5, 6])
17result = tf_tensor + np_array # Automatic conversion!
18print(f"Mixed operation: {result}") # [5, 7, 9]1# Check device placement
2print("Available devices:")
3for device in tf.config.list_physical_devices():
4 print(f" {device}")
5
6# Explicit device placement
7with tf.device('/GPU:0'): # Force GPU execution
8 gpu_tensor = tf.random.normal([1000, 1000])
9 result = tf.matmul(gpu_tensor, gpu_tensor)
10
11with tf.device('/CPU:0'): # Force CPU execution
12 cpu_tensor = tf.random.normal([1000, 1000])
13 result = tf.matmul(cpu_tensor, cpu_tensor)
14
15# TensorFlow automatically places operations on GPU when available!Key Insight: Tensors are like NumPy arrays on steroids—they support GPU acceleration, automatic differentiation, and distributed computing. All deep learning operations in TensorFlow work with tensors.
In Lesson 2, you manually computed gradients for backpropagation. TensorFlow's GradientTape automates this completely using automatic differentiation (autograd).
Automatic differentiation tracks all operations performed on tensors and automatically computes gradients using the chain rule. No manual calculus needed!
1import tensorflow as tf
2
3# Simple example: f(x) = x^2
4x = tf.Variable(3.0)
5
6# Record operations with GradientTape
7with tf.GradientTape() as tape:
8 y = x ** 2 # f(x) = x^2
9
10# Compute gradient: dy/dx = 2x
11gradient = tape.gradient(y, x)
12
13print(f"x = {x.numpy()}")
14print(f"y = f(x) = {y.numpy()}")
15print(f"dy/dx = {gradient.numpy()}") # 2 * 3 = 6
16
17# Explanation:
18# At x = 3, the slope of f(x) = x^2 is 2*3 = 6
19# If we increase x slightly, y increases 6 times faster1# Function: f(x, y) = x^2 + y^2 + 2xy
2x = tf.Variable(2.0)
3y = tf.Variable(3.0)
4
5with tf.GradientTape() as tape:
6 z = x**2 + y**2 + 2*x*y
7
8# Compute gradients for both variables
9gradients = tape.gradient(z, [x, y])
10grad_x, grad_y = gradients
11
12print(f"f({x.numpy()}, {y.numpy()}) = {z.numpy()}")
13print(f"∂f/∂x = {grad_x.numpy()}") # 2x + 2y = 2*2 + 2*3 = 10
14print(f"∂f/∂y = {grad_y.numpy()}") # 2y + 2x = 2*3 + 2*2 = 10
15
16# These gradients tell us how to adjust x and y to minimize f1# Simple neural network: y = sigmoid(Wx + b)
2import numpy as np
3
4# Initialize weights and bias
5W = tf.Variable(tf.random.normal([5, 1]))
6b = tf.Variable(tf.zeros([1]))
7
8# Sample data
9x = tf.constant(np.random.randn(10, 5), dtype=tf.float32)
10y_true = tf.constant(np.random.randint(0, 2, (10, 1)), dtype=tf.float32)
11
12# Forward pass and loss computation
13with tf.GradientTape() as tape:
14 # Forward pass
15 logits = tf.matmul(x, W) + b
16 y_pred = tf.sigmoid(logits)
17
18 # Binary cross-entropy loss
19 loss = -tf.reduce_mean(
20 y_true * tf.math.log(y_pred + 1e-7) +
21 (1 - y_true) * tf.math.log(1 - y_pred + 1e-7)
22 )
23
24# Automatic backpropagation!
25gradients = tape.gradient(loss, [W, b])
26grad_W, grad_b = gradients
27
28print(f"Loss: {loss.numpy():.4f}")
29print(f"Gradient W shape: {grad_W.shape}")
30print(f"Gradient b shape: {grad_b.shape}")
31
32# Update weights (gradient descent)
33learning_rate = 0.1
34W.assign_sub(learning_rate * grad_W) # W = W - lr * grad_W
35b.assign_sub(learning_rate * grad_b) # b = b - lr * grad_b
36
37print("\nWeights updated!")1# Complete training loop
2def train_step(x, y_true, W, b, learning_rate):
3 """Single training step"""
4 with tf.GradientTape() as tape:
5 # Forward pass
6 logits = tf.matmul(x, W) + b
7 y_pred = tf.sigmoid(logits)
8
9 # Loss
10 loss = -tf.reduce_mean(
11 y_true * tf.math.log(y_pred + 1e-7) +
12 (1 - y_true) * tf.math.log(1 - y_pred + 1e-7)
13 )
14
15 # Backpropagation
16 gradients = tape.gradient(loss, [W, b])
17
18 # Update weights
19 W.assign_sub(learning_rate * gradients[0])
20 b.assign_sub(learning_rate * gradients[1])
21
22 return loss
23
24# Training loop
25print("Training with GradientTape:")
26for epoch in range(100):
27 loss = train_step(x, y_true, W, b, learning_rate=0.1)
28
29 if (epoch + 1) % 20 == 0:
30 print(f"Epoch {epoch+1}, Loss: {loss.numpy():.4f}")
31
32print("\nTraining complete!")1# By default, GradientTape can only compute gradients once
2x = tf.Variable(3.0)
3
4# Use persistent=True to compute multiple gradients
5with tf.GradientTape(persistent=True) as tape:
6 y = x ** 2
7 z = x ** 3
8
9# Compute gradients multiple times
10dy_dx = tape.gradient(y, x) # 2x = 6
11dz_dx = tape.gradient(z, x) # 3x^2 = 27
12
13print(f"dy/dx = {dy_dx.numpy()}")
14print(f"dz/dx = {dz_dx.numpy()}")
15
16# Clean up persistent tape
17del tape1# Compute second derivatives (gradient of gradient)
2x = tf.Variable(3.0)
3
4with tf.GradientTape() as tape1:
5 with tf.GradientTape() as tape2:
6 y = x ** 3 # f(x) = x^3
7
8 # First derivative: dy/dx = 3x^2
9 dy_dx = tape2.gradient(y, x)
10
11# Second derivative: d²y/dx² = 6x
12d2y_dx2 = tape1.gradient(dy_dx, x)
13
14print(f"First derivative: dy/dx = {dy_dx.numpy()}") # 3 * 3^2 = 27
15print(f"Second derivative: d²y/dx² = {d2y_dx2.numpy()}") # 6 * 3 = 181# GradientTape automatically watches tf.Variable
2# For regular tensors, use tape.watch()
3x = tf.constant(3.0) # Not a Variable!
4
5with tf.GradientTape() as tape:
6 tape.watch(x) # Explicitly watch constant tensor
7 y = x ** 2
8
9gradient = tape.gradient(y, x)
10print(f"Gradient: {gradient.numpy()}") # 6Key Insight: GradientTape automates backpropagation completely. You define the forward pass, and TensorFlow automatically computes all gradients using the chain rule. This is the foundation of deep learning frameworks!
Keras is TensorFlow's high-level API for building neural networks. It provides three ways to build models, each with different flexibility levels.
Perfect for linear stacks of layers (90% of use cases).
1import tensorflow as tf
2from tensorflow import keras
3
4# Build a neural network: Input(784) → Dense(128) → Dense(10)
5model = keras.Sequential([
6 keras.layers.Dense(128, activation='relu', input_shape=(784,)),
7 keras.layers.Dense(64, activation='relu'),
8 keras.layers.Dense(10, activation='softmax')
9])
10
11# View model architecture
12model.summary()
13
14# Output:
15# Model: "sequential"
16# _________________________________________________________________
17# Layer (type) Output Shape Param #
18# =================================================================
19# dense (Dense) (None, 128) 100480
20# dense_1 (Dense) (None, 64) 8256
21# dense_2 (Dense) (None, 10) 650
22# =================================================================
23# Total params: 109,386
24# Trainable params: 109,386
25# Non-trainable params: 0Understanding Layers:
For complex architectures: multiple inputs/outputs, branching, skip connections.
1# Same network using Functional API
2inputs = keras.Input(shape=(784,))
3x = keras.layers.Dense(128, activation='relu')(inputs)
4x = keras.layers.Dense(64, activation='relu')(x)
5outputs = keras.layers.Dense(10, activation='softmax')(x)
6
7model = keras.Model(inputs=inputs, outputs=outputs, name='mnist_model')
8
9model.summary()
10
11# Example: Multi-input model
12input1 = keras.Input(shape=(32,), name='input1')
13input2 = keras.Input(shape=(64,), name='input2')
14
15# Process each input separately
16x1 = keras.layers.Dense(16, activation='relu')(input1)
17x2 = keras.layers.Dense(32, activation='relu')(input2)
18
19# Combine
20combined = keras.layers.concatenate([x1, x2])
21output = keras.layers.Dense(1, activation='sigmoid')(combined)
22
23multi_input_model = keras.Model(
24 inputs=[input1, input2],
25 outputs=output
26)For custom training loops, dynamic architectures, research models.
1class CustomModel(keras.Model):
2 def __init__(self):
3 super(CustomModel, self).__init__()
4 self.dense1 = keras.layers.Dense(128, activation='relu')
5 self.dense2 = keras.layers.Dense(64, activation='relu')
6 self.dense3 = keras.layers.Dense(10, activation='softmax')
7
8 def call(self, inputs, training=False):
9 """Forward pass"""
10 x = self.dense1(inputs)
11 x = self.dense2(x)
12 return self.dense3(x)
13
14model = CustomModel()
15
16# Build the model by calling it once
17_ = model(tf.zeros([1, 784]))
18model.summary()1# Dense (Fully Connected)
2keras.layers.Dense(units=128, activation='relu')
3
4# Dropout (Regularization - prevents overfitting)
5keras.layers.Dropout(rate=0.3) # Drops 30% of neurons during training
6
7# Batch Normalization (Stabilizes training)
8keras.layers.BatchNormalization()
9
10# Convolutional (for images)
11keras.layers.Conv2D(filters=32, kernel_size=3, activation='relu')
12
13# Pooling (downsample)
14keras.layers.MaxPooling2D(pool_size=2)
15
16# Flatten (convert 2D to 1D)
17keras.layers.Flatten()
18
19# Recurrent (for sequences)
20keras.layers.LSTM(units=50, return_sequences=True)
21
22# Example: Complete CNN for image classification
23cnn_model = keras.Sequential([
24 # Convolutional block 1
25 keras.layers.Conv2D(32, 3, activation='relu', input_shape=(28, 28, 1)),
26 keras.layers.MaxPooling2D(2),
27
28 # Convolutional block 2
29 keras.layers.Conv2D(64, 3, activation='relu'),
30 keras.layers.MaxPooling2D(2),
31
32 # Dense layers
33 keras.layers.Flatten(),
34 keras.layers.Dense(128, activation='relu'),
35 keras.layers.Dropout(0.5),
36 keras.layers.Dense(10, activation='softmax')
37])1# L2 regularization (weight decay)
2keras.layers.Dense(
3 128,
4 activation='relu',
5 kernel_regularizer=keras.regularizers.l2(0.01)
6)
7
8# Dropout
9keras.layers.Dropout(0.5) # Drop 50% of neurons
10
11# Example: Model with regularization
12model = keras.Sequential([
13 keras.layers.Dense(128, activation='relu',
14 kernel_regularizer=keras.regularizers.l2(0.01)),
15 keras.layers.Dropout(0.5),
16 keras.layers.Dense(64, activation='relu'),
17 keras.layers.Dropout(0.3),
18 keras.layers.Dense(10, activation='softmax')
19])1class CustomDenseLayer(keras.layers.Layer):
2 def __init__(self, units=32):
3 super(CustomDenseLayer, self).__init__()
4 self.units = units
5
6 def build(self, input_shape):
7 """Create weights when input shape is known"""
8 self.w = self.add_weight(
9 shape=(input_shape[-1], self.units),
10 initializer='random_normal',
11 trainable=True
12 )
13 self.b = self.add_weight(
14 shape=(self.units,),
15 initializer='zeros',
16 trainable=True
17 )
18
19 def call(self, inputs):
20 """Forward pass"""
21 return tf.matmul(inputs, self.w) + self.b
22
23# Use custom layer
24model = keras.Sequential([
25 CustomDenseLayer(64),
26 keras.layers.Activation('relu'),
27 keras.layers.Dense(10, activation='softmax')
28])Key Insight: Keras provides three APIs with increasing flexibility:
Choose based on your needs—start with Sequential, upgrade when necessary!
Keras provides a streamlined workflow for training neural networks. The core steps are: compile, fit, evaluate, and predict.
Define optimizer, loss function, and metrics before training.
1model = keras.Sequential([
2 keras.layers.Dense(128, activation='relu', input_shape=(784,)),
3 keras.layers.Dropout(0.2),
4 keras.layers.Dense(10, activation='softmax')
5])
6
7# Compile: configure training
8model.compile(
9 optimizer='adam', # Optimization algorithm
10 loss='sparse_categorical_crossentropy', # Loss function
11 metrics=['accuracy'] # Metrics to monitor
12)
13
14# Alternative: customize optimizer
15model.compile(
16 optimizer=keras.optimizers.Adam(learning_rate=0.001),
17 loss='sparse_categorical_crossentropy',
18 metrics=['accuracy', 'top_k_categorical_accuracy']
19)Common Optimizers:
Common Loss Functions:
1# Sample data
2import numpy as np
3x_train = np.random.randn(1000, 784)
4y_train = np.random.randint(0, 10, 1000)
5
6x_test = np.random.randn(200, 784)
7y_test = np.random.randint(0, 10, 200)
8
9# Train the model
10history = model.fit(
11 x_train, y_train,
12 epochs=10, # Number of training epochs
13 batch_size=32, # Samples per gradient update
14 validation_split=0.2, # Use 20% of training data for validation
15 verbose=1 # Progress bar (0=silent, 1=progress, 2=one line per epoch)
16)
17
18# With separate validation data
19history = model.fit(
20 x_train, y_train,
21 epochs=10,
22 batch_size=32,
23 validation_data=(x_test, y_test),
24 verbose=1
25)
26
27# Output during training:
28# Epoch 1/10
29# 25/25 [==============================] - 1s 20ms/step - loss: 2.3026 - accuracy: 0.1125 - val_loss: 2.3026 - val_accuracy: 0.1000
30# Epoch 2/10
31# 25/25 [==============================] - 0s 8ms/step - loss: 2.2892 - accuracy: 0.1375 - val_loss: 2.2892 - val_accuracy: 0.1500
32# ...Understanding Hyperparameters:
1# Evaluate on test data
2test_loss, test_accuracy = model.evaluate(x_test, y_test, verbose=0)
3
4print(f"Test Loss: {test_loss:.4f}")
5print(f"Test Accuracy: {test_accuracy:.4f}")
6
7# Output:
8# Test Loss: 2.2845
9# Test Accuracy: 0.14501# Make predictions on new data
2predictions = model.predict(x_test[:5])
3
4print("Predictions shape:", predictions.shape) # (5, 10)
5print("First prediction probabilities:")
6print(predictions[0])
7# [0.1, 0.05, 0.15, 0.08, 0.12, 0.09, 0.11, 0.1, 0.1, 0.1]
8
9# Get class with highest probability
10predicted_classes = np.argmax(predictions, axis=1)
11print(f"Predicted classes: {predicted_classes}")
12# [2, 4, 0, 1, 3]
13
14# Compare with true labels
15print(f"True classes: {y_test[:5]}")1import matplotlib.pyplot as plt
2
3# history object contains training metrics
4print("Available metrics:", history.history.keys())
5# dict_keys(['loss', 'accuracy', 'val_loss', 'val_accuracy'])
6
7# Plot training & validation loss
8plt.figure(figsize=(12, 4))
9
10plt.subplot(1, 2, 1)
11plt.plot(history.history['loss'], label='Training Loss')
12plt.plot(history.history['val_loss'], label='Validation Loss')
13plt.xlabel('Epoch')
14plt.ylabel('Loss')
15plt.title('Loss Over Time')
16plt.legend()
17plt.grid(True, alpha=0.3)
18
19plt.subplot(1, 2, 2)
20plt.plot(history.history['accuracy'], label='Training Accuracy')
21plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
22plt.xlabel('Epoch')
23plt.ylabel('Accuracy')
24plt.title('Accuracy Over Time')
25plt.legend()
26plt.grid(True, alpha=0.3)
27
28plt.tight_layout()
29plt.show()1# Early stopping: stop when validation loss stops improving
2early_stopping = keras.callbacks.EarlyStopping(
3 monitor='val_loss',
4 patience=3, # Wait 3 epochs for improvement
5 restore_best_weights=True
6)
7
8# Model checkpoint: save best model
9checkpoint = keras.callbacks.ModelCheckpoint(
10 'best_model.h5',
11 monitor='val_accuracy',
12 save_best_only=True,
13 verbose=1
14)
15
16# Learning rate scheduler
17lr_scheduler = keras.callbacks.ReduceLROnPlateau(
18 monitor='val_loss',
19 factor=0.5, # Reduce LR by half
20 patience=2,
21 verbose=1
22)
23
24# Train with callbacks
25history = model.fit(
26 x_train, y_train,
27 epochs=50,
28 batch_size=32,
29 validation_data=(x_test, y_test),
30 callbacks=[early_stopping, checkpoint, lr_scheduler],
31 verbose=1
32)1# Save entire model (architecture + weights + optimizer state)
2model.save('my_model.h5')
3
4# Load model
5loaded_model = keras.models.load_model('my_model.h5')
6
7# Save only weights
8model.save_weights('model_weights.h5')
9
10# Load weights (model architecture must be defined first)
11model.load_weights('model_weights.h5')
12
13# Save as TensorFlow SavedModel format (recommended for production)
14model.save('saved_model/')
15
16# Load SavedModel
17loaded_model = keras.models.load_model('saved_model/')Key Insight: Keras provides a complete training pipeline:
This workflow handles batching, epochs, validation, and metrics automatically!
Build, train, and evaluate a neural network to recognize handwritten digits from the famous MNIST dataset. This comprehensive example demonstrates the complete TensorFlow workflow from data loading to prediction visualization.
1# Complete MNIST Digit Classifier with TensorFlow
2# A real-world deep learning project demonstrating the full workflow
3
4import tensorflow as tf
5from tensorflow import keras
6import numpy as np
7import matplotlib.pyplot as plt
8
9print("=" * 70)
10print("MNIST DIGIT CLASSIFICATION WITH TENSORFLOW")
11print("=" * 70)
12print(f"TensorFlow version: {tf.__version__}")
13print(f"GPU available: {len(tf.config.list_physical_devices('GPU')) > 0}")
14print()
15
16# ============================================================================
17# STEP 1: LOAD AND EXPLORE DATA
18# ============================================================================
19
20print("STEP 1: Loading MNIST Dataset")
21print("-" * 70)
22
23# Load MNIST dataset (built into Keras)
24(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
25
26print(f"Training images: {x_train.shape}") # (60000, 28, 28)
27print(f"Training labels: {y_train.shape}") # (60000,)
28print(f"Test images: {x_test.shape}") # (10000, 28, 28)
29print(f"Test labels: {y_test.shape}") # (10000,)
30print()
31
32# Explore the data
33print("Data Exploration:")
34print(f" Image shape: {x_train[0].shape}")
35print(f" Pixel value range: [{x_train.min()}, {x_train.max()}]")
36print(f" Number of classes: {len(np.unique(y_train))}")
37print(f" Classes: {np.unique(y_train)}")
38print()
39
40# Visualize sample images
41fig, axes = plt.subplots(2, 5, figsize=(12, 5))
42fig.suptitle('Sample MNIST Digits', fontsize=16, fontweight='bold')
43
44for i, ax in enumerate(axes.flat):
45 ax.imshow(x_train[i], cmap='gray')
46 ax.set_title(f'Label: {y_train[i]}', fontweight='bold')
47 ax.axis('off')
48
49plt.tight_layout()
50plt.show()
51
52# ============================================================================
53# STEP 2: PREPROCESS DATA
54# ============================================================================
55
56print("\nSTEP 2: Preprocessing Data")
57print("-" * 70)
58
59# Normalize pixel values to [0, 1]
60x_train = x_train.astype('float32') / 255.0
61x_test = x_test.astype('float32') / 255.0
62
63print(f"After normalization:")
64print(f" Pixel range: [{x_train.min():.2f}, {x_train.max():.2f}]")
65print()
66
67# Reshape for fully connected network: (28, 28) → (784,)
68x_train_flat = x_train.reshape(-1, 784)
69x_test_flat = x_test.reshape(-1, 784)
70
71print(f"Flattened shapes:")
72print(f" Training: {x_train_flat.shape}") # (60000, 784)
73print(f" Test: {x_test_flat.shape}") # (10000, 784)
74print()
75
76# ============================================================================
77# STEP 3: BUILD NEURAL NETWORK
78# ============================================================================
79
80print("STEP 3: Building Neural Network")
81print("-" * 70)
82
83model = keras.Sequential([
84 keras.layers.Dense(128, activation='relu', input_shape=(784,)),
85 keras.layers.Dropout(0.2),
86 keras.layers.Dense(64, activation='relu'),
87 keras.layers.Dropout(0.2),
88 keras.layers.Dense(10, activation='softmax')
89], name='mnist_classifier')
90
91print("Model Architecture:")
92model.summary()
93print()
94
95# Count parameters
96total_params = model.count_params()
97print(f"Total trainable parameters: {total_params:,}")
98print()
99
100# ============================================================================
101# STEP 4: COMPILE MODEL
102# ============================================================================
103
104print("STEP 4: Compiling Model")
105print("-" * 70)
106
107model.compile(
108 optimizer='adam',
109 loss='sparse_categorical_crossentropy',
110 metrics=['accuracy']
111)
112
113print("Optimizer: Adam")
114print("Loss: Sparse Categorical Crossentropy")
115print("Metrics: Accuracy")
116print()
117
118# ============================================================================
119# STEP 5: TRAIN MODEL
120# ============================================================================
121
122print("STEP 5: Training Model")
123print("-" * 70)
124
125# Callbacks for better training
126early_stopping = keras.callbacks.EarlyStopping(
127 monitor='val_loss',
128 patience=3,
129 restore_best_weights=True,
130 verbose=1
131)
132
133# Train the model
134history = model.fit(
135 x_train_flat, y_train,
136 epochs=15,
137 batch_size=128,
138 validation_split=0.2,
139 callbacks=[early_stopping],
140 verbose=1
141)
142
143print("\nTraining Complete!")
144print()
145
146# ============================================================================
147# STEP 6: EVALUATE MODEL
148# ============================================================================
149
150print("STEP 6: Evaluating Model")
151print("-" * 70)
152
153# Evaluate on test set
154test_loss, test_accuracy = model.evaluate(x_test_flat, y_test, verbose=0)
155
156print(f"Test Loss: {test_loss:.4f}")
157print(f"Test Accuracy: {test_accuracy:.4f} ({test_accuracy*100:.2f}%)")
158print()
159
160# Evaluate on training set
161train_loss, train_accuracy = model.evaluate(x_train_flat, y_train, verbose=0)
162
163print(f"Training Loss: {train_loss:.4f}")
164print(f"Training Accuracy: {train_accuracy:.4f} ({train_accuracy*100:.2f}%)")
165print()
166
167# ============================================================================
168# STEP 7: MAKE PREDICTIONS
169# ============================================================================
170
171print("STEP 7: Making Predictions")
172print("-" * 70)
173
174# Predict on test set
175predictions = model.predict(x_test_flat[:10], verbose=0)
176
177print("Prediction shape:", predictions.shape) # (10, 10)
178print()
179
180# Analyze first prediction
181print("First image prediction probabilities:")
182for i, prob in enumerate(predictions[0]):
183 bar = '█' * int(prob * 50)
184 print(f" Digit {i}: {prob:.4f} {bar}")
185
186predicted_digit = np.argmax(predictions[0])
187true_digit = y_test[0]
188print(f"\nPredicted: {predicted_digit}, True: {true_digit}")
189print()
190
191# ============================================================================
192# STEP 8: VISUALIZE RESULTS
193# ============================================================================
194
195print("STEP 8: Visualizing Results")
196print("-" * 70)
197
198# Plot training history
199fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))
200
201ax1.plot(history.history['loss'], label='Training Loss', linewidth=2)
202ax1.plot(history.history['val_loss'], label='Validation Loss', linewidth=2)
203ax1.set_xlabel('Epoch', fontweight='bold')
204ax1.set_ylabel('Loss', fontweight='bold')
205ax1.set_title('Training & Validation Loss', fontweight='bold', fontsize=14)
206ax1.legend()
207ax1.grid(True, alpha=0.3)
208
209ax2.plot(history.history['accuracy'], label='Training Accuracy', linewidth=2)
210ax2.plot(history.history['val_accuracy'], label='Validation Accuracy', linewidth=2)
211ax2.set_xlabel('Epoch', fontweight='bold')
212ax2.set_ylabel('Accuracy', fontweight='bold')
213ax2.set_title('Training & Validation Accuracy', fontweight='bold', fontsize=14)
214ax2.legend()
215ax2.grid(True, alpha=0.3)
216
217plt.tight_layout()
218plt.show()
219
220# Visualize predictions
221fig, axes = plt.subplots(2, 5, figsize=(12, 6))
222fig.suptitle('Model Predictions on Test Set', fontsize=16, fontweight='bold')
223
224for i, ax in enumerate(axes.flat):
225 # Show image
226 ax.imshow(x_test[i], cmap='gray')
227
228 # Get prediction
229 pred = model.predict(x_test_flat[i:i+1], verbose=0)
230 predicted = np.argmax(pred)
231 confidence = np.max(pred)
232 true_label = y_test[i]
233
234 # Color: green if correct, red if wrong
235 color = 'green' if predicted == true_label else 'red'
236
237 ax.set_title(f'Pred: {predicted} ({confidence:.2f})\nTrue: {true_label}',
238 color=color, fontweight='bold', fontsize=10)
239 ax.axis('off')
240
241plt.tight_layout()
242plt.show()
243
244# Confusion matrix visualization
245from sklearn.metrics import confusion_matrix
246import seaborn as sns
247
248# Get all predictions
249all_predictions = model.predict(x_test_flat, verbose=0)
250predicted_classes = np.argmax(all_predictions, axis=1)
251
252# Compute confusion matrix
253cm = confusion_matrix(y_test, predicted_classes)
254
255plt.figure(figsize=(10, 8))
256sns.heatmap(cm, annot=True, fmt='d', cmap='Blues',
257 xticklabels=range(10), yticklabels=range(10))
258plt.xlabel('Predicted Label', fontweight='bold', fontsize=12)
259plt.ylabel('True Label', fontweight='bold', fontsize=12)
260plt.title('Confusion Matrix: MNIST Classification',
261 fontweight='bold', fontsize=14)
262plt.show()
263
264# ============================================================================
265# SUMMARY
266# ============================================================================
267
268print("\n" + "=" * 70)
269print("PROJECT SUMMARY")
270print("=" * 70)
271print()
272print(f"✓ Dataset: MNIST (70,000 handwritten digits)")
273print(f"✓ Model: 3-layer neural network (128 → 64 → 10 neurons)")
274print(f"✓ Parameters: {total_params:,} trainable weights")
275print(f"✓ Training: {len(history.history['loss'])} epochs with early stopping")
276print(f"✓ Test Accuracy: {test_accuracy*100:.2f}%")
277print()
278print("TensorFlow Concepts Demonstrated:")
279print(" 1️⃣ Data loading with keras.datasets")
280print(" 2️⃣ Sequential API for model building")
281print(" 3️⃣ compile(), fit(), evaluate(), predict() workflow")
282print(" 4️⃣ Callbacks (EarlyStopping)")
283print(" 5️⃣ Training visualization and analysis")
284print()
285print("=" * 70)
286print("🎉 You built a real deep learning model with TensorFlow!")
287print("=" * 70)This comprehensive example demonstrates the complete TensorFlow workflow for building a production-quality image classifier.
1. Real-World Dataset
MNIST contains 70,000 images of handwritten digits (0-9), split into:
This is the "Hello World" of computer vision—simple enough to train quickly but complex enough to demonstrate real concepts.
2. Complete Neural Network Pipeline
Data Preprocessing:
Model Architecture:
Training Configuration:
3. TensorFlow Features Demonstrated
Sequential API:
Simple, readable model definition—perfect for feedforward networks.
Built-in Datasets:
keras.datasets.mnist.load_data() loads data automatically.
Training Workflow:
compile(): Configure optimizer and lossfit(): Train model with automatic batchingevaluate(): Test performancepredict(): Make predictionsCallbacks:
Early stopping monitors validation loss and stops when improvement plateaus.
4. Performance Analysis
Expected Results:
Visualization:
1. Production-Ready Pattern
This workflow scales to any supervised learning problem:
2. GPU Acceleration
TensorFlow automatically uses GPU when available—no code changes needed!
3. Transfer to Other Domains
Replace MNIST with:
Training Dynamics:
Model Capacity:
Generalization:
This example is your template for building deep learning models with TensorFlow!
Excellent work! You've mastered TensorFlow fundamentals and built a real image classifier with 98% accuracy. You now understand tensors, automatic differentiation, the Keras API, and the complete training workflow.
Next: In Lesson 4: Introduction to PyTorch, we'll explore Facebook's deep learning framework—TensorFlow's main competitor. You'll learn:
PyTorch is favored in research for its flexibility and intuitive API. After learning both frameworks, you'll be equipped to use whichever best fits your project!
Your AI Journey Continues → Ready to master PyTorch? Let's go! 🚀
Your progress is saved in this browser, on this device only.