Learn the fundamentals of computer vision, understanding how machines interpret visual data, the CV pipeline, and setting up your development environment with OpenCV and PyTorch.
Computer Vision (CV) is one of the most exciting fields in artificial intelligence, enabling machines to interpret and understand visual information from the world. From self-driving cars to medical image analysis, face recognition to augmented reality, computer vision powers countless applications that are transforming industries.
At its core, computer vision aims to replicate the human visual system's ability to perceive, process, and understand images and videos. This involves extracting meaningful information from visual data and making decisions based on that analysis.
In this lesson, we'll explore the fundamentals of computer vision, understand how digital images work, learn about the main CV tasks, and set up our development environment with industry-standard tools like OpenCV and PyTorch.
Computer Vision is a field of artificial intelligence that trains computers to interpret and understand the visual world. Using digital images from cameras, videos, and deep learning models, machines can accurately identify and classify objects, and then react to what they "see."
Key Applications:
The Evolution of Computer Vision:
1# Timeline of Computer Vision milestones
2cv_evolution = {
3 "1960s": "Early pattern recognition research",
4 "1970s": "Edge detection algorithms (Sobel, Canny)",
5 "1980s": "Image segmentation, shape analysis",
6 "1990s": "Face detection (Viola-Jones), SIFT features",
7 "2000s": "HOG descriptors, real-time object detection",
8 "2012": "AlexNet - Deep learning revolution begins",
9 "2014": "VGGNet, GoogLeNet - Deeper networks",
10 "2015": "ResNet - Skip connections enable very deep nets",
11 "2017": "Transformer attention in vision",
12 "2020s": "Vision Transformers (ViT), CLIP, diffusion models"
13}
14
15for year, milestone in cv_evolution.items():
16 print(f"{year}: {milestone}")The field has evolved from simple rule-based approaches to sophisticated deep learning systems capable of superhuman performance in specific tasks.
Understanding how images are stored and represented digitally is fundamental to computer vision. An image is essentially a 2D array of pixels, where each pixel contains intensity or color values.
Pixel and Image Structure:
1import numpy as np
2import cv2
3
4# A grayscale image is a 2D array
5grayscale_image = np.array([
6 [0, 50, 100, 150, 200, 255],
7 [0, 50, 100, 150, 200, 255],
8 [0, 50, 100, 150, 200, 255]
9], dtype=np.uint8)
10
11print(f"Shape: {grayscale_image.shape}") # (3, 6) - height x width
12print(f"Data type: {grayscale_image.dtype}") # uint8 (0-255)
13
14# A color image has 3 channels (RGB or BGR)
15color_image = np.zeros((100, 100, 3), dtype=np.uint8)
16
17# Set a red pixel (OpenCV uses BGR format)
18color_image[50, 50] = [0, 0, 255] # BGR: Blue=0, Green=0, Red=255
19
20print(f"Color image shape: {color_image.shape}") # (100, 100, 3)Image Properties:
1def analyze_image(image_path):
2 """Analyze key properties of an image."""
3 img = cv2.imread(image_path)
4
5 if img is None:
6 raise ValueError(f"Could not load image: {image_path}")
7
8 height, width = img.shape[:2]
9 channels = img.shape[2] if len(img.shape) > 2 else 1
10
11 print(f"Dimensions: {width}x{height}")
12 print(f"Channels: {channels}")
13 print(f"Total pixels: {width * height:,}")
14 print(f"Memory size: {img.nbytes / 1024:.2f} KB")
15 print(f"Pixel value range: [{img.min()}, {img.max()}]")
16 print(f"Mean intensity: {img.mean():.2f}")
17
18 return img
19
20# Working with PyTorch tensors
21import torch
22from torchvision import transforms
23
24def image_to_tensor(image):
25 """Convert OpenCV image to PyTorch tensor."""
26 # OpenCV loads as BGR, convert to RGB
27 rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
28
29 # Convert to tensor and normalize
30 transform = transforms.Compose([
31 transforms.ToTensor(), # Converts to [C, H, W] format, scales to [0, 1]
32 transforms.Normalize(
33 mean=[0.485, 0.456, 0.406], # ImageNet statistics
34 std=[0.229, 0.224, 0.225]
35 )
36 ])
37
38 tensor = transform(rgb_image)
39 print(f"Tensor shape: {tensor.shape}") # [3, H, W]
40 print(f"Tensor range: [{tensor.min():.3f}, {tensor.max():.3f}]")
41
42 return tensorKey Concepts:
A typical computer vision system follows a structured pipeline from raw image input to final output. Understanding this pipeline helps design robust CV solutions.
Complete CV Pipeline:
1import cv2
2import numpy as np
3import torch
4import torch.nn as nn
5from torchvision import transforms, models
6
7class ComputerVisionPipeline:
8 """A complete CV pipeline for image classification."""
9
10 def __init__(self, model_name='resnet18', num_classes=10):
11 self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
12
13 # Stage 1: Preprocessing transforms
14 self.preprocess = transforms.Compose([
15 transforms.ToPILImage(),
16 transforms.Resize(256),
17 transforms.CenterCrop(224),
18 transforms.ToTensor(),
19 transforms.Normalize(
20 mean=[0.485, 0.456, 0.406],
21 std=[0.229, 0.224, 0.225]
22 )
23 ])
24
25 # Stage 2: Load model
26 self.model = models.resnet18(weights='IMAGENET1K_V1')
27 self.model.fc = nn.Linear(512, num_classes)
28 self.model = self.model.to(self.device)
29 self.model.eval()
30
31 def acquire_image(self, source):
32 """Stage 1: Image Acquisition"""
33 if isinstance(source, str):
34 # Load from file
35 image = cv2.imread(source)
36 elif isinstance(source, np.ndarray):
37 # Already a numpy array
38 image = source
39 else:
40 raise ValueError("Source must be path or numpy array")
41
42 # Convert BGR to RGB
43 image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
44 return image
45
46 def preprocess_image(self, image):
47 """Stage 2: Preprocessing"""
48 # Apply transformations
49 tensor = self.preprocess(image)
50 # Add batch dimension
51 tensor = tensor.unsqueeze(0)
52 return tensor.to(self.device)
53
54 def extract_features(self, tensor):
55 """Stage 3: Feature Extraction (done by CNN layers)"""
56 # Get features before final classification layer
57 features = []
58 x = tensor
59
60 for name, layer in self.model.named_children():
61 if name == 'fc':
62 break
63 x = layer(x)
64 if name in ['layer3', 'layer4']:
65 features.append(x)
66
67 return x, features
68
69 def classify(self, features):
70 """Stage 4: Classification"""
71 # Flatten and pass through FC layer
72 x = torch.flatten(features, 1)
73 logits = self.model.fc(x)
74 probabilities = torch.softmax(logits, dim=1)
75 return probabilities
76
77 def postprocess(self, probabilities, class_names=None):
78 """Stage 5: Post-processing"""
79 probs = probabilities.cpu().detach().numpy()[0]
80
81 top_k = 5
82 top_indices = np.argsort(probs)[-top_k:][::-1]
83
84 results = []
85 for idx in top_indices:
86 name = class_names[idx] if class_names else f"Class {idx}"
87 results.append({
88 'class': name,
89 'confidence': float(probs[idx])
90 })
91
92 return results
93
94 def run(self, source, class_names=None):
95 """Execute full pipeline"""
96 image = self.acquire_image(source)
97 tensor = self.preprocess_image(image)
98
99 with torch.no_grad():
100 features, _ = self.extract_features(tensor)
101 probs = self.classify(features)
102
103 return self.postprocess(probs, class_names)
104
105# Usage example
106pipeline = ComputerVisionPipeline(num_classes=1000)
107# results = pipeline.run("image.jpg", class_names=imagenet_classes)Pipeline Stages Explained:
Computer vision encompasses several fundamental tasks, each addressing different aspects of visual understanding.
Image Classification:
1import torch
2from torchvision import models, transforms
3from PIL import Image
4
5def classify_image(image_path, top_k=5):
6 """Classify an image using a pretrained model."""
7 # Load pretrained model
8 model = models.resnet50(weights='IMAGENET1K_V2')
9 model.eval()
10
11 # Preprocessing
12 preprocess = transforms.Compose([
13 transforms.Resize(256),
14 transforms.CenterCrop(224),
15 transforms.ToTensor(),
16 transforms.Normalize(
17 mean=[0.485, 0.456, 0.406],
18 std=[0.229, 0.224, 0.225]
19 )
20 ])
21
22 # Load and preprocess image
23 image = Image.open(image_path).convert('RGB')
24 input_tensor = preprocess(image).unsqueeze(0)
25
26 # Inference
27 with torch.no_grad():
28 output = model(input_tensor)
29 probabilities = torch.softmax(output, dim=1)
30
31 # Get top-k predictions
32 top_probs, top_indices = torch.topk(probabilities, top_k)
33
34 return list(zip(top_indices[0].tolist(), top_probs[0].tolist()))Object Detection:
1import torch
2from torchvision.models.detection import fasterrcnn_resnet50_fpn_v2
3
4def detect_objects(image_tensor, confidence_threshold=0.5):
5 """Detect objects in an image."""
6 # Load pretrained Faster R-CNN
7 model = fasterrcnn_resnet50_fpn_v2(weights='DEFAULT')
8 model.eval()
9
10 with torch.no_grad():
11 predictions = model([image_tensor])
12
13 # Filter by confidence
14 pred = predictions[0]
15 mask = pred['scores'] > confidence_threshold
16
17 boxes = pred['boxes'][mask].cpu().numpy()
18 labels = pred['labels'][mask].cpu().numpy()
19 scores = pred['scores'][mask].cpu().numpy()
20
21 return {
22 'boxes': boxes, # [x1, y1, x2, y2] format
23 'labels': labels, # Class indices
24 'scores': scores # Confidence scores
25 }
26
27# COCO class names (80 classes)
28COCO_CLASSES = [
29 '__background__', 'person', 'bicycle', 'car', 'motorcycle',
30 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light',
31 # ... more classes
32]Semantic Segmentation:
1from torchvision.models.segmentation import deeplabv3_resnet101
2
3def segment_image(image_tensor):
4 """Perform semantic segmentation."""
5 model = deeplabv3_resnet101(weights='DEFAULT')
6 model.eval()
7
8 with torch.no_grad():
9 output = model(image_tensor.unsqueeze(0))
10
11 # Get segmentation mask
12 segmentation_map = output['out'].argmax(1).squeeze().cpu().numpy()
13
14 return segmentation_map
15
16# Visualize segmentation
17import matplotlib.pyplot as plt
18
19def visualize_segmentation(image, seg_map):
20 """Visualize segmentation results."""
21 fig, axes = plt.subplots(1, 2, figsize=(12, 6))
22
23 axes[0].imshow(image)
24 axes[0].set_title('Original Image')
25 axes[0].axis('off')
26
27 axes[1].imshow(seg_map, cmap='tab20')
28 axes[1].set_title('Segmentation Map')
29 axes[1].axis('off')
30
31 plt.tight_layout()
32 plt.show()Instance Segmentation:
1from torchvision.models.detection import maskrcnn_resnet50_fpn_v2
2
3def instance_segmentation(image_tensor, threshold=0.5):
4 """Perform instance segmentation."""
5 model = maskrcnn_resnet50_fpn_v2(weights='DEFAULT')
6 model.eval()
7
8 with torch.no_grad():
9 predictions = model([image_tensor])
10
11 pred = predictions[0]
12 mask = pred['scores'] > threshold
13
14 return {
15 'boxes': pred['boxes'][mask],
16 'labels': pred['labels'][mask],
17 'scores': pred['scores'][mask],
18 'masks': pred['masks'][mask] # Instance masks
19 }Create a complete development environment for computer vision projects with OpenCV, PyTorch, and essential utilities.
1"""
2Complete Computer Vision Environment Setup and Verification
3This script sets up and verifies all essential CV tools.
4"""
5
6import sys
7import os
8
9# =============================================================================
10# INSTALLATION COMMANDS (run in terminal)
11# =============================================================================
12"""
13# Create virtual environment
14python -m venv cv_env
15source cv_env/bin/activate # Linux/Mac
16cv_env\Scripts\activate # Windows
17
18# Install core packages
19pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
20pip install opencv-python opencv-contrib-python
21pip install numpy pandas matplotlib seaborn
22pip install scikit-learn scikit-image
23pip install pillow albumentations
24pip install jupyter notebook
25
26# Optional: Additional tools
27pip install timm # PyTorch Image Models
28pip install ultralytics # YOLOv8
29pip install transformers # Vision Transformers
30"""
31
32# =============================================================================
33# ENVIRONMENT VERIFICATION
34# =============================================================================
35
36def verify_environment():
37 """Verify all required packages are installed correctly."""
38
39 print("=" * 60)
40 print("Computer Vision Environment Verification")
41 print("=" * 60)
42
43 # Check Python version
44 print(f"\nPython Version: {sys.version}")
45
46 packages = {}
47
48 # NumPy
49 try:
50 import numpy as np
51 packages['NumPy'] = np.__version__
52 except ImportError:
53 packages['NumPy'] = 'NOT INSTALLED'
54
55 # OpenCV
56 try:
57 import cv2
58 packages['OpenCV'] = cv2.__version__
59 except ImportError:
60 packages['OpenCV'] = 'NOT INSTALLED'
61
62 # PyTorch
63 try:
64 import torch
65 packages['PyTorch'] = torch.__version__
66 packages['CUDA Available'] = str(torch.cuda.is_available())
67 if torch.cuda.is_available():
68 packages['CUDA Version'] = torch.version.cuda
69 packages['GPU'] = torch.cuda.get_device_name(0)
70 except ImportError:
71 packages['PyTorch'] = 'NOT INSTALLED'
72
73 # TorchVision
74 try:
75 import torchvision
76 packages['TorchVision'] = torchvision.__version__
77 except ImportError:
78 packages['TorchVision'] = 'NOT INSTALLED'
79
80 # PIL/Pillow
81 try:
82 from PIL import Image
83 import PIL
84 packages['Pillow'] = PIL.__version__
85 except ImportError:
86 packages['Pillow'] = 'NOT INSTALLED'
87
88 # Matplotlib
89 try:
90 import matplotlib
91 packages['Matplotlib'] = matplotlib.__version__
92 except ImportError:
93 packages['Matplotlib'] = 'NOT INSTALLED'
94
95 # scikit-image
96 try:
97 import skimage
98 packages['scikit-image'] = skimage.__version__
99 except ImportError:
100 packages['scikit-image'] = 'NOT INSTALLED'
101
102 # Print results
103 print("\n" + "-" * 40)
104 print("Package Versions:")
105 print("-" * 40)
106 for package, version in packages.items():
107 status = "OK" if version != 'NOT INSTALLED' else "MISSING"
108 print(f"{package:20} {version:20} [{status}]")
109
110 return packages
111
112
113# =============================================================================
114# BASIC CV OPERATIONS TEST
115# =============================================================================
116
117def test_basic_operations():
118 """Test basic computer vision operations."""
119 import numpy as np
120 import cv2
121 import torch
122 from torchvision import transforms
123
124 print("\n" + "=" * 60)
125 print("Testing Basic CV Operations")
126 print("=" * 60)
127
128 # Create a test image
129 test_image = np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8)
130 print(f"\n1. Created test image: {test_image.shape}")
131
132 # Test OpenCV operations
133 gray = cv2.cvtColor(test_image, cv2.COLOR_BGR2GRAY)
134 print(f"2. Converted to grayscale: {gray.shape}")
135
136 blurred = cv2.GaussianBlur(gray, (5, 5), 0)
137 print(f"3. Applied Gaussian blur: {blurred.shape}")
138
139 edges = cv2.Canny(blurred, 50, 150)
140 print(f"4. Detected edges (Canny): {edges.shape}")
141
142 # Test PyTorch operations
143 transform = transforms.Compose([
144 transforms.ToPILImage(),
145 transforms.Resize((256, 256)),
146 transforms.ToTensor(),
147 transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
148 ])
149
150 tensor = transform(test_image)
151 print(f"5. Converted to PyTorch tensor: {tensor.shape}")
152
153 # Test GPU availability
154 if torch.cuda.is_available():
155 tensor_gpu = tensor.cuda()
156 print(f"6. Moved tensor to GPU: {tensor_gpu.device}")
157 else:
158 print("6. GPU not available, using CPU")
159
160 print("\nAll basic operations completed successfully!")
161
162
163# =============================================================================
164# LOAD PRETRAINED MODEL TEST
165# =============================================================================
166
167def test_pretrained_model():
168 """Test loading and using a pretrained model."""
169 import torch
170 from torchvision import models
171
172 print("\n" + "=" * 60)
173 print("Testing Pretrained Model Loading")
174 print("=" * 60)
175
176 # Load ResNet18 (smaller model for quick test)
177 print("\nLoading ResNet18...")
178 model = models.resnet18(weights='IMAGENET1K_V1')
179 model.eval()
180 print(f"Model loaded successfully!")
181 print(f"Number of parameters: {sum(p.numel() for p in model.parameters()):,}")
182
183 # Test inference
184 dummy_input = torch.randn(1, 3, 224, 224)
185
186 with torch.no_grad():
187 output = model(dummy_input)
188
189 print(f"Input shape: {dummy_input.shape}")
190 print(f"Output shape: {output.shape}")
191 print(f"Top prediction: class {output.argmax().item()}")
192
193 print("\nPretrained model test completed successfully!")
194
195
196# =============================================================================
197# RUN ALL TESTS
198# =============================================================================
199
200if __name__ == "__main__":
201 packages = verify_environment()
202
203 # Only run tests if core packages are installed
204 if packages.get('OpenCV') != 'NOT INSTALLED' and \
205 packages.get('PyTorch') != 'NOT INSTALLED':
206 test_basic_operations()
207 test_pretrained_model()
208 else:
209 print("\nPlease install missing packages before running tests.")
210
211 print("\n" + "=" * 60)
212 print("Environment verification complete!")
213 print("=" * 60)This comprehensive setup script verifies your computer vision environment by checking all required packages, testing basic CV operations with OpenCV, and confirming PyTorch model loading. Running this script ensures your development environment is correctly configured for computer vision projects.
In the next lesson, we'll dive deep into image processing fundamentals, covering filtering operations, edge detection algorithms, morphological transformations, and color space conversions.
Your progress is saved in this browser, on this device only.