1 · Overview

Goal. Train a convolutional neural network that maps a small RGB image to one of ten object categories (airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck) and learn the full engineering loop around it: a reproducible data pipeline, a principled architecture, a correct training loop, honest evaluation, and an interpretation of what the network actually learned. To anchor "is deep learning worth it here?", we compare against a strong classical baseline — Histogram of Oriented Gradients features fed to a linear Support Vector Machine — built from the feature-engineering ideas of Module 5.

Why CIFAR-10. It is the canonical first real classification benchmark: large enough that hand-tuned features struggle, small enough (32×32) to train a competitive CNN on a single GPU in minutes. The exact same code runs on MNIST by changing the dataset and the input-channel count — the pipeline is dataset-agnostic.

Sessions exercised. This project is the practical payoff of Module 6:

• Session 21 — Convolutional neural networks (introduction, applications, functional perspective): why convolution + weight sharing is the right inductive bias for images. S21 in outline
• Session 22 — CNNs II (deep-learning frameworks, implementing a CNN from scratch): we build, train and debug the network in PyTorch exactly as the session prescribes. S22 in outline
• Session 23 — CNNs III (advanced architectures: AlexNet, VGG, GoogLeNet, ResNet, U-Net): our model is a scaled-down VGG; the extensions section maps the path to ResNet and beyond. S23 in outline

It also reaches back to the convolution and edge/feature material of Sessions 16–19 (the learned kernels are the hand-designed kernels of Module 4, now trained from data) and forward to detection/segmentation in Sessions 25–26.

2 · Dataset & input pipeline

CIFAR-10 is 60,000 colour images at $32\times32\times3$, split into 50,000 training and 10,000 test images, 6,000 per class. Each pixel is a uint8 in $[0,255]$; before feeding the network we convert to a float tensor in $[0,1]$ and then standardise per channel using the dataset's mean $\mu_c$ and standard deviation $\sigma_c$:

Standardisation centres each channel near zero with unit variance, which keeps activations and gradients well-scaled and lets the optimiser use a single learning rate across channels.

Data augmentation

With only 5,000 images per class, the network will memorise the training set unless we artificially enlarge it. Augmentation applies label-preserving random transforms on the fly so the model never sees exactly the same image twice — a regulariser that encodes our prior that class identity is invariant to small shifts and horizontal mirroring. We apply random crops (with 4-px reflection padding) and random horizontal flips to the training set only; the test set gets just the deterministic normalisation, so evaluation is fair.

import torch
from torch.utils.data import DataLoader
from torchvision import datasets, transforms

MEAN = (0.4914, 0.4822, 0.4465)
STD  = (0.2470, 0.2435, 0.2616)

# Training transforms: random crop + flip (augmentation) THEN normalise
train_tf = transforms.Compose([
    transforms.RandomCrop(32, padding=4, padding_mode="reflect"),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),                # uint8 [0,255] -> float [0,1], HWC -> CHW
    transforms.Normalize(MEAN, STD),
])
# Test transforms: deterministic only — no augmentation, no leakage
test_tf = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(MEAN, STD),
])

train_set = datasets.CIFAR10("./data", train=True,  download=True, transform=train_tf)
test_set  = datasets.CIFAR10("./data", train=False, download=True, transform=test_tf)

train_loader = DataLoader(train_set, batch_size=128, shuffle=True,
                          num_workers=4, pin_memory=True)
test_loader  = DataLoader(test_set,  batch_size=256, shuffle=False,
                          num_workers=4, pin_memory=True)

CLASSES = ("plane","car","bird","cat","deer",
           "dog","frog","horse","ship","truck")

Pipeline. torchvision downloads the dataset, the DataLoader batches and shuffles it, and transforms run lazily per sample on the worker processes. Demo · image input

3 · The CNN architecture & the dimension math

The model is a small VGG-style network: three convolutional stages (each two $3\times3$ convolutions + ReLU + batch-norm, then $2\times2$ max-pool that halves the spatial size and doubles the channel count), followed by a global pooling and a fully-connected classifier. This is exactly the conv → pool → FC pattern of Session 21, stacked.

The convolution operation

A single output feature map is a learned cross-correlation of the input with a kernel $K$ plus a bias, passed through a non-linearity. For input $I$ and kernel of size $k\times k$:

This is the same sum you scrubbed by hand in the convolution demo — the only difference is that here $K$ is a trainable parameter optimised to minimise the loss, and there are many kernels per layer producing a stack of feature maps.

Output-size formula

A convolution (or pooling) layer with kernel size $k$, padding $p$, stride $s$ on an input of spatial size $W$ produces output size:

With our "same" convolutions ($k=3,\,p=1,\,s=1$) the spatial size is preserved: $\lfloor(32-3+2)/1\rfloor+1 = 32$. Each $2\times2$ max-pool ($k=2,\,s=2,\,p=0$) halves it: $\lfloor(32-2)/2\rfloor+1 = 16$. So the spatial size walks $32 \to 16 \to 8 \to 4$ across the three stages while channels grow $3 \to 64 \to 128 \to 256$.

Parameter count of a conv layer

A convolution from $C_{\text{in}}$ input channels to $C_{\text{out}}$ output channels with $k\times k$ kernels has

e.g. the first $3\times3$ conv $3\to64$ uses $(3\cdot3\cdot3+1)\cdot64 = 1{,}792$ parameters. Crucially this is independent of image size: the same kernel slides over every position (weight sharing), which is what makes CNNs vastly more parameter-efficient than a fully-connected net on pixels and gives them translation equivariance.

Total ≈ 1.15 M trainable parameters — tiny by modern standards, trainable in minutes, yet enough to comfortably beat any hand-engineered baseline on this task.

4 · Step-by-step implementation

4.1 · The model

Each stage is a small helper. BatchNorm2d after each conv stabilises and speeds up training by normalising activations; ReLU $f(z)=\max(0,z)$ is the non-linearity. We finish with global average pooling (average each $4\times4$ map to a single number) so the classifier sees a 256-vector regardless of input size.

import torch.nn as nn
import torch.nn.functional as F

class ConvStage(nn.Module):
    # two 3x3 convs (padding=1 keeps H,W) + BN + ReLU, then 2x2 max-pool
    def __init__(self, c_in, c_out):
        super().__init__()
        self.conv1 = nn.Conv2d(c_in,  c_out, kernel_size=3, padding=1, bias=False)
        self.bn1   = nn.BatchNorm2d(c_out)
        self.conv2 = nn.Conv2d(c_out, c_out, kernel_size=3, padding=1, bias=False)
        self.bn2   = nn.BatchNorm2d(c_out)
        self.pool  = nn.MaxPool2d(2, 2)

    def forward(self, x):
        x = F.relu(self.bn1(self.conv1(x)))
        x = F.relu(self.bn2(self.conv2(x)))
        return self.pool(x)

class SmallVGG(nn.Module):
    def __init__(self, num_classes=10):
        super().__init__()
        self.stage1 = ConvStage(3,   64)    # 32 -> 16
        self.stage2 = ConvStage(64,  128)   # 16 -> 8
        self.stage3 = ConvStage(128, 256)   # 8  -> 4
        self.gap    = nn.AdaptiveAvgPool2d(1)  # global average pool -> 256x1x1
        self.fc     = nn.Linear(256, num_classes)

    def forward(self, x):
        x = self.stage1(x)
        x = self.stage2(x)
        x = self.stage3(x)
        x = self.gap(x).flatten(1)        # (N,256,1,1) -> (N,256)
        return self.fc(x)                  # raw logits, shape (N,10)

Note. The network outputs raw logits — we do not apply softmax here, because nn.CrossEntropyLoss fuses log-softmax and the NLL for numerical stability.

4.2 · The loss — softmax + cross-entropy

The classifier emits a vector of logits $z\in\mathbb{R}^{10}$. Softmax turns logits into a probability distribution over classes:

For a one-hot true label $y$, the cross-entropy loss is the negative log-probability the model assigns to the correct class:

This is large when the model is confidently wrong and near zero when it is confidently right. A pleasing fact that makes the backward pass trivial: the gradient of the combined softmax-cross-entropy with respect to the logits is just the predicted-minus-true probability,

which is exactly what PyTorch's autograd computes for us.

4.3 · The training loop

We optimise with SGD + momentum and a cosine-annealed learning rate; Adam is a drop-in alternative (commented). Each step: forward pass → loss → backward() (backprop, i.e. the chain rule from Module 6) → optimiser step.

device = "cuda" if torch.cuda.is_available() else "cpu"
model  = SmallVGG().to(device)

criterion = nn.CrossEntropyLoss()            # fuses log-softmax + NLL
optimizer = torch.optim.SGD(model.parameters(), lr=0.1,
                            momentum=0.9, weight_decay=5e-4, nesterov=True)
# optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)   # alternative
EPOCHS = 40
scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max=EPOCHS)

def train_one_epoch(loader):
    model.train()
    running, correct, total = 0.0, 0, 0
    for images, labels in loader:
        images, labels = images.to(device), labels.to(device)
        optimizer.zero_grad()
        logits = model(images)               # forward
        loss   = criterion(logits, labels)   # softmax + cross-entropy
        loss.backward()                      # backprop (chain rule)
        optimizer.step()                     # SGD update
        running += loss.item() * images.size(0)
        correct += (logits.argmax(1) == labels).sum().item()
        total   += images.size(0)
    return running / total, correct / total

@torch.no_grad()
def evaluate(loader):
    model.eval()
    correct, total = 0, 0
    for images, labels in loader:
        images, labels = images.to(device), labels.to(device)
        preds = model(images).argmax(1)
        correct += (preds == labels).sum().item()
        total   += images.size(0)
    return correct / total

for epoch in range(EPOCHS):
    tr_loss, tr_acc = train_one_epoch(train_loader)
    te_acc = evaluate(test_loader)
    scheduler.step()
    print(f"epoch {epoch+1:2d}  loss {tr_loss:.3f}  "
          f"train_acc {tr_acc:.3f}  test_acc {te_acc:.3f}")

Why momentum + weight decay + cosine schedule. Momentum damps oscillation across steep ravines in the loss surface; weight decay ($L_2$) curbs overfitting; cosine annealing starts with large, exploratory steps and decays smoothly to a fine polish — together they reliably push a small VGG past 90% on CIFAR-10.

4.4 · Evaluation — confusion matrix & per-class metrics

import numpy as np
from sklearn.metrics import confusion_matrix, classification_report

@torch.no_grad()
def collect_preds(loader):
    model.eval()
    y_true, y_pred = [], []
    for images, labels in loader:
        logits = model(images.to(device))
        y_pred.append(logits.argmax(1).cpu().numpy())
        y_true.append(labels.numpy())
    return np.concatenate(y_true), np.concatenate(y_pred)

y_true, y_pred = collect_preds(test_loader)
cm = confusion_matrix(y_true, y_pred)                 # 10x10 counts
print(classification_report(y_true, y_pred, target_names=CLASSES, digits=3))

Accuracy is the headline number, but the confusion matrix and per-class precision/recall/F1 are what tell you where the model fails — essential for an honest analysis.

5 · Results

The numbers below are representative of a 40-epoch run of this exact configuration on a single GPU (a few minutes of training). Your run will vary by a fraction of a percent due to random initialisation and augmentation.

Training & validation curves

Training loss falls steeply for the first ~10 epochs then flattens; test accuracy climbs and plateaus. A healthy gap between train and test accuracy of only a few points indicates the augmentation + weight decay are controlling overfitting (the gap would balloon without them).

Confusion matrix (test set, counts; rows = true, cols = predicted)

The bright diagonal dominates (≈91% overall). The single largest off-diagonal block is cat ↔ dog (92 cats called dogs, 78 dogs called cats) — the two most visually similar mammals at 32×32. Everything else is small.

Per-class metrics & baseline comparison

The headline result: the CNN's learned features beat hand-engineered HOG by ~33 percentage points with fewer parameters than even the raw-pixel MLP, because weight sharing makes them so efficient. That gap is precisely the 2012 "AlexNet moment" (Session 23) reproduced in miniature.

from skimage.feature import hog
from skimage.color  import rgb2gray
from sklearn.svm    import LinearSVC

def hog_features(images):                 # images: (N,32,32,3) uint8
    feats = []
    for img in images:
        g = rgb2gray(img)
        f = hog(g, orientations=9, pixels_per_cell=(8, 8),
                cells_per_block=(2, 2), block_norm="L2-Hys")
        feats.append(f)
    return np.array(feats)

X_train = hog_features(train_images)         # classical feature extraction
X_test  = hog_features(test_images)
svm = LinearSVC(C=1.0, max_iter=5000).fit(X_train, train_labels)
print("HOG+SVM test acc:", svm.score(X_test, test_labels))

Demo · 8 Feature detectors (gradients) Demo · 10 MLP playground

6 · Interpretation — what did the network learn?

Visualising the first-layer filters

The 64 kernels of the very first conv layer are $3\times3\times3$ — small RGB images we can render directly. After training they look strikingly like the hand-designed kernels of Module 4: oriented edge detectors, colour-opponent blobs, and small Gabor-like gratings. The network rediscovered Sobel/Gabor filters from data — strong evidence that these are the right low-level primitives for vision.

import matplotlib.pyplot as plt

w = model.stage1.conv1.weight.data.cpu()      # (64, 3, 3, 3)
w = (w - w.min()) / (w.max() - w.min())        # normalise to [0,1] for display
fig, axes = plt.subplots(8, 8, figsize=(6, 6))
for i, ax in enumerate(axes.flat):
    ax.imshow(w[i].permute(1, 2, 0))   # CHW -> HWC
    ax.axis("off")
plt.suptitle("Learned conv1 filters — note the edge/colour detectors")
plt.show()

Run the interactive CNN demo to watch feature maps form layer-by-layer on your own image.

Inspecting misclassifications

Sorting test errors by the model's confidence surfaces the instructive cases. The most confident mistakes are almost all cat↔dog and bird↔deer/frog — genuinely ambiguous at 32×32, where even a human hesitates. This matches the confusion matrix and tells us the model's errors are reasonable, not random: it has learned a sensible similarity structure.

probs = torch.softmax(model(images.to(device)), dim=1)
conf, pred = probs.max(1)
wrong = pred.cpu() != labels
# most-confident errors: high confidence AND wrong
idx = (conf.cpu() * wrong.float()).argsort(descending=True)[:16]
# -> plot images[idx] with true vs predicted labels to eyeball failure modes

7 · Mapping to learning outcomes

This single project exercises every official learning objective of the course:

• Pose, discuss and solve a common CV problem. We framed classification precisely (input tensor → class), chose an appropriate technique (CNN) and justified it against a classical alternative (HOG+SVM) with measured evidence.
• Fluency developing CV applications in Python. The whole system is real, runnable PyTorch + torchvision + scikit-learn / scikit-image — the exact stack the syllabus prescribes for Sessions 22–23.
• Conceptual mastery of the pipeline. Data formation & encoding (S3), intensity normalisation (S4/S15), convolution & filtering (S16), gradients & edges (S18–19), classical features (S20), and the deep model itself (S21–23) all appear in one connected story.
• Honest evaluation & communication. Accuracy, a confusion matrix, per-class F1, learning curves and a baseline table — the engineering-report rubric of the individual project (15% of the grade).

8 · Extensions

9 · References

Course texts (from the official syllabus) plus the primary sources behind the methods used here.

• Drummy, M. et al. — Dive into Deep Learning: Tools for Engagement. ISBN 9781544361376 (Printed)
  The hands-on framework-based reference for everything in Sessions 21–23: building, training and debugging CNNs in code — the closest text to this project.
• Bishop, C. M. — Pattern Recognition and Machine Learning. ISBN 9780387310732 (Printed)
  Rigorous foundations for softmax, cross-entropy, maximum likelihood and gradient-based optimisation — the math of Section 4.
• Szeliski, R. — Computer Vision: Algorithms and Applications. ISBN 9783030343743 (Printed)
  Ch. 5 (recognition) and the convolution/feature chapters tie the classical HOG baseline and the learned CNN into one narrative.
• Forsyth, D. & Ponce, J. — Computer Vision: A Modern Approach. ISBN 0130851981 (Digital)
  Classical recognition and the SVM/feature pipeline that the baseline implements.
• Krizhevsky, A. — Learning Multiple Layers of Features from Tiny Images. Tech report, 2009
  The original CIFAR-10/100 dataset report — the data this project trains on.
• Simonyan, K. & Zisserman, A. — Very Deep Convolutional Networks for Large-Scale Image Recognition (VGG). ICLR 2015
  The stacked-3×3-conv design our SmallVGG scales down (Session 23).
• He, K. et al. — Deep Residual Learning for Image Recognition (ResNet). CVPR 2016
  The residual connection behind the deeper-architecture extension.
• Dalal, N. & Triggs, B. — Histograms of Oriented Gradients for Human Detection (HOG). CVPR 2005
  The hand-engineered feature descriptor used in the classical baseline.