Convolutional Neural Networks (CNNs)
The purpose of this notebook is to practice implementing and training CNNs. We start with a 1-dimensional convolutional layer in NumPy.
We will then use PyTorch, an optimised machine learning framework for Python based on the torch library to implement machine learning architectures.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import torch
# Importing losses, activation functions and layers from PyTorch
from torch.nn import Sequential, CrossEntropyLoss, Conv1d, MaxPool1d, Flatten, Linear, ReLU, Softmax, Parameter
from torch.utils.data import TensorDataset, DataLoader
# Changing default font sizes
plt.rc('xtick', labelsize=8) # Tick labels
plt.rc('ytick', labelsize=8) # Tick labels
plt.rc('legend', fontsize=10) # Legend
plt.rc('axes', titlesize=15, labelsize=10) # Title and 'x' and 'y' labels
You will be working with the Human Activity Recognition (HAR) Using Smartphones dataset. This consists of the readings from an accelerometer carried by a human doing different activities. The six activities are: walking horizontally, walking upstairs, walking downstairs, sitting, standing and laying down. The accelerometer is inside a smartphone, and it takes six readings every 0.02 seconds: linear and gyroscopic acceleration in the x, y and z directions.
The goal is to use the accelerometer data to predict the type of activity (a multi-class classification task).
1. Loading and plotting the data
x_train = np.load('./data/HAR/x_train.npy')
y_train = np.load('./data/HAR/y_train.npy')
x_val = np.load('./data/HAR/x_val.npy')
y_val = np.load('./data/HAR/y_val.npy')
x_test = np.load('./data/HAR/x_test.npy')
y_test = np.load('./data/HAR/y_test.npy')
The input data consists of 6 features over 128 time steps, and the output is a single integer from 0 to 5, which denotes the class.
There are 7352 examples in the training set, 2447 examples in the validation set and 500 in the test set. We can check the shapes ourselves.
print(x_train.shape)
print(y_train.shape)
print(x_val.shape)
print(y_val.shape)
print(x_test.shape)
print(y_test.shape)
(7352, 6, 128)
(7352, 1)
(2447, 6, 128)
(2447, 1)
(500, 6, 128)
(500, 1)
classes = [
'Walking',
'Walking upstairs',
'Walking downstairs',
'Sitting',
'Standing',
'Laying'
]
# Plot a randomly selected example from each class
for l, label in enumerate(range(len(classes))):
inx = np.where(y_train[:, 0] == label)[0]
i = np.random.choice(inx)
x_example = x_train[i].T
fig, ax = plt.subplots(figsize=(10, 1))
ax.imshow(x_example.T, cmap='Greys', vmin=-1, vmax=1)
ax.set_ylabel('Reading')
ax.set_xlabel('Time step')
ax.set_title(classes[l])
2. 1D convolutional layer in NumPy
Now we will implement a 1D convolutional layer in numpy. The following function is designed to perform a 1D convolution on an input signal, given weight and bias parameters. The layer should have no padding and a stride of 1. The output should consist of the pre-activations of the layer, meaning that no activation function is then applied.
Notation
c_inandc_out(equal ton_filters) represent the number of input and output channels (respectively).l_inandl_outdenote the length of the input and output signals (respectively).kis the length of the convolving kernel/filter.
For a convolutional (or max pool) layer, the input and output lengths are related as follows:
l o u t = ⌊ l i n + 2 × padding − kernel_size stride + 1 ⌋ l_{out}=\left\lfloor\frac{l_{in}+2×\text{padding}-\text{kernel\_size}}{\text{stride}}+1\right\rfloor lout=⌊stridelin+2×padding−kernel_size+1⌋
## EDIT THIS FUNCTION
def conv1d(x, weight, bias):
"""
Performs a 1D convolution over an input signal.
Parameters:
x: Input signal of shape (batch_size, c_in, l_in)
weight: Learnable weights, shape (c_out, c_in, k)
bias: Bias parameters of size c_out
Returns:
An array of shape (batch_size, c_out, l_out)
"""
batch_size = x.shape[0] ## <-- SOLUTION
l_in = x.shape[2] ## <-- SOLUTION
c_out = weight.shape[0] ## <-- SOLUTION
k = weight.shape[2] ## <-- SOLUTION
l_out = l_in - k + 1 ## <-- SOLUTION
outputs = np.zeros((batch_size, c_out, l_out))
np.testing.assert_allclose(l_out, 113)
for i in range(l_out):
outputs[:, :, i] = (x[:, np.newaxis, :, i:i+k] * weight).sum(axis=(2, 3)) + bias ## <-- SOLUTION
return outputs
Now we can compare our layer with the PyTorch implementation.
# Test your layer
n_filters = 8
batch_size = 16
k = 16
l_in = 128
# PyTorch
conv_layer = Conv1d(x_train.shape[1], n_filters, k)
inputs = torch.randn((batch_size, x_train.shape[1], l_in))
y_torch = conv_layer(inputs)
# Our layer
y = conv1d(inputs.numpy(), conv_layer.weight.detach().numpy(), conv_layer.bias.detach().numpy())
np.allclose(y, y_torch.detach().numpy(), atol=1e-4) ## <-- should be 'True'
True
3. Building a CNN in PyTorch
You should now build the CNN model in PyTorch to train on the HAR dataset. This model should consist of:
- A
Conv1dlayer with 8 filters, kernel size of 16 and aReLUactivation function- The input shape should be
(6, 128)
- The input shape should be
- A
MaxPoo1dlayer with a pooling window size of 16 and a stride of 2 - A
Flattenlayer - A
Linearlayer with 6 neurons and aSoftmaxactivation
The function below should build and compile this model, using an Adam optimiser and a CrossEntropyLoss criterion.
def get_model(x_train, n_filters, k, pool_size, stride_pool, classes, l2_reg=1e-3):
"""
CNN model in PyTorch:
- Layers are Conv1d(+ReLU), MaxPool1d, Flatten and Linear(+Softmax).
- It features an Adam optimiser and CrossEntropyLoss criterion.
- Conv1d and Linear layers have regularised weights according to l2_reg.
Parameters:
x_train: Training data
n_filters: Number of filters to be used in the convolutional layer
k: Kernel size in the convolutional layer
pool_size: MaxPool1d window size
stride_pool: Stride of the MaxPool1d sliding window
classes: List containing the output classes
l2_reg: Positive float corresponding to the regularisation coefficient of Conv1d and Linear
Returns:
Model, criterion and optimiser.
"""
l_out_conv = x_train.shape[2] - k + 1 # Length after Conv1d (note that the stride is 1) ## <-- SOLUTION
l_out_pool = (l_out_conv - pool_size) // stride_pool + 1 # Length after MaxPool1d ## <-- SOLUTION
l_in_linear = n_filters * l_out_pool # Size before Linear layer ## <-- SOLUTION
np.testing.assert_allclose(l_in_linear, 392)
model = Sequential(
Conv1d(x_train.shape[1], n_filters, kernel_size=k),
ReLU(), ## <-- SOLUTION
MaxPool1d(kernel_size=pool_size, stride=stride_pool), ## <-- SOLUTION
Flatten(), ## <-- SOLUTION
Linear(l_in_linear, len(classes)), ## <-- SOLUTION
Softmax(dim=1), ## <-- SOLUTION
)
# L2 regularisation
for layer in model.children():
if isinstance(layer, Conv1d) or isinstance(layer, Linear):
layer.weight_regularizer = Parameter(l2_reg * torch.ones_like(layer.weight))
criterion = CrossEntropyLoss() ## <-- SOLUTION
optimiser = torch.optim.Adam(model.parameters()) ## <-- SOLUTION
return model, criterion, optimiser
# Run your function to get the model and print it
n_filters = 8
k = 16
pool_size = 16
stride_pool = 2
l2_reg = 1e-3
model, criterion, optimiser = get_model(x_train, n_filters, k, pool_size, stride_pool, classes, l2_reg) ## <-- SOLUTION
print(model)
Sequential(
(0): Conv1d(6, 8, kernel_size=(16,), stride=(1,))
(1): ReLU()
(2): MaxPool1d(kernel_size=16, stride=2, padding=0, dilation=1, ceil_mode=False)
(3): Flatten(start_dim=1, end_dim=-1)
(4): Linear(in_features=392, out_features=6, bias=True)
(5): Softmax(dim=1)
)
4. Training a CNN
Now we are ready to train the model. The training_loop function should run the training for max_num_epochs of 200 and batch_size of 128, featuring early stopping to monitor the validation accuracy. Set the max_patience parameter to 5 epochs. The function should return the training and validation history, which includes the loss and accuracy.
# Numpy arrays to PyTorch tensors
x_train_tensor = torch.tensor(x_train, dtype=torch.float32)
y_train_tensor = torch.tensor(y_train)
x_val_tensor = torch.tensor(x_val, dtype=torch.float32)
y_val_tensor = torch.tensor(y_val)
# Creating training and validation datasets
train_dataset = TensorDataset(x_train_tensor, y_train_tensor.squeeze())
val_dataset = TensorDataset(x_val_tensor, y_val_tensor.squeeze())
# Creating corresponding DataLoaders
train_loader = DataLoader(train_dataset, batch_size=128, shuffle=True)
val_loader = DataLoader(val_dataset, batch_size=128, shuffle=False)
class EarlyStopping:
"""
EarlyStopping class.
Attributes:
max_patience: Amount of epochs with no improvement after which training is stopped.
patience: Stores the number of epochs with no improvement.
best_valid_loss: Stores the current value of the best (minimum) validation loss.
early_stop: True if training needs to be stopped due to the early stopping condition being met.
Methods:
step(val_loss):
Checks current state after an epoch and updates best_loss, patience and early_stop accordingly.
"""
def __init__(self, max_patience=5):
self.max_patience = max_patience
self.patience = 0
self.best_valid_loss = float('inf') ## <-- SOLUTION
self.early_stop = False
def step(self, val_loss):
if val_loss < self.best_valid_loss: ## <-- SOLUTION
self.best_valid_loss = val_loss ## <-- SOLUTION
self.patience = 0 ## <-- SOLUTION
else:
self.patience += 1
if self.patience >= self.max_patience:
self.early_stop = True
def training_loop(train_loader, val_loader, max_num_epochs=200, max_patience=5):
"""
Training loop with early stopping to monitor the validation accuracy.
Parameters:
train_loader: Training DataLoader
val_loader: Validation DataLoader
max_num_epochs: Maximum number of epochs
max_patience: max_patience attribute of the EarlyStopping class
Returns:
Model, criterion and optimiser.
"""
history = {'training_loss': [], 'validation_loss': [], 'training_accuracy': [], 'validation_accuracy': []}
early_stopping = EarlyStopping(max_patience=5)
for epoch in range(max_num_epochs):
tr_loss = 0.0
tr_accuracy = 0
val_loss = 0.0
val_accuracy = 0
# Training
model.train()
for inputs, labels in train_loader:
optimiser.zero_grad() # Setting gradients to zero
outputs = model(inputs)
loss = criterion(outputs, labels) ## <-- SOLUTION
tr_loss += loss.item()
tr_accuracy += (torch.max(outputs, 1)[1] == labels).sum().item() / (len(train_loader) * labels.size(0)) ## <-- SOLUTION
loss.backward() # Computes gradients of the loss
optimiser.step() # Optimisation step (parameters are updated)
history['training_loss'].append(tr_loss/len(train_loader))
history['training_accuracy'].append(100*tr_accuracy)
# Validation
model.eval()
with torch.no_grad():
for inputs, labels in val_loader:
outputs = model(inputs)
loss = criterion(outputs, labels) ## <-- SOLUTION
val_loss += loss.item()
val_accuracy += (torch.max(outputs, 1)[1] == labels).sum().item() / (len(val_loader) * labels.size(0)) ## <-- SOLUTION
history['validation_loss'].append(val_loss/len(val_loader))
history['validation_accuracy'].append(100*val_accuracy)
# Calculate accuracy and print
print(f"Epoch {epoch + 1}/{max_num_epochs}, Training loss: {tr_loss/len(train_loader)}, Training accuracy: {100*tr_accuracy}%, Validation loss: {val_loss/len(val_loader)}, Validation accuracy: {100*val_accuracy}%")
# Check for early stopping
early_stopping.step(val_loss / len(val_loader))
if early_stopping.early_stop:
print("Early stopping acting.") ## <-- SOLUTION
break ## <-- SOLUTION
return history
# Calling the training loop
max_num_epochs = 200
max_patience = 5
history = training_loop(train_loader, val_loader, max_num_epochs, max_patience)
Epoch 1/200, Training loss: 1.7121318414293487, Training accuracy: 30.34752155172414%, Validation loss: 1.6579939246177673, Validation accuracy: 45.6640625%
Epoch 2/200, Training loss: 1.5362633055654065, Training accuracy: 56.62715517241378%, Validation loss: 1.5142081260681153, Validation accuracy: 55.39062499999999%
Epoch 3/200, Training loss: 1.457470114888816, Training accuracy: 61.172259852216705%, Validation loss: 1.4736222505569458, Validation accuracy: 59.35677083333333%
Epoch 4/200, Training loss: 1.417420794223917, Training accuracy: 63.833512931034484%, Validation loss: 1.4364268779754639, Validation accuracy: 60.68489583333334%
Epoch 5/200, Training loss: 1.388573251921555, Training accuracy: 65.13046490147785%, Validation loss: 1.4158733427524566, Validation accuracy: 62.22916666666668%
Epoch 6/200, Training loss: 1.3700056261029736, Training accuracy: 67.11437807881775%, Validation loss: 1.3984680473804474, Validation accuracy: 62.97135416666667%
Epoch 7/200, Training loss: 1.3590136170387268, Training accuracy: 66.95658866995073%, Validation loss: 1.3878288209438323, Validation accuracy: 63.127604166666686%
Epoch 8/200, Training loss: 1.3516001372501767, Training accuracy: 68.26123768472905%, Validation loss: 1.37613445520401, Validation accuracy: 64.10416666666667%
Epoch 9/200, Training loss: 1.344404882398145, Training accuracy: 69.12523091133005%, Validation loss: 1.3671935498714447, Validation accuracy: 67.97135416666667%
Epoch 10/200, Training loss: 1.3371149774255424, Training accuracy: 71.12068965517244%, Validation loss: 1.3587223291397095, Validation accuracy: 69.10416666666667%
Epoch 11/200, Training loss: 1.3309343231135402, Training accuracy: 71.72105911330051%, Validation loss: 1.3578250110149384, Validation accuracy: 70.15885416666667%
Epoch 12/200, Training loss: 1.3236742245739903, Training accuracy: 73.12384544334977%, Validation loss: 1.3490940630435944, Validation accuracy: 70.47135416666667%
Epoch 13/200, Training loss: 1.316469132900238, Training accuracy: 73.51831896551727%, Validation loss: 1.3431268513202668, Validation accuracy: 71.40885416666667%
Epoch 14/200, Training loss: 1.3090957773142848, Training accuracy: 74.33805418719213%, Validation loss: 1.3374877393245697, Validation accuracy: 75.83072916666666%
Epoch 15/200, Training loss: 1.3022815276836526, Training accuracy: 76.62022783251233%, Validation loss: 1.3333993017673493, Validation accuracy: 76.3984375%
Epoch 16/200, Training loss: 1.2962056973884846, Training accuracy: 77.96143780788175%, Validation loss: 1.3244478344917296, Validation accuracy: 77.6484375%
Epoch 17/200, Training loss: 1.2900241490068107, Training accuracy: 78.54256465517241%, Validation loss: 1.3174363672733307, Validation accuracy: 78.41145833333334%
Epoch 18/200, Training loss: 1.2849656836739902, Training accuracy: 78.79656711822659%, Validation loss: 1.3200359404087068, Validation accuracy: 77.41666666666667%
Epoch 19/200, Training loss: 1.2789889286304343, Training accuracy: 79.89532019704434%, Validation loss: 1.3159491896629334, Validation accuracy: 79.01822916666667%
Epoch 20/200, Training loss: 1.2746241031021908, Training accuracy: 80.14739839901476%, Validation loss: 1.3115354359149933, Validation accuracy: 79.05729166666667%
Epoch 21/200, Training loss: 1.268436600422037, Training accuracy: 81.13262007389159%, Validation loss: 1.3029167354106903, Validation accuracy: 78.97916666666667%
Epoch 22/200, Training loss: 1.263866765745755, Training accuracy: 81.50400246305415%, Validation loss: 1.3027888059616088, Validation accuracy: 79.29166666666666%
Epoch 23/200, Training loss: 1.2596432443322807, Training accuracy: 82.23907019704434%, Validation loss: 1.2978974342346192, Validation accuracy: 79.36979166666666%
Epoch 24/200, Training loss: 1.2545871631852512, Training accuracy: 82.97221366995075%, Validation loss: 1.292573618888855, Validation accuracy: 79.99479166666666%
Epoch 25/200, Training loss: 1.2506435061323231, Training accuracy: 82.63354371921179%, Validation loss: 1.289669579267502, Validation accuracy: 80.54166666666664%
Epoch 26/200, Training loss: 1.2458197618352955, Training accuracy: 83.38977832512319%, Validation loss: 1.2845512449741363, Validation accuracy: 80.79687499999999%
Epoch 27/200, Training loss: 1.241009025738157, Training accuracy: 84.12869458128074%, Validation loss: 1.2831344127655029, Validation accuracy: 81.08854166666667%
Epoch 28/200, Training loss: 1.2372907441237877, Training accuracy: 84.38846982758619%, Validation loss: 1.2743450820446014, Validation accuracy: 82.41666666666667%
Epoch 29/200, Training loss: 1.2339484938259782, Training accuracy: 84.5597290640394%, Validation loss: 1.2788837432861329, Validation accuracy: 80.2890625%
Epoch 30/200, Training loss: 1.230920409334117, Training accuracy: 85.13700738916256%, Validation loss: 1.2676789581775665, Validation accuracy: 81.9296875%
Epoch 31/200, Training loss: 1.2266510951107945, Training accuracy: 85.69311884236454%, Validation loss: 1.2738824486732483, Validation accuracy: 81.96614583333334%
Epoch 32/200, Training loss: 1.2224209123644336, Training accuracy: 85.86437807881775%, Validation loss: 1.263941866159439, Validation accuracy: 82.39583333333333%
Epoch 33/200, Training loss: 1.2200829592244378, Training accuracy: 86.36853448275862%, Validation loss: 1.2582094848155976, Validation accuracy: 83.68750000000001%
Epoch 34/200, Training loss: 1.2159077052412361, Training accuracy: 86.52439963054186%, Validation loss: 1.256572663784027, Validation accuracy: 82.78645833333333%
Epoch 35/200, Training loss: 1.21336764302747, Training accuracy: 86.72452278325122%, Validation loss: 1.2526042759418488, Validation accuracy: 83.45052083333331%
Epoch 36/200, Training loss: 1.2097959169026078, Training accuracy: 86.951585591133%, Validation loss: 1.2550598978996277, Validation accuracy: 83.68750000000001%
Epoch 37/200, Training loss: 1.208302919206948, Training accuracy: 87.31142241379312%, Validation loss: 1.2496072232723237, Validation accuracy: 83.84375%
Epoch 38/200, Training loss: 1.204239874050535, Training accuracy: 87.94450431034484%, Validation loss: 1.2462993741035462, Validation accuracy: 83.70572916666667%
Epoch 39/200, Training loss: 1.2009871992571601, Training accuracy: 88.48522167487684%, Validation loss: 1.2450489103794098, Validation accuracy: 84.17447916666666%
Epoch 40/200, Training loss: 1.1983779607148006, Training accuracy: 88.55834359605912%, Validation loss: 1.246843844652176, Validation accuracy: 83.39322916666664%
Epoch 41/200, Training loss: 1.1961354099470993, Training accuracy: 88.71998152709357%, Validation loss: 1.2500394701957702, Validation accuracy: 82.86458333333331%
Epoch 42/200, Training loss: 1.194460455713601, Training accuracy: 88.68149630541872%, Validation loss: 1.2375515460968018, Validation accuracy: 84.29166666666667%
Epoch 43/200, Training loss: 1.1921263933181763, Training accuracy: 89.21836514778329%, Validation loss: 1.2429938435554504, Validation accuracy: 83.90104166666667%
Epoch 44/200, Training loss: 1.190616498733389, Training accuracy: 89.18757697044337%, Validation loss: 1.235194605588913, Validation accuracy: 84.78125000000001%
Epoch 45/200, Training loss: 1.187075908841758, Training accuracy: 89.370381773399%, Validation loss: 1.240226572751999, Validation accuracy: 83.78385416666664%
Epoch 46/200, Training loss: 1.1855433028319786, Training accuracy: 89.67633928571429%, Validation loss: 1.2349590957164764, Validation accuracy: 83.90104166666666%
Epoch 47/200, Training loss: 1.184152744967362, Training accuracy: 89.7205972906404%, Validation loss: 1.2351206958293914, Validation accuracy: 84.4296875%
Epoch 48/200, Training loss: 1.1808069451101895, Training accuracy: 89.8706896551724%, Validation loss: 1.2322359025478362, Validation accuracy: 84.390625%
Epoch 49/200, Training loss: 1.179458708598696, Training accuracy: 90.18819273399015%, Validation loss: 1.2292146563529969, Validation accuracy: 85.23177083333334%
Epoch 50/200, Training loss: 1.1773293121107693, Training accuracy: 90.21705665024628%, Validation loss: 1.2295194864273071, Validation accuracy: 85.09635416666667%
Epoch 51/200, Training loss: 1.1754406012337784, Training accuracy: 90.50184729064037%, Validation loss: 1.228010755777359, Validation accuracy: 84.41145833333333%
Epoch 52/200, Training loss: 1.1741080119692047, Training accuracy: 90.3960129310345%, Validation loss: 1.2280026853084565, Validation accuracy: 84.72135416666666%
Epoch 53/200, Training loss: 1.1731348202146332, Training accuracy: 90.83474445812806%, Validation loss: 1.2285800397396087, Validation accuracy: 84.44791666666667%
Epoch 54/200, Training loss: 1.1703439169916614, Training accuracy: 91.01754926108373%, Validation loss: 1.2257872700691224, Validation accuracy: 84.44791666666667%
Epoch 55/200, Training loss: 1.1694629829505394, Training accuracy: 91.41394704433499%, Validation loss: 1.2266152143478393, Validation accuracy: 84.48697916666666%
Epoch 56/200, Training loss: 1.169368433541265, Training accuracy: 90.72891009852216%, Validation loss: 1.2266651093959808, Validation accuracy: 84.50781250000001%
Epoch 57/200, Training loss: 1.1658331751823425, Training accuracy: 91.70451046798027%, Validation loss: 1.2231320858001709, Validation accuracy: 84.625%
Epoch 58/200, Training loss: 1.1654142490748702, Training accuracy: 91.53132697044336%, Validation loss: 1.226263427734375, Validation accuracy: 84.29427083333336%
Epoch 59/200, Training loss: 1.1627582743250091, Training accuracy: 91.61791871921184%, Validation loss: 1.2233623445034028, Validation accuracy: 85.03645833333331%
Epoch 60/200, Training loss: 1.1622245147310455, Training accuracy: 91.70643472906403%, Validation loss: 1.2226067543029786, Validation accuracy: 85.23177083333333%
Epoch 61/200, Training loss: 1.1605095226189186, Training accuracy: 92.01431650246303%, Validation loss: 1.2181911230087281, Validation accuracy: 84.8984375%
Epoch 62/200, Training loss: 1.1589927940533078, Training accuracy: 91.86229987684726%, Validation loss: 1.2235017716884613, Validation accuracy: 84.31250000000001%
Epoch 63/200, Training loss: 1.158447723964165, Training accuracy: 91.73529864532017%, Validation loss: 1.2224792778491973, Validation accuracy: 84.84114583333334%
Epoch 64/200, Training loss: 1.1564319421505105, Training accuracy: 91.92387623152709%, Validation loss: 1.2213149666786194, Validation accuracy: 84.1953125%
Epoch 65/200, Training loss: 1.156325584855573, Training accuracy: 91.9508158866995%, Validation loss: 1.2250536918640136, Validation accuracy: 84.4296875%
Epoch 66/200, Training loss: 1.154990816938466, Training accuracy: 92.17210591133004%, Validation loss: 1.218558484315872, Validation accuracy: 85.30989583333331%
Early stopping acting.
# Plot the learning curves
fig = plt.figure(figsize=(12, 5))
fig.add_subplot(121)
plt.plot(history['training_loss'], label='Train')
plt.plot(history['validation_loss'], label='Validation')
plt.xlabel("Epoch", size=12)
plt.ylabel("Cross-entropy loss", size=12)
plt.title("Learning curve")
plt.legend()
fig.add_subplot(122)
plt.plot(history['training_accuracy'], label='Train')
plt.plot(history['validation_accuracy'], label='Validation')
plt.xlabel("Epoch", size=12)
plt.ylabel("Categorical accuracy", size=12)
plt.title("Accuracy as a function of the epoch")
plt.legend()
plt.show()
5. Evaluating the test performance of a CNN
Now let’s take a look at some model predictions!
# Get the model predictions
preds = model(torch.tensor(x_test, dtype=torch.float32)).detach().numpy() ## <-- SOLUTION
# Plot some example predictions
num_preds = preds.shape[0]
num_examples = 10
inx = np.random.choice(num_preds, num_examples, replace=False)
gs = {"width_ratios": [2, 1]}
for i in inx:
x_example = x_test[i]
true_label = y_test[i][0]
prediction = preds[i]
pred_class = np.argmax(prediction)
fig, ax = plt.subplots(figsize=(14, 1), ncols=2, gridspec_kw=gs)
ax[0].imshow(x_example, cmap='Greys', vmin=-1, vmax=1)
ax[0].set_ylabel('Reading')
ax[0].set_xlabel('Time step')
ax[0].set_title("True label: {}\nPredicted label: {}".format(classes[true_label], classes[pred_class]))
ax[1].bar(['Walk', 'Walk\nUp', 'Walk\nDown', 'Sit', 'Stand', 'Lay'], prediction)
ax[1].set_title("Class predictions")
Questions
- Are there particular classes where the model tends to be more uncertain in its predictions?
- What effect does the weight regularisation have on the training and the final model? Try experimenting with no weight regularisation, and different values of the regularisation coefficient.
- How did the
max_patiencehyperparameter impact the training run above? What would have happened if we setmax_patienceto zero?