VGG Network is the basis for one of the most popular image recognition techniques. It is worth learning because it opens a lot of avenues. You need to understand how a Convolutional Neural Network (CNN) to understand VGGNet. If you are not familiar with CNN architecture please feel free to go through this tutorial first.
About VGG Network
VGGNet is a kind of Convolutional Neural Network (CNN) that can extract features more successfully. In VGGNet, we stack multiple Convolution layers. VGGNets can be shallow or deep. In shallow VGGNet, usually, only two sets of four convolution layers are added as we will see soon. And in deep VGGNet, more than four Convolution layers can be added. Two commonly used deep VGGNet is VGG16 which uses 16 layers a total and VGG19 which uses a total of 19 layers. We can add a batch normalization layer or avoid it. But I will use it in this tutorial.
You can read more about the architecture more in this link.
We are going to work on a mini VGGNet today. So it will be much simpler and easier to run but still powerful for a lot of use cases.
One important characteristic of miniVGGNet is, it uses all 3×3 filters. That’s the reason it can generalize so well. Let’s just get started and build a mini VGGNet in Keras and TensorFlow.
I used Google Colaboratory notebook and enabled GPU for this. Otherwise, the training is very slow.
Mini VGG Network Development, Training, and Evaluation
Time to start working. We will experiment with it a little to demonstrate how we can play with it.
These are the necessary imports:
import tensorflow as tf
from keras.models import Sequential
from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers.core import Activation
from keras.layers.core import Flatten
from keras.layers.core import Dropout
from keras.layers.core import Dense
from keras import backend as K
from sklearn.preprocessing import LabelBinarizer
from sklearn.metrics import classification_report
from keras.optimizers import SGD
from keras.datasets import cifar10
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
That’s a lot of imports!
We will use the cifar-10 dataset from TensorFlow which is a public dataset available in the TensorFlow library.
I used two different networks just as an experiment. The first one is the popular one. I am saying popular because I found this architecture in Kaggle and some other tutorials.
class MiniVGGNet:
@staticmethod
def build(width, height, depth, classes):
# initialize the model along with the input shape to be
# "channels last" and the channels dimension itself
model = Sequential()
inputShape = (height, width, depth)
chanDim = -1
if K.image_data_format() == "channels_first":
inputShape = (depth, height, width)
chanDim = 1
# first CONV => Activation => CONV => Activation => POOL layer set
model.add(Conv2D(32, (3, 3), padding="same",
input_shape=inputShape))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(Conv2D(32, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
# second CONV => Activation => CONV => Activation => POOL layer set
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
# Dense Layer
model.add(Flatten())
model.add(Dense(512))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Dropout(0.5))
# softmax classifier
model.add(Dense(classes))
model.add(Activation("softmax"))
# return the constructed network architecture
return model
Let’s load and prepare our cifar-10 dataset.
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
x_train = x_train.astype("float") / 255.0
x_test = x_test.astype("float") / 255.0
The cifar-10 dataset has 10 labels. These are the labels in the cifar-10 dataset:
labelNames = ["airplane", "automobile", "bird", "cat", "deer",
"dog", "frog", "horse", "ship", "truck"]
Using the LabelBinarizer to binarize the labels:
lb = LabelBinarizer()
y_train = lb.fit_transform(y_train)
y_test = lb.transform(y_test)
Compiling the model here. The evaluation metric is “accuracy” and we will run for 10 epochs.
optimizer = tf.keras.optimizers.legacy.SGD(learning_rate=0.01, decay=0.01/40, momentum=0.9,
nesterov=True)
model = miniVGGNet.build(width = 32, height = 32, depth = 3, classes=10)
model.compile(loss='categorical_crossentropy', optimizer = optimizer,
metrics=['accuracy'])
h = model.fit(x_train, y_train, validation_data=(x_test, y_test),
batch_size = 64, epochs=10, verbose=1)
Here is the result:
Epoch 1/10
782/782 [==============================] - 424s 539ms/step - loss: 1.6196 - accuracy: 0.4592 - val_loss: 1.4083 - val_accuracy: 0.5159
Epoch 2/10
782/782 [==============================] - 430s 550ms/step - loss: 1.1437 - accuracy: 0.6039 - val_loss: 1.0213 - val_accuracy: 0.6505
Epoch 3/10
782/782 [==============================] - 430s 550ms/step - loss: 0.9634 - accuracy: 0.6618 - val_loss: 0.8495 - val_accuracy: 0.7013
Epoch 4/10
782/782 [==============================] - 427s 546ms/step - loss: 0.8532 - accuracy: 0.6998 - val_loss: 0.7881 - val_accuracy: 0.7215
Epoch 5/10
782/782 [==============================] - 425s 543ms/step - loss: 0.7773 - accuracy: 0.7280 - val_loss: 0.8064 - val_accuracy: 0.7228
Epoch 6/10
782/782 [==============================] - 421s 538ms/step - loss: 0.7240 - accuracy: 0.7451 - val_loss: 0.6757 - val_accuracy: 0.7619
Epoch 7/10
782/782 [==============================] - 420s 537ms/step - loss: 0.6843 - accuracy: 0.7579 - val_loss: 0.6564 - val_accuracy: 0.7715
Epoch 8/10
782/782 [==============================] - 420s 537ms/step - loss: 0.6405 - accuracy: 0.7743 - val_loss: 0.6833 - val_accuracy: 0.7706
Epoch 9/10
782/782 [==============================] - 422s 540ms/step - loss: 0.6114 - accuracy: 0.7828 - val_loss: 0.6188 - val_accuracy: 0.7848
Epoch 10/10
782/782 [==============================] - 421s 538ms/step - loss: 0.5799 - accuracy: 0.7946 - val_loss: 0.6166 - val_accuracy: 0.7898
After 10 epochs accuracy becomes 79.46% on training data and 78.98% on validation data.
Keeping this in mind, I wanted to change a few things in this network and see the results. Let’s redefine the network above. I used 64 filters all throughout, 256 neurons in the dense layer, and 40% dropout in the last dropout layer.
Here is the new mini VGG network again:
class miniVGGNet:
@staticmethod
def build(width, height, depth, classes):
model = Sequential()
inputShape = (height, width, depth)
chanDim = -1
if K.image_data_format() == "channels_first":
inputShape = (depth, height, width)
chanDim = 1
# first Conv => Activation => Conv => Activation => Pool layer set
model.add(Conv2D(64, (3, 3), padding="same",
input_shape=inputShape))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
# second Conv => Activation => Conv => Activation => Pool layer set
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(Conv2D(64, (3, 3), padding="same"))
model.add(Activation("relu"))
model.add(BatchNormalization(axis=chanDim))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
# Dense Layer
model.add(Flatten())
model.add(Dense(300))
model.add(Activation("relu"))
model.add(BatchNormalization())
model.add(Dropout(0.4))
model.add(Dense(classes))
model.add(Activation("softmax"))
return model
We will use the same parameters for optimization and running the model. But I used 20 epochs here.
optimizer = tf.keras.optimizers.legacy.SGD(learning_rate=0.01, decay=0.01/40, momentum=0.9,
nesterov=True)
model = miniVGGNet.build(width = 32, height = 32, depth = 3, classes=10)
model.compile(loss='categorical_crossentropy', optimizer = optimizer,
metrics=['accuracy'])
h = model.fit(x_train, y_train, validation_data=(x_test, y_test),
batch_size = 64, epochs=20, verbose=1)
Here are the results:
Epoch 1/20
782/782 [==============================] - 22s 18ms/step - loss: 1.5210 - accuracy: 0.4697 - val_loss: 1.1626 - val_accuracy: 0.5854
Epoch 2/20
782/782 [==============================] - 14s 18ms/step - loss: 1.0706 - accuracy: 0.6219 - val_loss: 0.9913 - val_accuracy: 0.6586
Epoch 3/20
782/782 [==============================] - 14s 18ms/step - loss: 0.8947 - accuracy: 0.6826 - val_loss: 0.8697 - val_accuracy: 0.6941
Epoch 4/20
782/782 [==============================] - 14s 18ms/step - loss: 0.7926 - accuracy: 0.7208 - val_loss: 0.7649 - val_accuracy: 0.7294
Epoch 5/20
782/782 [==============================] - 14s 18ms/step - loss: 0.7192 - accuracy: 0.7470 - val_loss: 0.6937 - val_accuracy: 0.7593
Epoch 6/20
782/782 [==============================] - 13s 17ms/step - loss: 0.6641 - accuracy: 0.7640 - val_loss: 0.6899 - val_accuracy: 0.7639
Epoch 7/20
782/782 [==============================] - 13s 17ms/step - loss: 0.6141 - accuracy: 0.7805 - val_loss: 0.6589 - val_accuracy: 0.7742
Epoch 8/20
782/782 [==============================] - 13s 17ms/step - loss: 0.5774 - accuracy: 0.7960 - val_loss: 0.6565 - val_accuracy: 0.7734
Epoch 9/20
782/782 [==============================] - 14s 17ms/step - loss: 0.5430 - accuracy: 0.8077 - val_loss: 0.6092 - val_accuracy: 0.7921
Epoch 10/20
782/782 [==============================] - 14s 18ms/step - loss: 0.5145 - accuracy: 0.8177 - val_loss: 0.5904 - val_accuracy: 0.7944
Epoch 11/20
782/782 [==============================] - 13s 17ms/step - loss: 0.4922 - accuracy: 0.8256 - val_loss: 0.6041 - val_accuracy: 0.7975
Epoch 12/20
782/782 [==============================] - 14s 18ms/step - loss: 0.4614 - accuracy: 0.8381 - val_loss: 0.5889 - val_accuracy: 0.7981
Epoch 13/20
782/782 [==============================] - 14s 18ms/step - loss: 0.4358 - accuracy: 0.8457 - val_loss: 0.5590 - val_accuracy: 0.8120
Epoch 14/20
782/782 [==============================] - 13s 17ms/step - loss: 0.4186 - accuracy: 0.8508 - val_loss: 0.5555 - val_accuracy: 0.8092
Epoch 15/20
782/782 [==============================] - 13s 17ms/step - loss: 0.4019 - accuracy: 0.8582 - val_loss: 0.5739 - val_accuracy: 0.8108
Epoch 16/20
782/782 [==============================] - 14s 17ms/step - loss: 0.3804 - accuracy: 0.8658 - val_loss: 0.5577 - val_accuracy: 0.8136
Epoch 17/20
782/782 [==============================] - 13s 17ms/step - loss: 0.3687 - accuracy: 0.8672 - val_loss: 0.5544 - val_accuracy: 0.8170
Epoch 18/20
782/782 [==============================] - 13s 17ms/step - loss: 0.3541 - accuracy: 0.8744 - val_loss: 0.5435 - val_accuracy: 0.8199
Epoch 19/20
782/782 [==============================] - 13s 17ms/step - loss: 0.3438 - accuracy: 0.8758 - val_loss: 0.5533 - val_accuracy: 0.8167
Epoch 20/20
782/782 [==============================] - 13s 17ms/step - loss: 0.3292 - accuracy: 0.8845 - val_loss: 0.5491 - val_accuracy: 0.8199
If you notice, after 10 epochs, the accuracy was slightly higher than the previous network and after 20 epochs the accuracy is really good. 88.45% on training data and 81.99% on validation data.
Presenting the training and validation accuracies and training and validation losses in the same plot:
%matplotlib inline
plt.close('all')
plt.style.use("ggplot")
plt.figure(figsize=(8, 6))
plt.plot(np.arange(0, 20), h.history["loss"], label="train_loss")
plt.plot(np.arange(0, 20), h.history["val_loss"], label="val_loss")
plt.plot(np.arange(0, 20), h.history["accuracy"], label="train_acc")
plt.plot(np.arange(0, 20), h.history["val_accuracy"], label="val_acc")
plt.title("Training Loss and Accuracy")
plt.xlabel("No of Epochs")
plt.ylabel("Loss/Accuracy")
plt.legend()
plt.show()
Training loss went down very smoothly, and validation loss went down as well with some bumps.
Conclusion
Please feel free to experiment with it. Try different parameters as per the project and see how it works for you. We will work on a deep network later.
