Facial Expression Recognition using Keras

In this tutorial, we will learn about Facial Expression Recognition using Keras.

What is Facial Recognition?

It is a technology that allows computer systems to identify or recognize someone by analyzing, comparing, and matching their facial features.
It analyses the features like…
1] Distance between the eyes.
2] Shape and Size of nose, Lips, ears, and skin.
3] color and shade of nose, Lips, ears, and skin.

What is Facial Expression Recognition?

It refers to a machine or system identifying the emotional state or the intentions of a person by analyzing, comparing, and matching the facial expression of that person. There are 6 common expressions that it recognizes
1] Happiness
2] Anger
3] Disgust
4] Surprise
5] Fear
6] Sadness

Stages of Project

This project will be conducted in 5 stages.
1] Importing Libraries
2] Plot Sample Images
3] Training and Testing Split
4] CNN Model
5] Evaluate

Code: Importing Libraries

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import os

from tensorflow.keras.preprocessing.image import ImageDataGenerator
from TensorFlow. keras.layers import Dense, Input, Dropout, Flatten, Conv2D
from tensorflow.keras.layers import BatchNormalization, Activation, MaxPooling2D
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau

1. Importing Libraries: We have imported the libraries, we require for the project.
   a] NumPy: It is a collection of mathematical tools and functions we require. It even assist in the data visualization.
   b] Matplotlib: Its a library used in plotting function with dynamic and animated visualization techniques.
   c] Seaborn: It is like a successor to Matplotlib. It offers more high-level, attractive, dynamic data visualization options.
2. Importing Sub-Libraries : From these libraries we will extract the useful function that we required like…
   a] Dense, Input, Flatten, Conv2D: These are the function used in CNN layer to analyze the image or video that is given as input.
   b]BatchNomalization, Activation, MaxPooling2D: These functions help improve the training of the model. For e.g. Activation function involves non-linearity. BatchNormalization reduces the dependencies of the model and MaxPooling reduces the spatial dimension of the layers

Code:  Plot Sample Images

import matplotlib.pyplot as plt; 
plt.imshow(Image.open('path_to_image.png')); 
plt.axis('off'); plt.show()


for expression in os.listdir("train/"):
print(str(len(os.listdir("train/" + expression))) + " " + expression + " images")

1. In this code snippet, we have imported the dataset from our system. Download link given below. Then we have plotted few images from it to understand data’s quality and variety.
2. The second part indicates the testing and training split. It’s mentioned on Kaggle that 80% is train and 20% is test. This will display the number of image in each section for training.

Download Link: “https://www.kaggle.com/datasets/davilsena/ckdataset/data”

Code:  Generating Training Batches

img_size = 48
batch_size = 64

datagen_train = ImageDataGenerator(horizontal_flip=True)

train_generator = datagen_train.flow_from_directory("train/",
                                                    target_size=(img_size,img_size),
                                                    batch_size=batch_size,
                                                    shuffle=True)

1.  Image Data Generators: It is a class or function used in deep learning. They create batch of images by rotating or flipping and zooming the original/available data. By doing this, it helps increase the variety in data and improve the outcome.
2. train_generator = datagen_train.flow_from_directory(…): This line is used to build a generator for training data by loading images from a directory. Here’s what each parameter means:

# Initialising the CNN
model = Sequential()

# 1 - Convolution layer
model.add(Conv2D(64, input_shape=(48, 48,1)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))

# 2nd Convolution layer
model.add(Conv2D(128,(5,5)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))


# Flattening
model.add(Flatten())

# Fully connected layer 1st layer
model.add(Dense(128))
model.add(BatchNormalization())
model.add(Activation('relu'))

model.add(Dense(7, activation='softmax'))

opt = Adam(lr=0.0005)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()

%%time

epochs = 15

reduce_lr = ReduceLROnPlateau(monitor='val_loss, min_lr=0.00001, mode='auto')
checkpoint = ModelCheckpoint("model_weights.h5", monitor='val_accuracy',save_weights_only=True, mode='max')
callbacks = [PlotLossesKerasTF(), checkpoint, reduce_lr]

history = model.fit(
    x=train_generator,
    steps_per_epoch=steps_per_epoch,
    epochs=epochs,
)

Output