Smile detection using OpenCV Python

In this tutorial, we will learn how to detect a smile in the face using Python OpenCV library. To achieve this we are using a trained model for smile and face. You can find both the haar cascades by clicking on this link. First, we need to install Python OpenCV library on our system. If you don’t know how to install Python OpenCV library Click here.

This program will detect the smile on our face using a webcam and draws a rectangle on the detected smile. Let’s move to our coding section.

Smile Detection Code

import cv2
face_cascade=cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
smile_cascade = cv2.CascadeClassifier('haarcascade_smile.xml')

First, import Python OpenCV library. Then add to haar cascade objects for face and smile which are .xml files as shown in the code. You have to download those things externally and give the path of those in CascadeClassifier function.

cap = cv2.VideoCapture(0)

The above line will capture the video from our webcam if we give an argument as ‘0’ in VideoCapture.

while cap.isOpened():
    _, img = cap.read()

    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    smiles = smile_cascade.detectMultiScale(gray, 2, 4)
    faces = face_cascade.detectMultiScale(gray, 2, 4)

cap.read() will return an image frame at every loop. Then cvtColor will change that frame into grayscale because we don’t need a colored image to detect the objects. Moreover, the grayscale image will take less space to run and the process of object detection will be speeded up.

Using detectMultiScale() we are finding the coordinates of the detected object. We have added both face and smile cascade above.

for (x1, y1, w1, h1) in faces:
    cv2.rectangle(img, (x1, y1), (x1 + w1, y1 + h1), (255, 0, 0), 3)
    face_gray= gray[y1:y1 + h1, x1:x1 + w1]
    face_color = img[y1:y1 + h1, x1:x1 + w1]

The object faces have values of coordinates for detected face, width, and height fo that detected object too. So we iterate through all those values and will draw a rectangle around the face using cv2.rectangle(). In this function, we need to give an argument of the image on which we want to draw rectangle, coordinates, the color of the rectangle in BGR format, and at last the width of a line. face_gray will be having face area only in grayscale format. Similarly, face_color will be having colored face area as an image array.

for (x, y , w ,h) in smiles:

    cv2.rectangle(face_color, (x,y), (x+w, y+h), (0, 0 ,255), 3)

But our goal is to detect smiles in the face. For that, we will again iterate through all detected smile but we only draw the smile on the detected face to avoid unnecessary smile detection outside of the face. We will draw the rectangle around that smile.

cv2.imshow('img', img)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cv2.imshow() will show the output. If we press ‘q’ then the program will terminate and waitkey is responsible to keep cv2 window on the screen.

cap.release()
cv2.destroyAllWindows()

These two lines will be used if the program is terminated then the camera will be released and the second line will destroy all the windows of that program that were loaded previously.

The complete Python program for smile detection using OpenCV

Now, we will see the full code of smile detection. below is our complete code to detect smile:

import cv2
face_cascade=cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
smile_cascade = cv2.CascadeClassifier('haarcascade_smile.xml')

cap = cv2.VideoCapture(0)

while cap.isOpened():
    _, img = cap.read()

    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    smiles = smile_cascade.detectMultiScale(gray, 2, 4)
    faces = face_cascade.detectMultiScale(gray, 2, 4)
    for (x1, y1, w1, h1) in faces:
        cv2.rectangle(img, (x1, y1), (x1 + w1, y1 + h1), (255, 0, 0), 3)
        face_gray= gray[y1:y1 + h1, x1:x1 + w1]
        face_color = img[y1:y1 + h1, x1:x1 + w1]
        for (x, y , w ,h) in smiles:

            cv2.rectangle(face_color, (x,y), (x+w, y+h), (0, 0 ,255), 3)

    
    cv2.imshow('img', img)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

I hope you will be able to detect the smile using this code. Use a finely trained haar cascade to detect the objects precisely.