Optical Flow in OpenCV Python

Hello Programmers, In this tutorial, we will learn about Optical Flow in OpenCV using Python.

Optical flow is the flow (motion) of objects between two consecutive frames caused by the movement of the subject or camera. The movement of the subject (object) in a video frame can be detected using optical flow.

We will use OpenCV to determine Optical flow in the video frame. OpenCV is an open-source Computer vision library that provides tools for image processing, Images based Machine learning functionalities, etc. We need to install OpenCV and NumPy libraries for this tutorial on our local machine. To do so, open Command Prompt/Terminal and write

pip install numpy

pip install opencv-python

Python Code for optical Flow using OpenCV

# Importing cv2 and numpy library
import cv2 as cv
import numpy as np

# Read the video input
cap = cv.VideoCapture("car_moving.mp4")

# read the first captured frame of the video and stored in first_cap_frame
ret, first_cap_frame = cap.read()

# BGR frame is converted into grayscale 
first_gray = cv.cvtColor(first_cap_frame, cv.COLOR_BGR2GRAY)

# Creates a mask image with zeros
mask = np.zeros_like(first_cap_frame)

# Saturation for mask image is set to 255
mask[..., 1] = 255

In this code, first, we will import openCV and Numpy library. Input video file will be read by cv.VideoCapture() function and the first frame of the video will be saved using cap.read(). As the video frame is in BGR color format, we will convert it into grayscale format.

We will create an image mask with zero pixel values which will provide a black background. Another mask with maximum saturation value i.e 255 is created.

# While loop is implemented to run each and every frame in a loop
while(cap.isOpened()):
  
  # Read the current frame from the video and store it in frame variable
  ret, frame = cap.read()
  
  # New window is opened and Input frame is displayed
  cv.imshow("input", frame)
  
  # Each frame is converted into grayscale
  current_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
  
  # optical flow using Farneback method
  optical_flow = cv.calcOpticalFlowFarneback(first_gray, current_gray,
                  None,
                  0.2, 4, 12, 2, 3, 1.1, 0)
  
  # Magnitude and Angle of 2D vector is calculated
  mag, ang = cv.cartToPolar(optical_flow[..., 0], optical_flow[..., 1])
  
  # Determines mask value using normalized magnitude
  mask[..., 0] = ang * 180 / np.pi / 2
  
  
  mask[..., 2] = cv.normalize(mag, None, 0, 255, cv.NORM_MINMAX)
  
  # Converts HSV (Hue Saturation Value) to RGB (or BGR)
  rgb_frame = cv.cvtColor(mask, cv.COLOR_HSV2BGR)
  
  # Displays new output frame
  cv.imshow("optical flow", rgb_frame)
  
  
  first_gray = current_gray
  
  # Every frame is updated in period of 1 millisecond. When user presses "k" key, the window gets terminated
  if cv.waitKey(1) & 0xFF == ord('k'):
    break

# captured frame is released and display window is terminated after video frame ends
cap.release()
cv.destroyAllWindows()

While loop is used to run each and every frame in a loop to get a motion video. Every current frame is then converted into grayscale format.

cv.calcOpticalFlowFarneback() function will create an optical flow for every current frame running inside the loop. The magnitude and angle for a 2D vector are calculated to adjust the pace of the flow on the output frame. cv.imshow() will display the output image.

Every frame is updated at 1 millisecond time interval. When the user presses “k” key, the window gets terminated. This is done using cv.waitkey(). cap.release() and cv.destroyAllWindows() functions will release and terminate the current running window automatically after the video ends.

Combining all the codes

# Importing cv2 and numpy library
import cv2 as cv
import numpy as np

# Read the video input
cap = cv.VideoCapture("car_moving.mp4")

# read the first captured frame of the video and stored in first_cap_frame
ret, first_cap_frame = cap.read()

# BGR frame is converted into grayscale 
first_gray = cv.cvtColor(first_cap_frame, cv.COLOR_BGR2GRAY)

# Creates a mask image with zeros
mask = np.zeros_like(first_cap_frame)

# Saturation for mask image is set to 255
mask[..., 1] = 255

# While loop is implemented to run each and every frame in a loop
while(cap.isOpened()):
  
  # Read the current frame from the video and store it in frame variable
  ret, frame = cap.read()
  
  # New window is opened and Input frame is displayed
  cv.imshow("input", frame)
  
  # Each frame is converted into grayscale
  current_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)
  
  # optical flow using Farneback method
  optical_flow = cv.calcOpticalFlowFarneback(first_gray, current_gray,
                  None,
                  0.2, 4, 12, 2, 3, 1.1, 0)
  
  # Magnitude and Angle of 2D vector is calculated
  mag, ang = cv.cartToPolar(optical_flow[..., 0], optical_flow[..., 1])
  
  # Determines mask value using normalized magnitude
  mask[..., 0] = ang * 180 / np.pi / 2
  
  
  mask[..., 2] = cv.normalize(mag, None, 0, 255, cv.NORM_MINMAX)
  
  # Converts HSV (Hue Saturation Value) to RGB (or BGR)
  rgb_frame = cv.cvtColor(mask, cv.COLOR_HSV2BGR)
  
  # Displays new output frame
  cv.imshow("optical flow", rgb_frame)
  
  
  first_gray = current_gray
  
  # Every frame is updated in period of 1 millisecond. When user presses "k" key, the window gets terminated
  if cv.waitKey(1) & 0xFF == ord('k'):
    break

# captured frame is released and display window is terminated after video frame ends
cap.release()
cv.destroyAllWindows()

Video frame:

Optical flow frame:

Thus we have successfully created an optical flow using OpenCV in Python.