JavaScript image Classification in browser with ml5.js using MobileNet

In this tutorial, I am going to show you how to perform the image classification task on the browser using the ml5 JavaScript library with the MobileNet model.

I hope, you are already familiar with the ml5.js library. The ml5.js is built on top of TensorFlow.js to perform neural network machine learning tasks on the client-side.

Well, you will be amazed after knowing how easy it is to do the image classification task using the ml5.js library. With this library, you have to write the very little amount of code to do image classification, object detection task.

To classify a single image, here we are going to use the MobileNet architecture. MobileNet is an architecture proposed by the tech giant Google to make the model size smaller as well as making it easier to perform the image classification tasks.

Image classification in ml5.js JavaScript code

First of all, I want to inform you that our image classification code will work on the server. You can use node.js server or apache server or any other server you like to run the source code.

Now let’s start writing our code. before we go forward, include our required JavaScript libraries from CDN:

<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/p5.min.js"></script> 
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/addons/p5.dom.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/addons/p5.sound.min.js"></script>
<script src="https://unpkg.com/[email protected]/dist/ml5.min.js"></script>
<script src="main.js"></script>

Our complete HTML code will look something like you can see below:

<!DOCTYPE html>
<html lang="en">
  <head>
    <title>Getting Started with ml5.js</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/p5.min.js"></script> 
    <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/addons/p5.dom.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/addons/p5.sound.min.js"></script>
   
    <script src="https://unpkg.com/[email protected]/dist/ml5.min.js"></script>
    <script src="main.js"></script>
  </head>

  <body>
   
  </body>
</html>

Very simple and state forward. Isn’t it. You can give any name to this file with file type .html. We don’t need anything in this HTML file to do now. All we need to do is in our main.js file.

Now let’s start coding in our main.js file. All the code below will be inside the main.js file.

At the very first, we use the p5 setup() function to initiate our environment:

function setup()
{
  createCanvas(500,450);
  myImg=createImg('dog.jpg',onImgReady);
  myImg.hide();
  mobilenet = ml5.imageClassifier('MobileNet',onModelReady);
}

Inside our setup function, we have created a canvas to show our image using the p5 createCanvas() method. After that creating our imageusing the createImg() method where we pass our image path as well as the callback function onImgReady() to display the image on the web page:

function onImgReady()
{
  console.log("Img is ready"); // Check if the image is ready
  image(myImg,0,0,500,450); // Set top, left, width and heigh of image
}

The above code will show the notice in the console log when the image is ready. Also, the image() method set the top and left position of the image to the canvas, width and height.

With the ml5.imageClassifier() method, we have created an object that is able to classify an image. You can see that the first parameter of the ml5.imageClassifier() method os ‘MobileNet’ which means, it will use the MobileNet model to classify our image.

The second parameter in the ml5.js imageClassifier() method is onModelReady which is a callback function. So we have to create the function.

So below is our callback function onModelReady() which contains only two lines of code:

function onModelReady()
{
  console.log("Model is ready to predict");
  mobilenet.predict(myImg,result); // Predict image from model
}

Inside the onModelReady() function we show the notification when our model is ready. After that, we predict the image from the MobileNet model using the ml5.js predict() method. In the predict() method, we are using another callback function where we will pass the result. So we have to create this function.

Below is the result() function which is going to be called in the predict() method:

function result(err,res)
{

  if (err) {
    console.error(err);
  } else {
        console.log(res);
        console.log(res[0].label); // get the first item
  }

}

In our above function, we check for error and then show the result in the console. In the console, we can see the result as an array that contains the object for each item and each item contains labels with the confidence or accuracy score.

We also show the first item’s label just for an example. The items are in the order of accuracy score. The very first item of the result has the highest accuracy score.

 

Well, this tutorial shows you just the basic code to classify an image. You can use the object in HTML to show the result or labels anywhere you like.

I will not go through these as this tutorial only for understanding the basics of image classification as well as keep this tutorial easy to understand to minimize the complexity.

 

The complete code of main.js to classify an image

In the end, if we combine all of our JavaScript code snippets, then our main.js file will contain the javascript code that you can see below:

function setup()
{
  createCanvas(500,450);

  myImg=createImg('dog.jpg',onImgReady);
  myImg.hide();
    // Creating the mobilenet model object
  mobilenet = ml5.imageClassifier('MobileNet',onModelReady);
}


function onModelReady()
{
  console.log("Model is ready to predict");
  mobilenet.predict(myImg,result);  // Predict image from model
}


//Show result throughthis function
function result(err,res)
{
  if (err) {
    console.error(err);
  } else {
        console.log(res);
        console.log(res[0].label); // get the first item
  }

}

function onImgReady()
{
  console.log("Img is ready"); // Check if the image is ready
  image(myImg,0,0,500,450);  // Set top, left, width and heigh of image
}

That’s it…

Now if we run our HTML file on our browser, we can see the result on console looks like you can see below:

 

Also, read:

So we have successfully able to classify image on client-side in JavaScript with the help of the ml5.js library and the MobileNet model.

Img is ready
Model is ready to predict
Array(3) [ {…}, {…}, {…} ]
Labrador retriever

If we expand the array, it will look like:

 […]
​
0: Object { label: "Labrador retriever", confidence: 0.8145386576652527 }
​1: Object { label: "golden retriever", confidence: 0.13624528050422668 }
​2: Object { label: "Rhodesian ridgeback", confidence: 0.002484458265826106 }
​length: 3
​<prototype>:  Array []

As you can see, the array contains three objects which hold items with labels and confidence score.

 

You can clearly see that, we have to write only 35 lines of JavaScript code to classify our image. This is the beauty of the ml5.js library.

If the same thing we had to do with only TensorFlow.js than we must have to write even more code and it will become an even more complex task. Here you don’t have to know the complex mathematics that uses for the machine learning task.

The ml5.js library can be used by any programmer with knowledge in JavaScript for client-side browser. So we must say thanks for the contributors to this amazing library.

Leave a Reply

Your email address will not be published. Required fields are marked *