Howdy Folks, In this tutorial you will learn how to build an Image Classifying Web App using the popular Javascript Machine Learning library - "ml5"!!! Our finished product will look something like in the above banner.

Before continuing this tutorial, first, take a look at the demo here on Codepen - Image Classifier JS

You can download the completed project from GitHub - Image Classifier JS

Or you can get the final code for each file at the very end...

Prerequisites

I assume you are already familiar with basic Javascript. And you should have a decent understanding of HTML, and CSS. I am not going to explain the CSS code, because our main focus here is the HTML and Javascript. But if you didn't understand the CSS, then please comment below. I am always here to help you 😇.

Besides the things above mentioned, you will need:

  • High-Speed Internet Connection, for calling the ml5 library from a CDN (means another server). Make sure your internet connection is fast enough. Because speed is a huge factor here!!!
  • Then, a Modern Web browser(such as Firefox or Chrome)
  • Finally, a Text Editor.

I am going to use "visual studio code" as my text editor. You are free to use whatever code editor you like. And finally, make sure you have installed Live Server by Ritwick Dey in visual studio code. If you don't have Live Server installed, then:

  • Open visual studio code
  • Goto Extensions tab (Ctrl + Shift + X)
  • Search for "Live Server" by Ritwick Dey
  • Then Install it

If you want to jump straight into the project, then start from the "Getting Started" section.

Understanding Some Technical Jargons

If you are new to Machine Learning, then you maybe wondered about what this thing is all about. Machine Learning is nothing more than a branch of Artificial Intelligence that provides computers the ability to automatically learn and improve from experience without being explicitly programmed.

Here we are going to use a Javascript Machine Learning library called "ml5". The ml5 library aims to make machine learning approachable for every audience out there. This library provides us with all types of machine learning models!!

Now, what's a model?

You can think of models as a file/algorithm that has been trained to recognize certain types of patterns in the data. Once you trained it, you can give some new data to make some predictions.

For example, let's say you want to build an application that can recognize and classify images according to the content in it. You can train a model by providing it with lots and lots of images each tagged with a certain label that describes the image. After the training session, now your model will become efficient to classify a new image. If you give one, it will try to classify it into a class that it has seen before.

But when we use a machine learning library like ml5, we don't want to build a model from scratch and train it with millions and millions of data. Why? Because our library already comes prebaked with several models that we can use out of the box!!!

That's cool, right? Because we can build some cool web applications that can perform Face Detection, Image Classification, Sound Classification, Sentiment Analysis, and much much more by just importing the models!!!

And that's what we are going to do today!!! We will use a model called 'MobileNet'  from the ml5 library to build our Image Classification web app. According to their official docs, the MobileNet model was trained on a database of approximately 15 million images!!!

And one more important thing to note here is that the performance of our MobileNet model will entirely depend on those training data -- what is included, excluded, and how those images are labeled (or mislabeled).

I think now you are interested to do this project 🔥. But before that, let's plan our project.

Planning our Project

The break down of the final project will look something like this:

project structure break down

It contains:

  • A .container which holds everything together.
  • An .image-box which holds:
    • .image
    • .label-text
    • .accuracy-text
  • An .upload-btn
  • And a .predict-btn

But we are not going to build this whole app in just one go. Instead, we will split the project into two parts:

  • In the first part, we will build the Basic version.
  • And in the second part, we will build the Enhanced version.

The Basic version will look like this:

basic version

See it contains no ".upload-btn". It only has the other elements.

After testing the model with the default image, then we will try to build the Enhanced version.

Our Enhanced version is nothing more than just adding a .upload-btn and it will look like this:

Enhanced version with upload button

Now let's get started...

Getting Started

Create a folder structure similar to below:

folder structure 

  • First, create the root folder which holds everything and name it as "Image Classification" or anything you like.
  • Then open this folder inside visual studio code.
  • Then directly inside this root folder, create the following files:
    • index.html - this is going to hold all our HTML code.
    • styles.css - this will hold all of our CSS styles.
    • script.js - this is the most important file. This will hold all of our Javascript logic.
    • Finally, download this whole "img" folder --> img. Unzip it. Then put it directly inside our root folder. This "img" folder contains an image called "animal.jpeg", which we will use as the default image.

That's it. Now you would have a folder structure similar to above. Now let's start building the Basic version.

Basic Version

Let's start with the HTML. 

Open "index.html" and type the following:

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <!-- Link to our CSS stylesheet -->
  <link rel="stylesheet" type="text/css" href="./styles.css">
  <!-- CDN Link to the ml5 library -->
  <script type="text/javascript" src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js"></script>
  <title>Image Classification</title>
</head>
<body>

  <!-- container -->
  

  <!-- Link to our Javascript file  -->
  <script type="text/javascript" src="./script.js"></script>
</body>
</html>

These are some basic HTML codes. Besides the normal HTML code, have you noticed anything else?

Yes, we had a link to the "ml5" library just below the link to our CSS file. This is the most important link on this project. Never forget this!!!

Link to ml5 library CDN

This link will load the entire "ml5" library into our project from a remote server.

Now type the following code below this '<!-- container -->' comment in the index.html file:

<div class="container">

    <!-- image-box -->
    <div class="image-box">
      <img src="./img/animal.jpeg" class="image">
      <!-- label-text -->
      <p class="label-text">Unidentified</p>
      <!-- accuracy-text -->
      <p class="accuracy-text">Accuracy: 0%</p>
    </div>

    <!-- upload-btn will go here -->

    <!-- predict-btn (disabled) -->
    <button class="predict-btn" disabled>PREDICT</button>

  </div>

This will create all the necessary elements we need. Here we have the:

  • .container - which holds everything together.
  • .image-box - which holds the default .image, .label-text, and .accuracy-text.
  • Then a comment saying '<!-- upload-btn will go here -->'. This is the place where we will put the .upload-btn in the future.
  • Finally, a disabled .predict-btn. We will enable this once we finished loading the "MobileNet" model. We will do this from within the javascript file.

Now Open it with Live Server by right-clicking on the "index.html" file (inside visual studio code), then scroll down and click on the "Open with Live Server" option and take a look at it at the browser. You won't see any styling now. Because we only linked the "styles.css" file but didn't write any CSS styles. So now let's do that. Let's add some styles...

CSS

You can either copy or type these styles. Because CSS is not our main focus here.

Open the "styles.css" file you created and type or copy the following:

/* common styles */
* {
  padding: 0;
  margin: 0;
  box-sizing: border-box;
}

/* body */
body {
  width: 100vw; /* 100% viewport width */
  height: 100vh; /* 100% viewport height */
  overflow-x: hidden;
  display: flex;
  justify-content: center; /* horizontally center */
  align-items: center; /* vertically center */
  background: linear-gradient(to right, #0072ff, #00c6ff); /* background color */
  font-size: 1.2rem; /* default font-size */
  font-family: sans-serif;
}

/* .container */
.container {
  display: flex;
  flex-direction: column;
  justify-content: center; /* now this is for vertically center. Because of flex-direction: column; */
  align-items: center; /* horizontally center */
}

/* .image-box */
.image-box {
  background-color: #fff;
  padding: 15px; /* inner spacing */
  margin: 30px 0px 10px 0px; /* outer spacing. Order -> top right bottom left*/
  display: flex;
  flex-direction: column;
  align-items: center;
  text-align: center;
}

/* .image */
.image {
  width: 230px; /* the more you increase width, the more precise the prediction will be! */
  margin-bottom: 15px;
}

/* .label-text */
.label-text {
  font-size: 1.5rem;
  font-family: cursive;
  max-width: 200px;
  margin-bottom: 7px;
}

/* .accuracy-text */
.accuracy-text {
  margin-bottom: 2px;
}

/* .predict-btn */
.predict-btn {
  background-color: #00ff00;
  font-size: 1.4rem;
  padding: 5px;
  margin: 20px 0px; /* top-bottom left-right */
  outline: none;
  cursor: pointer;
  border-radius: 3px;
  box-shadow: 2px 2px 2px rgba(0, 0, 0, 0.5); /* x-offset y-offset blur-radius color  */
}

/* .upload-btn styles will go here */

Here you will again see a placeholder at the very bottom:

placeholder comment for .upload-btn styles

The CSS styles for the .upload-btn will go here later. 

Now it's time to make our app working. So let's work on the Javascript logic.

Javascript

The first thing, as always, is to grab a reference for the necessary HTML elements from the DOM. The elements we need are the .image, .label-text, .accuracy-text, and .predict-btn.

Open the "script.js" file and type the following:

// grab reference for needed elements
// .image
const image = document.querySelector('.image');
// .label-text
const labelText = document.querySelector('.label-text');
// .accuracy-text
const accuracyText = document.querySelector('.accuracy-text');
// .predict-btn
const predictBtn = document.querySelector('.predict-btn');
// .upload-btn will go here

Here we are selecting everything using the document.querySelector(). I hope you are already familiar with that. 

Now let's load our "MobileNet" model. Type this below the above code:



// Loading our MobileNet model
const model = ml5.imageClassifier("MobileNet", modelLoaded);

We load the model by calling the method ml5.imageClassifier(). Remember we can use the ml5 variable here because we had already linked the ml5 library from the CDN in the "index.html" file.

The ml5.imageClassifier() expects the model name as its first argument. Then as the second argument, we give the name of the callback function. In this case, our callback function is modelLoaded.

Now Let's create the modelLoaded callback function below the above code:

// modelLoaded() callback function definition
function modelLoaded() {
  
  // first just console.log() 'Model Loaded'
  console.log('Model Loaded!');
  // then enable the .predict-btn
  predictBtn.disabled = false;

  /* then attach an 'click' eventListener to .predict-btn */
  predictBtn.addEventListener('click', predict);
}

After our model has been successfully loaded, it activates this modelLoaded callback function we created. This function does the following:

  • First, it will just console.log() the message saying 'Model Loaded!'.
  • After that, it will enable the .predict-btn which was initially disabled by us.
  • Then, it will attach a 'click' event listener to that button so that the users can click on it. When someone clicks on it, a function named predict will get activated. This function is going to do the rest of the functionalities like classifying our .image  and updating the UI.

So let's create the predict() function below the previous code:

// predict() callback function definition
function predict() {
  /* take the .image and classify() it */
  model.classify(image, function (error, results) {
    // check if error happened
    if (error) {
      // just console.log() the error
      console.log('Error: ', error);
    } else {
      /* from "results" array, grab first element's "label" and "confidence" level */
      const label = results[0].label.toUpperCase();
      const accuracy = (results[0].confidence * 100).toFixed(2);

      /* update .label-text and .accuracy-text */
      labelText.innerText = label;
      /* Careful Here: Use back-ticks ``, Not single quote '' or double quote "" */
      accuracyText.innerText = `Accuracy: ${accuracy}%`;
    }
  });
}

Now the explanation for the above code: 

As you know, When someone clicks on the .predict-btn, this predict() callback gets activated. What this function does is that:

  • First of all, it will activate the classify() method that belongs to the "MobileNet" model:

model.classify() function

  • As you can see, it takes the entire image as its first argument. Then tries to classify it. After that, it gives either the results or the error to the callback function. See the callback function is the second argument.
  • Inside the callback function, we first check if any error had happened or not. If there is any error, then just console.log() it.
  • If there is no error, then it means that we have got some prediction/classification results!!! The results we got would almost look like this (these are the results for our default "animal.jpeg" image (the image of the monkey)):

results we got in Javascript Object format

  • See, it is an array of multiple Javascript objects. We don't need all of them. Instead, we need the one with the highest accuracy, and guess what, it is the first one. So we extracted the "label" and "confidence" like this:

extraction

  • Then updated the UI like this:

updating the UI

That's it, we had successfully built the Basic version of our Image Classification Web App!!!.

Go take a look at it in the browser!!! Play with it...

Enhanced Version

After playing it with, you may have experienced the lack of a file upload button. Haven't you? That's what we are going to do now. We are going to enhance our app by including a .upload-btn. Let's do it fast.

First, open the "index.html" file and type the following code below this comment '<!-- upload-btn will go here -->' and save the file:

    <input type="file" class="upload-btn" />

Secondly, open the "styles.css" file and include these styles at the very bottom of the file below this comment '/* .upload-btn styles will go here */' and save it too:

/* hide the default upload button */
.upload-btn::-webkit-file-upload-button {
  display: none;
}

/* create our own upload button */
.upload-btn::before {
  content: 'Upload Image';
  display: block;
  background-color: yellow;
  font-size: 1.3rem;
  text-align: center;
  padding: 5px 8px;
  margin: 15px 0px 0px 0px;
  border-radius: 3px; /* for curved edges */
  border: none;
  outline: none;
  cursor: pointer;
  box-shadow: 2px 3px 9px rgba(0, 0, 0, 0.5);
}

/* remove the "outline" and "border" */
input[type="file" i] {
  outline: none;
  border: none;
}
  • Don't worry about these long styles. Here we are just hiding our original ugly file upload button and then creating our own. 

And as the final step, now open the "script.js" file and do these two things:

  • First, grab the reference for the .upload-btn we created in the HTML file. Type this at the very top below this comment '// .upload-btn will go here':
const uploadBtn = document.querySelector('.upload-btn');
  • Then, attach a 'change' event listener to the .upload-btn like this at the very bottom of the file:
/* attach a 'change' eventListener to .upload-btn */
uploadBtn.addEventListener('change', function (event) {
  // check if user had selected any file
  if (event.target.files[0]) {
    /* create an ObjectURL for the selected/uploaded file */
    const objectURL = URL.createObjectURL(event.target.files[0]);

    /* then replace previous .image on DOM with the new file the user had uploaded */
    image.src = objectURL;
  }
});
  • Why a 'change' event listener? Because, whenever the user tries to upload a new file/image, an event called 'change' will get emitted. That's why we are listening to the 'change' event. 
  • When that happens, the callback function will get activated. Inside this callback function:
    • First, we are checking if the user had selected any file or not:

checking

  • If the user had selected any file/image, then we need to update the current .image in the browser with the file the user had selected, right? That's what we are doing in the last two lines:

last two lines

That's it, we had reached the end!!! Hooray!!!

Now it's the testing time!!!

After making sure the code you typed up to this point inside all the files are correct by cross-checking it with the final code provided below, try to upload some different images and try to predict/classify it. Go and see how well the model classifies each of your images. Play with it, Folks!!!

Here are the final codes.

Final code for "index.html":

<!DOCTYPE html>
<html>

<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <!-- Link to our CSS stylesheet -->
  <link rel="stylesheet" type="text/css" href="./styles.css">
  <!-- CDN Link to the ml5 library -->
  <script type="text/javascript" src="https://unpkg.com/ml5@0.4.3/dist/ml5.min.js"></script>
  <title>Image Classification</title>
</head>

<body>

  <!-- container -->
  <div class="container">

    <!-- image-box -->
    <div class="image-box">
      <img src="./img/animal.jpeg" class="image">
      <!-- label-text -->
      <p class="label-text">Unidentified</p>
      <!-- accuracy-text -->
      <p class="accuracy-text">Accuracy: 0%</p>
    </div>

    <!-- upload-btn will go here -->
    <input type="file" class="upload-btn" />

    <!-- predict-btn (disabled) -->
    <button class="predict-btn" disabled>PREDICT</button>

  </div>

  <!-- Link to our Javascript file  -->
  <script type="text/javascript" src="./script.js"></script>
</body>

</html>

Final code for "styles.css":

/* common styles */
* {
  padding: 0;
  margin: 0;
  box-sizing: border-box;
}

/* body */
body {
  width: 100vw; /* 100% viewport width */
  height: 100vh; /* 100% viewport height */
  overflow-x: hidden;
  display: flex;
  justify-content: center; /* horizontally center */
  align-items: center; /* vertically center */
  background: linear-gradient(to right, #0072ff, #00c6ff); /* background color */
  font-size: 1.2rem; /* default font-size */
  font-family: sans-serif;
}

/* .container */
.container {
  display: flex;
  flex-direction: column;
  justify-content: center; /* now this is for vertically center. Because of flex-direction: column; */
  align-items: center; /* horizontally center */
}

/* .image-box */
.image-box {
  background-color: #fff;
  padding: 15px; /* inner spacing */
  margin: 30px 0px 10px 0px; /* outer spacing. Order -> top right bottom left*/
  display: flex;
  flex-direction: column;
  align-items: center;
  text-align: center;
}

/* .image */
.image {
  width: 230px; /* the more you increase width, the more precise the prediction will be! */
  margin-bottom: 15px;
}

/* .label-text */
.label-text {
  font-size: 1.5rem;
  font-family: cursive;
  max-width: 200px;
  margin-bottom: 7px;
}

/* .accuracy-text */
.accuracy-text {
  margin-bottom: 2px;
}

/* .predict-btn */
.predict-btn {
  background-color: #00ff00;
  font-size: 1.4rem;
  padding: 5px;
  margin: 20px 0px; /* top-bottom left-right */
  outline: none;
  cursor: pointer;
  border-radius: 3px;
  box-shadow: 2px 2px 2px rgba(0, 0, 0, 0.5); /* x-offset y-offset blur-radius color  */
}

/* .upload-btn styles will go here */
/* hide the default upload button */
.upload-btn::-webkit-file-upload-button {
  display: none;
}

/* create our own upload button */
.upload-btn::before {
  content: 'Upload Image';
  display: block;
  background-color: yellow;
  font-size: 1.3rem;
  text-align: center;
  padding: 5px 8px;
  margin: 15px 0px 0px 0px;
  border-radius: 3px; /* for curved edges */
  border: none;
  outline: none;
  cursor: pointer;
  box-shadow: 2px 3px 9px rgba(0, 0, 0, 0.5);
}

/* remove the "outline" and "border" */
input[type="file" i] {
  outline: none;
  border: none;
}

Final code for "script.js":

// grab reference for needed elements
// .image
const image = document.querySelector('.image');
// .label-text
const labelText = document.querySelector('.label-text');
// .accuracy-text
const accuracyText = document.querySelector('.accuracy-text');
// .predict-btn
const predictBtn = document.querySelector('.predict-btn');
// .upload-btn will go here
const uploadBtn = document.querySelector('.upload-btn');

// Loading our MobileNet model
const model = ml5.imageClassifier("MobileNet", modelLoaded);

// modelLoaded() callback function definition
function modelLoaded() {
  
  // first just console.log() 'Model Loaded'
  console.log('Model Loaded!');
  // then enable the .predict-btn
  predictBtn.disabled = false;

  /* then attach an 'click' eventListener to .predict-btn */
  predictBtn.addEventListener('click', predict);
}

// predict() callback function definition
function predict() {
  /* take the .image and classify() it */
  model.classify(image, function (error, results) {
    // check if error happened
    if (error) {
      // just console.log() the error
      console.log('Error: ', error);
    } else {
      /* from "results" array, grab first element's "label" and "confidence" level */
      const label = results[0].label.toUpperCase();
      const accuracy = (results[0].confidence * 100).toFixed(2);

      /* update .label-text and .accuracy-text */
      labelText.innerText = label;
      /* Careful Here: Use back-ticks ``, Not single quote '' or double quote "" */
      accuracyText.innerText = `Accuracy: ${accuracy}%`;
    }
  });
}

/* attach a 'change' eventListener to .upload-btn */
uploadBtn.addEventListener('change', function (event) {
  // check if user had selected any file
  if (event.target.files[0]) {
    /* create an ObjectURL for the selected/uploaded file */
    const objectURL = URL.createObjectURL(event.target.files[0]);

    /* then replace previous .image on DOM with the new file the user had uploaded */
    image.src = objectURL;
  }
});

Wrapping Up

I hope you enjoyed this tutorial. If you had any doubts, then please comment them below. Thank you for your attention and patience ;)