What Is Clustering?

Clustering is a task that consists of grouping a set of (unlabeled) objects into subsets of objects called Clusters. Each Cluster is made up of a collection of objects that are similar (or considered similar) to each other, but different from the objects in other Clusters.

In the ML field, Clustering falls under unsupervised learning; that is, for this technique we only have a set of input data, and from it we must obtain information about the structure of the output domain, which is information we don’t have.

It is important not to confuse Clustering with Classification problems. Classification techniques fall under supervised learning because for each data point we have information about its input and output variables; that is, every data point or object is labeled. However, in cases where we don’t have the output for each data point and we want to group these objects into similar groups, we must apply one of the Clustering techniques to find out where this data comes from.

Let’s suppose the following example case: We have a data set (DS1) with the hair color of a set of people. As input we have the RGB hair color of each person and as output a finite set of labels {Moreno, Rubio, Castaño, Canoso} (Dark, Blond, Brown, Gray). On the other hand, we have another data set (DS2) with the RGB hair color of another set of people, but in this case we don’t have an output label telling us whether the individual is Moreno, Rubio, Castaño or Canoso. In both cases we want to build a model that lets us label new people as Morenos, Rubios, etc. How would we approach this problem for DS1 and DS2?

• DS1: The first case is a classification problem, since for each individual we have their RGB hair color and their corresponding class {Moreno, Rubio, Castaño, Canoso}. In this case we must obtain a function (or hypothesis) based on the training data so that it labels each new input (RGB hair color) correctly:

f(x=color)={Moreno,Rubio,Castaño,Canoso}

• DS2: The second case is a clear example of Clustering, since we have to obtain information about its structure or output domain based on the hair color data in the data set. To do so, we must use one of the Clustering techniques and tell it to group all the data into 4 Clusters. Once the Clustering is done, we can identify which individuals are Morenos, Rubios, Castaños or Canosos just by looking at the mean value (or Centroid) of each Cluster. An example of Clustering of this data set seen in 2D would be the following:

In this example we can see how it divides the individuals into 4 groups based on their hair color, and how we can then identify and label each group of people.

This example is a very simple and didactic one to see the difference between Classification and Clustering tasks; to make it very clear that the former falls under supervised learning and the latter under unsupervised learning. In practical cases where Clustering techniques have to be applied, we usually have no prior information about the data to analyze, nor about the models that generated this data. In the example we have assumed that we would only have people with these four hair types, and we don’t know whether there would be people with hair dyed blue, green, etc. This would make the Clustering task rather inaccurate, since it would possibly group people considered Morenos into different Clusters or assume that Castaño and Rubio people belong to the same Cluster.

         In short, Clustering techniques are appropriate for unsupervised learning cases in which you want to group data and gain a high-level understanding of how the data was generated and how it is organized, without any prior knowledge of it.