Differences between Clustering and Segmentation: How these types of member grouping can be applied in retail analytics

This article outlines approaches to member grouping, types of member grouping, their purposes and differences, an interface used to manage different types of groups, and changes to be implemented in an AI analytics platform to use these tools.

Terms, definitions, types of grouping

When we analyze a set of people, it is helpful to split the entire set of people into subsets. This process is called grouping.

Without grouping, when we need to take some action pertaining to the people in our set, we must either work with the entire set or work with individuals. Either option may be appropriate to some situations, but working with the entire set offers no flexibility, while working with individuals is often prohibitively difficult. Grouping is an effective compromise. Using groups allows us to work with a limited number of people who have something in common while keeping the overall process manageable, since the number of groups is usually relatively small.

Let us consider a set of people and existing approaches to split or extract some subsets out of it. The image below graphically depicts a set of customers of a store.

These people can be distinguished by features like age and gender, and the location of their icons in the image reflects their buying habits.

With a large set of people, there are nearly infinite ways to split them into groups or extract groups from the set.

Simple/Manual grouping

Users of AI analytics-powered solutions could decide to group people by family relationships, divide a set of people equally, or apply some other rule that might be hard to describe in a format that is understandable by a computer. Our AI developers call such a type of grouping a simple grouping, manual grouping, or just grouping.

For example, in the picture below, the user split a set of people into four parts that were supposed to be numerically equal. Each colour is a group.

Segments

If a group of people is extracted using some formal rule, we call such a group a segment. Examples of formal rules include:

• People 45-65 years old
• Women
• People who bought a burger
• Top 20% of spenders last year
• People who have not purchased beverages

Any logical combination of formal rules is still a formal rule. We can create a rule that combines multiple other rules, such as in the following logical expression:

(Women && People 45-65 years old) || (Men && 50-70 years old))

This expression denotes a segment of people who are either women 45-65 years old or men 50-70 years old.

We can create rules as complex as we need them to be. Once we define rules, the system extracts segments. In other words, a human user starts with an idea of how to split people into segments and describes formal rules consistent with their idea, and then the data analysis feature does the rest of the job.

It is worth mentioning that some criteria include time frames. A time frame might be either a specific date-time period (such as “June 1 - June 30”) or a floating period of time (such as “the last 30 days”). In the latter case, the members of segments might change during this time. Take the example of a segment of people who purchased cosmetics within the last 30 days. It is clear that over time, some people will enter the segment and others will leave it. If users want to examine these changes carefully, the AI analytics platform should allow them to adjust members of a segment so that it will contain those members who matched the criteria at the time the segment was created.

The images below represent example segmentations based on gender and on age. Each colour is a segment.

Clustering

The manual groupings and segmentations described above are fairly intuitive: a human operator knows a rule and applies it either manually or automatically to extract a subset of people.

But what if there is some hidden rule, something that a human being can barely see? Some complex patterns in people’s behaviour make it possible to split them into subsets where members’ commonality is much less obvious. This is where AI comes into play.

Given a set of members and some subset of their transactions (we can limit products and purchase time frame), the AI finds patterns in purchasing behaviour and splits people into clusters. Members of a cluster have similar product preferences and purchasing behavior.

In the picture above, the AI detected patterns in purchasing behaviour and created four clusters.

Usually, it is not obvious at first glance what members of a cluster have in common. We provide a variety of tools that allow us to analyze clusters and understand what unites their members.

Although the necessity of analyzing clusters after they are created may at first appear to be a drawback, it is in fact a benefit. It allows us to split people into clusters based not on our assumptions, but on real, objective facts about their behaviour, which in turn permits us to make predictions of a quality that was previously unattainable.

Suppose we sell cosmetics and want to find target groups to conduct promotions separately for both men’s and women’s cosmetics. The most obvious approach is to split buyers into two segments–men and women–and target each group with corresponding promotions. However, if we apply clustering, it will become apparent that the subset of people who frequently buy cosmetics for men has a significant number of women in it.

Purpose of grouping and objective/subjective data

There are various purposes for grouping people, but the most common are:

• Tracking the behaviour of a group as a whole.
• Targeting a group for some marketing activity.
• A mix of tracking and targeting.

Groups might also be used to search for some related information, such as transactions.

Tracking

There are three main types of tracking:

• Static tracking

  used when we want to know certain performance indicators (such as average checks or sales) for a given period of time. E.g., “What were the total sales for last year?”

• Dynamic tracking

  used when we want to know how certain performance indicators change over time, such as how monthly sales change month to month.

• Group size tracking

  used when we want to track how the size of a segment changes over time. E.g., “If we have a segment of people whose average check is more than $100, is the number of such people growing?”

Sometimes it makes sense to freeze the set of people so that you track the behaviour of the same fixed set of people, such as, “How do these specific 100 people perform over time?”

In other cases, what is needed is not to fix the group’s members, but to set criteria for segmenting and tracking the behaviour of the segment over time. In such cases, both the number of members in the segment and performance indicators will change. For example, suppose we run an advertising campaign on cosmetics and want to track what is successful. We can create a segment of people who purchased a cosmetic product at least twice within 6 months and then track 1) whether this segment is growing, and 2) if the sales and profit of this segment are growing.

Targeting

With a large client base, marketing to each separately is hardly practical. Rather, we create client groups and conduct actions within those groups. Depending on the goal of the marketing activity, different types of grouping may be more suitable.

If we target people based on age, gender, or on their performance indicators (such as the number of purchases or average checks), then a segment is a good choice. We define the criteria to create a segment and let the AI take over from there. In such cases, even before we create the segment, we know exactly what marketing activities we want to conduct.

But at other times, how to market and to whom are what we need to decide. We know that we want to conduct some personalized action, but we are not sure what groups to create or what to offer them. In that situation, clustering is the best fit. We can create clusters, analyze them to understand what is common between cluster members, determine what their needs might be, and then define what action to take for the cluster.

Objective and subjective data

Typically what we know about our clients includes:

• Demographic information (age, gender, marital status, etc)
• Geographic information (country, state/province, city)
• Information about purchases (what products, when, discounts applied, etc)

Demographic and geographic information are usually either provided by the clients or somehow collected from external sources. There are two main issues with this information:

• It might be inaccurate, due to mistakes, misinterpretation, or other factors.
• Even if it is accurate, it does not necessarily imply anything beyond its direct meaning. For example, if a person is 70+ years old, it does not necessarily follow that he purchases a specific style of clothes.

Purchase data, by comparison, is a much more reliable source of information for most of our purposes. A person might indicate in his profile that he is 70+ years old, but then it turns out that all he buys is toys for his grandchildren. What the person buys, how often, how much he or she spends, whether he or she uses discounts: these are what we consider to be objective data. We know that this information is true even if it seems to be inconsistent with the person’s demographic data.

When conducting marketing activities, we should take into account that purchase data enables much more accurate predictions. Thus, using purchase data-based segments or clusters is much more effective for planning marketing than using groups or segments based solely on demographics.

An important exception is when the purchase data available is insufficient. For example, if a customer’s purchase history consists of a single purchase, then basing our assumptions around their demographic data may produce better results.