Improve the return on your marketing investments with intelligent user segmentation in Amazon Personalize

Today, we’re excited to announce intelligent user segmentation powered by machine learning (ML) in Amazon Personalize, a new way to deliver personalized experiences to your users and run more effective campaigns through your marketing channels.

Traditionally, user segmentation depends on demographic or psychographic information to sort users into predefined audiences. More advanced techniques look to identify common behavioral patterns in the customer journey (such as frequent site visits, recent purchases, or cart abandonment) using business rules to derive users’ intent. These techniques rely on assumptions about the users’ preferences and intentions that limit their scalability, don’t automatically learn from changing user behaviors, and don’t offer user experiences personalized for each user. User segmentation in Amazon Personalize uses ML techniques, developed and perfected at Amazon, to learn what is relevant to users. Amazon Personalize automatically identifies high propensity users without the need to develop and maintain an extensive and brittle catalog of rules. This means you can create more effective user segments that scale with your catalog and learn from your users’ changing behavior to deliver what matters to them.

Amazon Personalize enables developers to build personalized user experiences with the same ML technology used by Amazon with no ML expertise required. We make it easy for developers to build applications capable of delivering a wide array of personalization experiences. You can start creating user segments quickly with the Amazon Personalize API or AWS Management Console and only pay for what you use, with no minimum fees or upfront commitments. All data is encrypted to be private and secure, and is only used to create your user segments.

This post walks you through how to use Amazon Personalize to segment your users based on preferences for grocery products using an Amazon Prime Pantry dataset.

Overview of solution

We’re introducing two new recipes that segment your users based on their interest in different product categories, brands, and more. Our item affinity recipe (aws-item-affinity) identifies users based on their interest in the individual items in your catalog, such as a movie, song, or product. The item attribute affinity recipe (aws-item-attribute) identifies users based on the attributes of items in your catalog, such as genre or brand. This allows you to better engage users with your marketing campaigns and improve retention through targeted messaging.

The notebook that accompanies this post demonstrates how to use the aws-item-affinity and aws-item-attribute recipe to create user segments based on their preferences for grocery products in an Amazon Prime Pantry dataset. We use one dataset group that contains user-item interaction data and item metadata. We use these datasets to train solutions using the two recipes and create user segments in batch.

To test the performance of the solution, we split the interactions data into a training set and test set. The Amazon Prime Pantry dataset has approximately 18 years of interaction data from August 9, 2000, to October 5, 2018, with approximately 1.7 million interactions. We hold out 5% of the most recent interactions and train on the remaining 95%. This results in a split where we use interactions from August 9, 2000, through February 1, 2018, to train the solution and use the remaining 8 months of interactions to simulate future activity as ground truth.

Results

When reproducing these tests in the notebook, your results may vary slightly. This is because when training, the solution the parameters of the underlying models are randomly initialized.

Let’s first review the results by looking at a few examples. We ran queries on three items, and identified 10 users that have a high propensity to engage with the items. We then look at the users’ shopping histories to assess if they would likely be interested in the queried product.

The following table shows the results of a segmentation query on gingerbread coffee, an item we might want to promote for the holiday season. Each row in the table shows the last three purchases of the 10 users returned from the query. Most of the users we identified are clearly coffee drinkers, having recently purchased coffee and coffee creamers. Interestingly, the item we queried on is a whole bean coffee, not a ground coffee. We see in the item histories that, where the information is available, the users have recently purchased whole bean coffee.

The next table shows a segmentation query on hickory liquid smoke, a seasoning used for barbecuing and curing bacon. We see a number of different grocery products that might accompany barbecue in the users’ recent purchases: barbecue sauces, seasonings, and hot sauce. Two of the users recently purchased Prague Powder No. 1 Pink Curing Salt, a product also used for curing bacon. We may have missed these two users if we had relied on rules to identify people interested in grilling.

Our third example shows a segmentation query on a decoration used to top cakes. We see that the users identified are not only bakers, but are also clearly interested in decorating their baked goods. We see recent purchases like other cake toppers, edible decorations, and fondant (an icing used to sculpt cakes).

These three examples make sense based on our editorial judgement, but to truly assess the performance of the recipe, we need to analyze more of the results. To do this broader assessment, we run the aws-item-affinity solution on 500 randomly selected items that appear in the test set to query a list of 2,262 users (1% of the users in the dataset). We then use the test set to assess how frequently the 2,262 users purchased the items during the test period. For comparison, we also assess how frequently 2,262 of the most active users purchased the items during the test period. The following table shows that the aws-item-affinity solution is four times better at identifying users that would purchase a given item.

Although these results are informative, they’re not a perfect reflection of the performance of the recipe because the user segmentation wasn’t used to promote the items which users later interacted with. The best way to measure performance is an online A/B test—running a marketing campaign on a list of users derived from the aws-item-affinity solution alongside a set of the most active users to measure the difference in engagement.

Conclusion

Amazon Personalize now makes it easy to run more intelligent user segmentation at scale, without having to maintain complex sets of rules or relying on broad assumptions about the preferences of your users. This allows you to better engage users with your marketing campaigns and improve retention through targeted messaging.

To learn more about Amazon Personalize, visit the product page.

About the Authors

Daniel Foley is a Senior Product Manager for Amazon Personalize. He is focused on building applications that leverage artificial intelligence to solve our customers’ largest challenges. Outside of work, Dan is an avid skier and hiker.

Debarshi Raha is a Senior Software Engineer for Amazon Personalize. He is passionate about building AI-based personalization systems at scale. In his spare time, he enjoys traveling and photography.

Ge Liu is an Applied Scientist at AWS AI Labs working on developing next generation recommender system for Amazon Personalize. Her research interests include Recommender System, Deep Learning, and Reinforcement Learning.

Haizhou Fu is a senior software engineer on the Amazon Personalize team working on designing and building recommendation systems and solutions for different industries. Outside of his work, he loves playing soccer, basketball and watching movies, reading and learning about physics, especially theories related to time and space.

Yifei Ma is a Senior Applied Scientist at AWS AI Labs working on recommender systems. His research interests lie in modeling and decision making in large-scale temporal domains, using tools in causal analysis, reinforcement learning, distributed deep learning, approximate inference, and uncertainty-driven exploration.