Amazon Personalize is a machine learning service that allows you to build and scale recommendation/personalization models in a quick and effective manner. The content below is designed to help you build out your first models for your given use case and makes assumptions that your data may not yet be in an ideal format for Amazon Personalize to use.
This repository assumes a base familiarity with the service and if you have not already done so it is recommended that you use the getting-started material below.
If you are not familiar with Amazon Personalize you can learn more about this tool on these pages:
By the end of this POC, you should have picked up the following skills:
- How to map datasets to Amazon Personalize.
- Which models or recipes are appropriate for which use cases.
- How to build models in a programmatic fashion.
- To interpret model metrics.
- To deploy models in a programmatic fashion.
- To obtain results from Personalize.
The notebooks have been scrubbed of all output before usage, however if you'd like to see a fully worked out example of this process, explore the notebooks in the completed folder.
- Deploying your working environment [see below]
- Validating and importing user-item-interaction data -
01_Validating_and_Importing_User_Item_Interaction_Data.ipynb - Validating and importing item-metadata -
02_Validating_and_Importing_Item_Metadata.ipynb - Creating and evaluating your first solutions -
03_Creating_and_Evaluating_Solutions.ipynb - Deploying campaigns and filters -
04_Deploying_Campaigns_and_Filters.ipynb - Deploying campaigns and filters -
05_Interacting_with_Campaigns_and_Filters.ipynb - Cleaning up the resources on your AWS account -
06_Clean_Up_Resources.ipynb
That shows the usual order of this process, however, if you are operating this as an assisted 2 day on-site POC, it is recommended that you at least import the user-item-interaction and item-metadata data before arriving in person.
As mentioned above, the first step is to deploy a CloudFormation template that will perform much of the initial setup work for you. In another browser window or tab, login to your AWS account. Once you have done that, open the link below in a new tab to start the process of deploying the items you need via CloudFormation.
Follow along with the screenshots below if you have any questions about deploying the stack.
Start by clicking Next at the bottom like this:
On this page you have a few tasks:
- Change the Stack name to something relevant like
PersonalizePOC - Change the Notebook Name (Optional)
- Alter the VolumeSize for the SageMaker EBS volume, default is 10GB, if your dataset is expected to be larger, please increase this accordingly.
When you are done click Next at the bottom.
This page is a bit longer, so scroll to the bottom to click Next. All of the defaults should be sufficient to complete the POC, if you have custom requirements, alter as necessary.
Again scroll to the bottom, check the box to enable the template to create new IAM resources and then click Create Stack.
For a few minutes CloudFormation will be creating the resources described above on your behalf it will look like this while it is provisioning:
Once it has completed you'll see green text like below indicating that the work has been completed:
Now that your environment has been created go to the service page for SageMaker by clicking Services in the top of the console and then searching for SageMaker and clicking the service.
From the SageMaker console, scroll until you see the green box indicating now many notebooks you have in service and click that.
On this page you will see a list of any SageMaker notebooks you have running, simply click the Open JupyterLab link on the Personalize POC notebook you have created
This will open the Jupyter environment for your POC; think of it as a web based data science IDE if you are not familiar with it. It should automatically open the PersonalizePOC folder for you, if not, just click on the folder icon in the browser on the left side of the screen and follow the documentation below to get started with your POC!
The core data for every algorithm supported in Amazon Personalize is user-item-interaction data; this notebook will guide you through the process of identifying this data, then formatting it for the service, defining your schema, and lastly importing it.
Open 01_Validating_and_Importing_User_Item_Interaction_Data.ipynb and follow along there.
Once you have completed this, you can move on to imporing metadata.
Amazon Personalize has several algorithms that can give you a results with no metadata. However, the User Personalization and HRNN-Metadata algorithm might be an interesting resource to deploy, depending your dataset.
Open 02_Validating_and_Importing_Item_Metadata.ipynb and follow along there.
Once you have completed this, you can move on to creating and evaluating your first solutions.
It is similar to the process for users, and the only algorithms that supports either data type is User Personalization and HRNN-Metadata.
In Amazon Personalize there is a concept of a solution, which is a trained model based on the data that you've provided to the service. All models are private and no data sharing occurs between accounts or even between dataset groups. This notebook will guide you through the process of training models; aka building a solution for:
- HRNN
- SIMS
- Personalized-Ranking
Something you may notice is that each of these algorithms or recipes solves a critically different problem. The goal is to show you how to build things that address a host of problems from a relatively simple dataset.
Open 03_Creating_and_Evaluating_Solutions.ipynb and follow along to build these solutions and see their results.
Once you have a series of trained solutions the next step is to deploy them. This is done inside 04_Deploying_Campaigns_and_Filters.ipynb
Here you will learn:
- Deployment and capacity planning
- How to create Item and Event Filters
Once you have a series of trained solutions the next step is to deploy them. This is done inside 05_Interacting_with_Campaigns_and_Filters.ipynb. Here you will learn:
- How to interact with a deployed solution (various approaches)
- Real-time interactions
- using filters with campaigns
- Batch exporting
Following these notebooks should have left you with a series of working models for your customer. From here, you will look to leverage how the customer accomplishes AB testing today against their goals (coversions, clicks, etc) and then start sending traffic to these models and monitoring those metrics. Over time this should build confidence and will be your path to production at scale.
More content on AB testing coming soon as well.
Finished with the POC? If you want to delete all the resources created in your AWS account while following along with these notebooks, please see the 06_Clean_Up_Resources.ipynb notebook. It will help you identify all of the Personalize resources deployed in your account and shows you how to delete them.