Amazon Rekognition is a computer vision service that makes it easy to add image and video analysis to your applications using proven, highly scalable, deep learning technology that requires no machine expertise to use. With Amazon Rekognition, you can identify objects, people, text, scenes, and activities in images and videos, as well as detect any inappropriate content. Amazon Rekognition also provides highly accurate facial analysis and facial search capabilities that you can use to detect, analyze, and compare faces for a wide variety of use cases.
Amazon Rekognition Custom Labels allows you to identify the objects and scenes in images that are specific to your business needs. For example, you can find your logo in social media posts, identify your products on store shelves, classify machine parts in an assembly line, distinguish healthy and infected plants etc. Amazon Rekognition Custom Labels provides a very simple end-to-end experience where you start by labeling a dataset. Custom Labels then build a custom machine learning model for you by inspecting the data and selecting the right machine learning algorithm. Once your model is trained you can start using your model immediately for image analysis. If you expect to process images in batches (e.g. once a day or week, or at scheduled times during the day), you can provision your custom model at scheduled times.
In this post, we show how you can build cost-optimal batch solution with Amazon Rekognition Custom Labels which provision your custom model at scheduled times, process all your images, and then deprovision your resources to avoid incurring extra cost.
This project contains source code and supporting files for a serverless application that you can deploy with the SAM CLI or using the Cloudformation links below. It includes the following files and folders:
- \functions - Code for the application's Lambda functions to check the presence of messages in a Queue, start or stop a Amazon Rekognition Custom Label Model, Analyse Images using a Custom Label Model.
- template.yaml - A template that defines the application's AWS resources.
This application creates a serverless Amazon Rekognition Custom Label Detection workflow which runs on a pre-defined schedule (note that the schedule is enabled by default at deployment). It demonstrates the power of Step Functions to orchestrate Lambda functions and other AWS resources to form complex and robust workflows, coupled with event-driven development using Amazon EventBridge.
Solution Architecture Diagram: The following architecture diagram shows how you can design a serverless workflow to process images in batches with Amazon Rekognition Custom Labels.
- As an image is stored in Amazon S3 bucket, it triggers a message which gets stored in an Amazon SQS queue.
- Amazon EventBridge is configured to trigger an AWS Step Function workflow at certain frequency (1 hour by default).
- As the workflow runs it checks the number of items in the Amazon SQS queue. If there are no items to process in the queue, workflow ends. If there are items to process in the queue, workflow starts the Amazon Rekognition Custom Labels model and enables Amazon SQS integration with a Lambda function to process those images.
- As integration between Amazon SQS queue and Lambda is enabled, Lambda start processing images using Amazon Rekognition Custom Labels.
- Once all the images are processed, workflow stops the Amazon Rekognition Custom Labels model and disables integration between Amazon SQS queue and Lambda function.
The application uses several AWS resources, including Amazon Simple Storage Service, Amazon Simple Queue Service, Step Functions state machines, Lambda functions and an EventBridge rule trigger. These resources are defined in the template.yaml file in this project. You can update the template to add AWS resources through the same deployment process that updates your application code.
Please note that this code is provided as a working sample. However, if you intend to use it in production, it is recommended that you implement production best practices including but not limited to error handling, message visibility settings, timeouts, storage lifecycle rules etc.
-
To deploy the sample application, you will require an AWS account. If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Your access to the AWS account must have IAM permissions to launch AWS CloudFormation templates that create IAM roles.
-
Please refer here for instructions on getting started with Amazon Rekognition Custom Labels. When deploying this application you will need to provide the following two parameters for your Custom Label Project.
- Amazon Rekognition Model Project ARN: The Amazon Resource Name (ARN) of the Amazon Rekognition Custom Labels project that contains the models you want to use.
- Amazon Rekognition Model Project Version ARN: The Amazon Resource Name(ARN) of the model version that you want to use.
The demo application is deployed as an AWS CloudFormation template.
Note
You are responsible for the cost of the AWS services used while running this sample deployment. There is no additional cost for using this sample. For full details, see the following pricing pages for each AWS service you will be using in this sample. Prices are subject to change.
- Deploy the latest CloudFormation template by following the link below for your preferred AWS region:
| Region | Launch Template |
|---|---|
| US East (N. Virginia) (us-east-1) |  |
| US East (Ohio) (us-east-2) | |
| US West (Oregon) (us-west-2) | |
| Europe (Ireland) (eu-west-1) | |
| Europe (London) (eu-west-2) | |
| Asia Pacific (Sydney) (ap-southeast-2) | |
| Europe (Frankfurt) (eu-central-1) | |
-
If prompted, login using your AWS account credentials.
-
You should see a screen titled "Create Stack" at the "Specify template" step. The fields specifying the CloudFormation template are pre-populated. Click the Next button at the bottom of the page.
-
On the "Specify stack details" screen you may customize the following parameters of the CloudFormation stack:
- Stack Name: (Default: LabelDetection) This is the name that is used to refer to this stack in CloudFormation once deployed.
- RekognitionModelProjectARN: The Amazon Rekognition Model Project Arn
- RekognitionModelProjectVersionARN: The Amazon Rekognition Model Project Version Arn
When completed, click Next
-
Configure stack options if desired, then click Next.
-
On the review you screen, you must check the boxes for:
- "I acknowledge that AWS CloudFormation might create IAM resources"
- "I acknowledge that AWS CloudFormation might create IAM resources with custom names"
- "I acknowledge that AWS CloudFormation might require the following capability: CAPABILITY_AUTO_EXPAND"
These are required to allow CloudFormation to create a Role to allow access to resources needed by the stack and name the resources in a dynamic way.
-
Click Create Change Set
-
On the Change Set screen, click Execute to launch your stack.
- You may need to wait for the Execution status of the change set to become "AVAILABLE" before the "Execute" button becomes available.
-
Wait for the CloudFormation stack to launch. Completion is indicated when the "Stack status" is "CREATE_COMPLETE".
- You can monitor the stack creation progress in the "Events" tab.
-
Note the url displayed in the Outputs tab for the stack. This is used to access the application.
To test your workflow, complete the following steps:
- Upload sample images to the input S3 bucket that was created by the solution (Example: xxxx-sources3bucket-xxxx).
- Go to AWS Step Function console and select the state machine created by the solution (Example: CustomCVStateMachine-xxxx). You will see an execution triggered by the EventBridge at every hour.
- To test the solution, you can also manually start the workflow by clicking on the “Start execution” button.
- As images are processed you can go to the output S3 bucket (Example: xxxx-finals3bucket-xxxx) to see the JSON output for each image. The Final S3 bucket holds the images that have been processed along with the inferenced custom label json. As the images get processed, they will be deleted from the source bucket.
To remove the application open the AWS CloudFormation Console, click on the name of the project, right-click and select "Delete Stack". Your stack will take some time to be deleted. You can track its progress in the "Events" tab. When it is done, the status will change from "DELETE_IN_PROGRESS" to "DELETE_COMPLETE". It will then disappear from the list.
Note: Please note that the provided configuration will ensure that the Amazon S3 buckets and their contents are retained when removing the application via the AWS Cloudformation console. This is to ensure that no data is accidently lost while removing the application. The buckets can be deleted from the S3 console.
This library is licensed under the MIT-0 License. See the LICENSE file.