- This write-up will only focus on AWS Sagemaker & AWS Greengrass Services with respect to ML model deployment to Edge devices.
- We will explore different ways to deploy the ML model to edge devices using Greengrass v1 & v2 and also using Sagemaker Edge Manager Agent.
ML Inference on Greengrass devices
- ML models can be trained using AWS Sagemaker or custom ML trainining ways like KubeFlow. Models are stored in AWS S3 for deployment to Greengrass devices.
- ML models are deployed as artifacts in your components to perform inference on your core devices.
ML Components
- AWS provides following Machine Learning components that can be deployed to edge devices to perform Machine Learning Inference.
- ML models can be trained using AWS Sagemaker or custom ML trainining ways like KubeFlow. Models are stored in AWS S3 for deployment to Greengrass devices.
- AWS-provided machine learning components are broadly categorized as follows:
- Model component — Contains machine learning models as Greengrass artifacts.
- Runtime component — Contains the script that installs the machine learning framework and its dependencies on the Greengrass core device.
- Inference component — Contains the inference code and includes component dependencies to install the machine learning framework and download pre-trained machine learning models.
- To perform custom machine learning inference with your own models that are stored in Amazon S3, or to use a different machine learning framework, you can use the recipes of the following public components as templates to create custom machine learning components.
- Further Reference:
- AWS provided ML model components:
- SageMaker Edge Manager
- DLR image classification
- DLR object detection
- DLR image classification model store
- DLR object detection model store
- DLR installer
- TensorFlow Lite image classification
- TensorFlow Lite object detection
- TensorFlow Lite image classification model store
- TensorFlow Lite object detection model store
- TensorFlow Lite installer
AWS SageMaker Edge Manager agent
- With SageMaker Edge Manager, you can use Amazon SageMaker Neo-compiled models directly on your core device.
- AWS SageMaker Edge Manager
Perform machine learning inference with Greengrass v1
- Supported ML model sources:
- AWS Sagemaker
- AWS S3
- Requires AWS Greengrass v1.6+
- Following ML runtimes and libraries are supported with AWS IoT Greengrass v1:
- Amazon SageMaker Neo deep learning runtime
- Apache MXNet
- TensorFlow
- Access machine learning resources from Lambda functions
- User-defined Lambda functions can access machine learning resources to run local inference on the AWS IoT Greengrass core.
- Checkout Access permissions required for machine learning resources
- Configure machine learning inference using the AWS Management Console
- Configure optimized machine learning inference using the AWS Management Console
- You can use the SageMaker Neo deep learning compiler to optimize the prediction efficiency of native machine learning inference models in Tensorflow, Apache MXNet, PyTorch, ONNX, and XGBoost frameworks for a smaller footprint and faster performance.
Perform machine learning inference with Greengrass v2
- Greengrass v2: Tutorial: Perform sample image classification inference using TensorFlow Lite
- This tutorial shows you how to use the TensorFlow Lite image classification inference component to perform sample image classification inference on a Greengrass core device.
- Greengrass v2: Perform sample image classification inference on images from a camera using TensorFlow Lite
- This tutorial shows you how to use the TensorFlow Lite image classification inference component to perform sample image classification inference on images from a camera locally on a Greengrass core device.