AWS IoT Greengrass & AWS Sagemaker for ML deployment & inference on Edge devices

 
  • This write-up will only focus on AWS Sagemaker & AWS Greengrass Services with respect to ML model deployment to Edge devices.
  • We will explore different ways to deploy the ML model to edge devices using Greengrass v1 & v2 and also using Sagemaker Edge Manager Agent.

ML Inference on Greengrass devices

  • ML models can be trained using AWS Sagemaker or custom ML trainining ways like KubeFlow. Models are stored in AWS S3 for deployment to Greengrass devices.
  • ML models are deployed as artifacts in your components to perform inference on your core devices.

ML Components

  • AWS provides following Machine Learning components that can be deployed to edge devices to perform Machine Learning Inference.
  • ML models can be trained using AWS Sagemaker or custom ML trainining ways like KubeFlow. Models are stored in AWS S3 for deployment to Greengrass devices.
  • AWS-provided machine learning components are broadly categorized as follows:
    • Model component — Contains machine learning models as Greengrass artifacts.
    • Runtime component — Contains the script that installs the machine learning framework and its dependencies on the Greengrass core device.
    • Inference component — Contains the inference code and includes component dependencies to install the machine learning framework and download pre-trained machine learning models.
  • To perform custom machine learning inference with your own models that are stored in Amazon S3, or to use a different machine learning framework, you can use the recipes of the following public components as templates to create custom machine learning components.
  • AWS provided ML model components:
    • SageMaker Edge Manager
    • DLR image classification
    • DLR object detection
    • DLR image classification model store
    • DLR object detection model store
    • DLR installer
    • TensorFlow Lite image classification
    • TensorFlow Lite object detection
    • TensorFlow Lite image classification model store
    • TensorFlow Lite object detection model store
    • TensorFlow Lite installer

AWS SageMaker Edge Manager agent

  • With SageMaker Edge Manager, you can use Amazon SageMaker Neo-compiled models directly on your core device.
  • AWS SageMaker Edge Manager

Perform machine learning inference with Greengrass v1

Perform machine learning inference with Greengrass v2

Further References