Automation in ML Ops Platform

Industry: Retail & CPG

Problem Context

The client is an UK consumer goods company that primarily manufactures fashion accessories such as watches, jewelry, and eyewear. They build Machine Learning models on local systems that need to be deployed and configured manually, making the entire process time-consuming and the system unscalable.

Challenges

Automating using Airflow by leveraging parameter store for around 100 models in production. Building a generic template to accommodate over 100 models of different frameworks. Rigorous testing of each model’s end-to-end pipeline in development environment before deploying in production environment.

Technologies Used

Amazon EC2

AWS Lambda

Amazon ECR

Amazon SageMaker

AWS Systems Manager

Amazon RDS

Amazon S3

AWS CodeCommit

Apache Airflow

Amazon Athena

Amazon Glue

Amazon QuickSight

Amazon API Gateway

Automating the deployment of over 100 models in client’s environment by leveraging Airflow DAG Scripts

Solution

S&M Tech Solutions is helping the client in the orchestration of training, evaluation, and deployment of over 100 models running on SageMaker by leveraging Apache Airflow. Once each model is deployed, the Production Model Lifecycle is such that the scoring, evaluation, and retraining of the model is automated. S&M Tech Solutions is also helping the client in the visualization of the performance of the deployed models using Amazon QuickSight and Amazon Athena.

Result

 Reducing the manual efforts of for set-up and configuration

 Generalized templates for deployments with ease

 Smart insights and Visualization on the performance of models