Deployment of Containerized Machine Learning Model Application on AWS Elastic Container Service(ECS)
Machine learning engineer has to build, train and also deploy the machine learning model using the data that has been provided to him so that end users around the world can use the trained model to make predictions. My objective here is to build a model using Logistic Regression algorithm to predict whether the patient is diagnosed as Benign or Malignant for Breast Cancer examination, convert this trained model into a web application, containerize and deploy it on Amazon Elastic Container Service(ECS).
In this blog, I will explain the below
- Process of training the machine learning model on the local machine, and saving the model as a pickle file so that we need not train the model every time we do predictions on it.
- Converting the trained model as web application using Flask web framework
- Uploading the folder to GitHub, building and pushing Docker Image to AWS Elastic Container Registry (ECR).
- Finally deploying the Containerized web app on Amazon Elastic Container Service(ECS).
1) Building, Training and Testing Machine Leaning Model
The process of building, training and testing the Logistic Regression model with Wisconsin Breast Cancer data set has already been explained in my previous blog Logistic Regression — Breast Cancer Prediction(link provided below) to understand this step better.The prediction is based on 30 variables present in the data set
2)Pickle file
The model after being properly trained will be ready to do predictions on the unseen data. To do predictions multiple times we need to run entire training process multiple times which is cumbersome. If we can convert the trained model and save it as pickle file, we don’t have to go through the training process each time to predict on new data parameters. This is done by Pickle, a python package that serializes the python object into byte streams and can be stored on the local disk. Here model.pkl is the pickle file of the trained model
3) Flask API
Next step in the process, will be to build a web application for predicting the breast cancer.To create a web application, we need to have a framework that creates both web apps and APIs.One of the popular web framework for Python is Flask.We will use Flask to build our web application.
First step in building the API will be to load the pickle file that has been saved in the previous step
Next import Flask and request methods from the flask library, next instantiate a Flask object named app.
Create a function called predict_cancer that takes input from a web browser to do predictions.The code is given below
The last step will be to specify the host and port on which the app should run.
All the previous code blocks were put together and named as app.py
Finally, we can do predictions on the web app from any web browser by copying the URL shown below. The link is longer, as it contains 30 variables
The result for above URL with the given data will be ‘0’, 0 means the patient is diagnosed as Benign for breast cancer.
Predicted result for observation [[-0.96666522 0.32786912 -0.93579507 -0.91104225 0.60962671 0.36569592 -0.10914833 -0.62181482 -0.63860111 0.53651178 -0.46379509 0.5132434 -0.45632075 -0.59189989 0.67370318 1.26928541 2.17185315 1.12535098 0.64821758 1.09244461 -0.96440581 -0.08750638 -0.94145109 -0.84547739 -0.07511418 -0.01862761 -0.10400188 -0.47718048 -0.5634723 0.05526303]] is: [‘0’]
A screen shot is shown below
4) Docker Containerization
Now the machine learning model web application can run on the local machine. The web app which runs on local machine may not run on another machine because of the following reasons
1) The OS and environment variables compatibility issues.
2) Python and its libraries such as Sci-kit learn, Pandas, Numpy, Matplotlib, Seaborn etc may not be installed on it. Even if they are installed, they could be different versions.
All these platform dependency problems can be solved by containerization. What is a Container ?
A container is a software that packages up code and all its dependencies, so the application runs quickly and reliably from one machine to another.
How can these containers be created ?
These are created by Docker software.
What is a Docker ?
A Docker is a tool designed to simplify the steps — create, deploy, and run applications on containers.
My working directory is my_docker. It contains the machine learning model file, pickle file model.pkl, text file requirements.txt and a docker file Dockerfile
requirements.txt
Contains all the python packages along with versions required to run the model
Dockerfile
Each command in the Dockerfile is explained as follows:
FROM python:3.7 :The base docker image is Python,we will build our image on the top of this image.
COPY ./requirements.txt /docker_app/requirements.txt:
The COPY command copies requirement.txt file on the current folder to a folder called docker_app within the image we are trying to build.
WORKDIR /docker_app: WORKDIR command changes the working directory to docker_app within the image.
RUN pip install — upgrade pip: installs and upgrade pip package
RUN pip install -r requirements.txt: installs all the packages listed in requirement text file.
COPY . /docker_app: copy everything else into image.
EXPOSE 5000: will expose port 5000 to the outside for accessing web app
CMD [“app.py”]: will start the app.py application whenever the container boots up.
4)Uploading folder to GitHub
my_docker folder containing all the above files has been uploaded to GitHub. One can access them through the link below
Next step is to create a Docker Image and push it to Amazon ECR.
5) Creating a Docker Image and push it to Amazon ECR
In this section, I will walk you through the steps in building, tagging and pushing a Docker Image into Amazon ECR.
What is Amazon ECR ?
Amazon Elastic Container Registry (ECR) is a fully managed container registry that makes it easy to store, manage, share, and deploy your container images and artifacts anywhere. Amazon ECR eliminates the need to operate your own container repositories or worry about scaling the underlying infrastructure. Amazon ECR hosts your images in a highly available and high-performance architecture, allowing you to reliably deploy images for your container applications.
Log into your AWS Console and open the Amazon ECR service.
Create a new repository cancer-repository
cancer-repository will be created with no Docker Image in it
The following commands are available once you open the view push commands tab
you can build, tag and push the docker images into Amazon ECR repository in many ways. We have chosen to use EC2 Instance. Create an EC2 instance, SSH to the instance, install all the packages and start building the Docker Image.
All the commands that are used to install packages, clone repository from GitHub, next give authentication to Docker to use Amazon ECR, next build, tag and push the Docker Image to the Amazon ECR repository as shown below
sudo yum install -y awscli # Install awsclisudo yum install -y git # Install gitsudo yum install -y docker # Install dockersudo service docker start # start dockersudo git clone https://github.com/spratapa/my_docker.git # get the my_docker file from gitaws ecr get-login-password --region ap-southeast-2 | docker login --username AWS --password-stdin <AWS Account Number>.dkr.ecr.ap-southeast-2.amazonaws.com # authenticate docker with Amazon ECRdocker build -t cancer-proj . #Build docker Image with name cancer-projdocker tag cancer-proj:latest <AWS Account Number>.dkr.ecr.ap-southeast-2.amazonaws.com/cancer-repository:latestdocker push <AWS Account Number>.dkr.ecr.ap-southeast-2.amazonaws.com/cancer-repository:latest # push the Image to Amazon ECR Repository
The Docker Image with Image tag latest is now available for use in the cancer-repository
Now we can create and deploy containers with the Docker Image present in the Amazon ECR. The number of containers to be deployed depends on the number of the end users across the globe. There are many container orchestration tools available to do this job for us. Kubernetes, Amazon ECS, Amazon EKS are some of those that are widely used. For this application, we used Amazon ECS service for deployment and management of Docker containers.
6) Deploying CancerApp on Amazon ECS
Before going through the deployment process using Amazon ECS, let us know What is Amazon ECS ?
Amazon ECS is a fully managed container orchestration service. Amazon ECS supports Docker and enables you to run and manage Docker containers. Applications you package as a container locally will deploy and run on Amazon ECS without the need for any configuration changes.
The following are steps involved in this section
a) Creating Amazon ECS Cluster
Cluster is a logical grouping of tasks or services.Your tasks and services are run on infrastructure that is registered to a cluster. The Infrastructure that is chosen for the cluster of this web app is EC2 instance.
The steps involved in creation of a Cluster named CancerAppCluster is shown below
Selecting EC2 Instance as Launch type for the Cluster
we have selected t2.micro as the EC2 Instance type for application as it is part of the free tier
Select default VPC, Subnets, Security Group and Enable Auto assign public IP
Finally, a Cluster CancerAppCancer has been created with EC2 Instance as launch type.
Next step in the process is to create a task definition. Let us see what is Task definition ?
b) Creating a new Task definition
Amazon ECS allows you to define tasks through a declarative JSON template called a Task Definition. Within a Task Definition you can specify one or more containers that are required for your task, including the Docker repository and image, memory and CPU requirements, shared data volumes, and how the containers are linked to each other. You can launch as many tasks as you want from a single Task Definition file that you can register with the service. Task Definition files also allow you to have version control over your application specification.
The followings are the steps involved in creating a new Task definitions
click create a new Task definition
select EC2 Instance type
Name the Task Definition as CancerAppTaskDef
Create a container CancerAppContainer by pulling the Docker Image ECR. Specify the host and container ports.
c) Running Tasks
In this step, we will run the Tasks that have been configured in Task definitions of the Cluster.
click Run new Task and specify Task Definition and Cluster names
Task is running now
Final step in the whole process is to test the app with unseen data.
6) Testing the CancerApp
Now that my CancerApp is running on Amazon ECS, it is time to do prediction from the app with unseen variables to know whether the patient is diagnosed as Benign or Malignant.
Open the EC2 Instance that is running the app, copy the public IPv4 address and open it on port 8888 to predict . The public address is 54.253.8.196.
Copy the URL given below in any web browser to see the result of the prediction.
The CancerApp is not accessible from the above public address and port 8888 which is shown below.
Why is my app not running ?
To troubleshoot this issue
Open the security group attached to the EC2 Instance , open the Edit inbound rules and we can see port 80 which is accessible from the internet, but we hosted the app on port 8888.
So, we should add Port 8888 and make it accessible to all internet users
Hurray! finally my CancerApp is in operation. Now when we copy the URL in the browser, it predicted the result as ‘0’ which means a patient is diagnosed as Benign for breast cancer.
