ML Model deployment in Containers: Part 1
Ready to deploy your machine learning model and can't choose from the options? Coming to the rescue is our article on ML Model deployment in Containers to help you navigate Dockers!
Taking Machine learning models to production is the ultimate aim. Making a software system available for use is called deployment. Scalability is the property of the system to handle a growing amount of work by adding resources to the system.
At Cafe I/O, we have been publishing content on various options to deploy your Machine Learning model. We have already discussed:
This is Part 1 of our series on ML Model deployment in Containers. Links to all the parts:
This post will cover:
Introduction to Containers
Virtualization vs Containerization
Docker
Conclusion
If you like our writing, do encourage us by Subscribing and Sharing
Introduction to Containers
To understand how containers came into being, we need to know more about traditional hosting which has been mentioned below:
It is important to have the same environment for both training and serving. In the other deployment methods, we have discussed how we were required to use the requirements.txt file to match the production environment with the development environment.
An alternative to the traditional approach is Container:
A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.
Containers provide isolation in the dependencies which are required to run the system.
Why use containers?
Containers offer a logical packaging mechanism in which applications can be abstracted from the environment in which they actually run. This decoupling allows container-based applications to be deployed easily and consistently, regardless of whether the target environment is a private data centre, the public cloud, or even a developer’s personal laptop.
This gives developers the ability to create predictable environments that are isolated from the rest of the applications and can be run anywhere. From an operations standpoint, apart from portability containers also give more granular control over resources giving your infrastructure improved efficiency which can result in better utilization of your compute resources.
Virtualization vs Containerization
We talked about AWS Lambda, GCP etc in our post on the ML model as Serverless Functions. These options are based on the idea of Virtual Machines. Virtual machines are an abstraction of physical hardware turning one server into many servers. It relies on the concept of Virtualization.
In the case of Virtual Machines, every user gets instances on a single physical server. It means that multiple operating systems might be running on the same hardware, exposing it to severe security risks. On the other hand, Containers provide a facility to run multiple applications of one operating system.
Containerization is the process of taking virtualization to the next level that powers multiple applications on a single Operating System which are developed and run in their own containers.
One of the biggest advantage that Containerization has over Virtualization is that everything runs on a single operating system which makes it safer and faster. It is also very shareable among the developers which is an added advantage.
With containers, you can basically do two things:
Create and run containers: Use Docker, RunC, cri-o etc
Create a system to manage the containers: Use Kubernetes, AWS Fargate, Apache Mesos, etc.
The terminology used in Container
After understanding the utility of the container, and comparing it with virtual machines, we are going to start with Docker, one of the most popular containers.
Docker
Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications.
The discussion around containers culminates here. Docker platform enables you to package and run an application in the container. As we understood before, the container environment is very secure and fast making docker lightweight and easy to run. It is very shareable across your team.
Docker is an amazing choice for deploying ML Models as it supports CI/CD (continuous integration and continuous delivery workflow) seamlessly. This is enabled by:
Supporting the developers to write code locally and share with other developers using Docker
The testing of the application can be pushed in Docker containers for automated or manual testing.
The bugs can be resolved by developers in the development environment and can be redeployed to the test environment.
Docker is highly portable making it easy to manage dynamic workload. It helps to scale up or tear down applications as the requirements. Docker is a highly resource-efficient environment, especially in small and medium deployments where resources are fewer.
Schematic diagram of Docker workflow:
Docker is based on a client-server architecture. The users or the Docker clients talk to the Docker daemon. A daemon manages the API requests and objects like images, networks etc. It does all the building, running and distribution in the Docker containers. The daemon can be run on the same system or it can be run remotely. The communication between the client and daemon happens through REST API or over UNIX sockets.
A Docker image is a read-only template that contains a set of instructions for creating a container that can run on the Docker platform. The registry displayed in the schematic diagram stores Docker images. The go-to option for a public registry is Docker Hub and Docker look for images on Docker Hub by default.
Conclusion
Containers hold the power to change how effective machine learning models are used by end-users by deploying them safely and making them fast. They also help in resource optimization. In part 1 of our series on ML model deployment in Containers, we introduced containers, compared virtualization and containerization, discussed Docker and Container orchestration.
In Part 2, we will see a hand-on deployment of an ML Model on containers and understand Kubernetes in-depth.
Do share, subscribe and support Coffee and Engineering!