Serving AI/ML Models with Microsoft Azure ML using MLFlow

By Anirban Das, Lead Azure Architect — Data & AI, anirban.das07@gmail.com

This notebook is part of a Quick Start guide based on the MLflow tutorial. The first part of the guide, MLflow Quick Start: Model Training and Logging, focuses on training a model and logging the training metrics, parameters, and model to the MLflow tracking server.

NOTE: We do not recommend using Run All because it takes several minutes to deploy and update models; models cannot be queried until they are active.

This part of the guide consists of the following sections:

Setup

• Don’t use Azure ML cluster ( use regular 5.4 cluster ( Soham to follow up why))
• Launch an Azure Databricks cluster
• Install MLflow
• Install the Azure ML SDK
• Create or load an Azure ML Workspace

Building an Azure Container Image for model deployment

• Use MLflow to build a Container Image for the trained model

Deploying the model (Container Image) to “dev” using Azure Container Instances (ACI)

• Create an ACI webservice deployment using the model’s Container Image

(Test the HTTP end point) Querying the deployed model in “dev”

• Load a sample input vector from the diabetes dataset
• Evaluate the sample input vector by sending an HTTP request

Deploying the model to production using Azure Kubernetes Service (AKS)

• Option 1: Create a new AKS cluster
• Option 2: Connect to an existing AKS cluster
• Deploy to the model’s image to the specified AKS cluster

Querying the deployed model in production

• Load a sample input vector from the wine dataset
• Evaluate the sample input vector by sending an HTTP request

Updating the production deployment

• Build an Azure Container Image for another model
• Deploy the new model’s image to the AKS cluster
• Query the updated model

Cleaning up the deployments

• Terminate the “dev” ACI webservice
• Terminate the production AKS webservice
• Remove the AKS cluster from the Azure ML Workspace

As in the first part of the Quick Start tutorial, this notebook uses ElasticNet models trained on the diabetes dataset in scikit-learn.

Setup

1. Ensure you are using or create a cluster specifying

• Databricks Runtime Version: Databricks Runtime 5.0 or above
• Python Version: Python 3

1. Install required libraries or if using Databricks Runtime 5.1 or above (but not Databricks Runtime for ML), run Cmd 6.
2. Create required libraries.

• Source PyPI and enter mlflow.
• Source PyPI and enter azureml-sdk[databricks].
• Source PyPI and enter scikit-learn==0.19.1.

1. Install the libraries into the cluster.
2. Attach this notebook to the cluster.

dbutils.library.installPyPI(“mlflow”)
dbutils.library.installPyPI(“scikit-learn”, “0.19.1”)
dbutils.library.installPyPI(“azureml-sdk”)
dbutils.library.restartPython()

Create or load an Azure ML Workspace

Before models can be deployed to Azure ML, you must create or obtain an Azure ML Workspace. The azureml.core.Workspace.create() function will load a workspace of a specified name or create one if it does not already exist. For more information about creating an Azure ML Workspace, see the Azure ML Workspace management documentation.

import azureml
from azureml.core import Workspace
import mlflow.azureml

workspace_name = “tshaikh”
workspace_location=”eastus2"
resource_group = “xxxxxxx”
subscription_id = “xxxxxxxxxxxxxxxxxxxxxxxxxx”

workspace = Workspace.create(name = workspace_name,
location = workspace_location,
resource_group = resource_group,
subscription_id = subscription_id,
exist_ok=True)

Building an Azure Container Image for model deployment

%md ### Use MLflow to build a Container Image for the trained model

Use the `mlflow.azuereml.build_image` function to build an Azure Container Image for the trained MLflow model. This function also registers the MLflow model with a specified Azure ML workspace. The resulting image can be deployed to Azure Container Instances (ACI) or Azure Kubernetes Service (AKS) for real-time serving.

Review the experiment

1. Open the experiment /Shared/experiments/DiabetesModel in the workspace.
2. Click a date to view a run.

Specify the run ID associated with an ElasticNet training run from part 1 of the Quick Start guide. You can find a run ID and model path from the experiment run, which can be found on the run details page:

Cmd 14

Python

run_id1="df174f8b3bbb40d68e440202439a3b4f"
model_path = "model"

Command took 0.01 seconds — by , 10:18:13 AM on unknown cluster

Cmd 15

Python

import mlflow.azureml
model_image, azure_model = mlflow.azureml.build_image(model_path=model_path,
workspace=workspace,
run_id=run_id1,
model_name="model",
image_name="model"
description="Sklearn ElasticNet image for predicting diabetes progression",
synchronous=False)

Registering model model Creating image

Command took 10.26 seconds — 6/5/2019, 10:18:17 AM on unknown cluster

Cmd 16

Python

model_image.wait_for_creation(show_output=True)

Deploying the model to “dev” using Azure Container Instances (ACI)

The ACI platform is the recommended environment for staging and developmental model deployments.

Create an ACI webservice deployment using the model’s Container Image

Using the Azure ML SDK, deploy the Container Image for the trained MLflow model to ACI.

from azureml.core.webservice import AciWebservice, Webservice

dev_webservice_name = “diabetes-model-1”
dev_webservice_deployment_config = AciWebservice.deploy_configuration()
dev_webservice = Webservice.deploy_from_image(name=dev_webservice_name, image=model_image, deployment_config=dev_webservice_deployment_config, workspace=workspace)

dev_webservice.wait_for_deployment()

Querying the deployed model in “dev”

Load diabetes dataset

from sklearn import datasets
diabetes = datasets.load_diabetes()

Create sample input vector

import pandas as pd
import numpy as np

X = diabetes.data
y = diabetes.target
Y = np.array([y]).transpose()
d = np.concatenate((X, Y), axis=1)
cols = [‘age’, ‘sex’, ‘bmi’, ‘bp’, ‘s1’, ‘s2’, ‘s3’, ‘s4’, ‘s5’, ‘s6’, ‘progression’]
data = pd.DataFrame(d, columns=cols)
sample = data.drop([“progression”], axis=1).iloc[[0]]

query_input = sample.to_json(orient=’split’)
query_input = eval(query_input)
query_input.pop(‘index’, None)
1
import pandas as pd
2
import numpy as np
3
​
4
X = diabetes.data
5
y = diabetes.target
6
Y = np.array([y]).transpose()
7
d = np.concatenate((X, Y), axis=1)
8
cols = [‘age’, ‘sex’, ‘bmi’, ‘bp’, ‘s1’, ‘s2’, ‘s3’, ‘s4’, ‘s5’, ‘s6’, ‘progression’]
9
data = pd.DataFrame(d, columns=cols)
10
sample = data.drop([“progression”], axis=1).iloc[[0]]
11

12
query_input = sample.to_json(orient=’split’)
13
query_input = eval(query_input)
14
query_input.pop(‘index’, None)
Out[11]: [0]

Evaluate the sample input vector by sending an HTTP request

Query the ACI webservice’s scoring endpoint by sending an HTTP POST request that contains the input vector.

import requests
import json

def query_endpoint_example(scoring_uri, inputs, service_key=None):
headers = {
“Content-Type”: “application/json”,
}
if service_key is not None:
headers[“Authorization”] = “Bearer {service_key}”.format(service_key=service_key)

print(“Sending batch prediction request with inputs: {}”.format(inputs))
response = requests.post(scoring_uri, data=json.dumps(inputs), headers=headers)
preds = json.loads(response.text)
print(“Received response: {}”.format(preds))
return preds

dev_webservice.scoring_uri

dev_prediction = query_endpoint_example(scoring_uri=dev_webservice.scoring_uri, inputs=query_input)

Deploying the model to production using Azure Kubernetes Service (AKS). Do Option 1 or Option 2.

Option 1: Create a new AKS cluster

If you do not have an active AKS cluster for model deployment, create one using the Azure ML SDK.

from azureml.core.compute import AksCompute, ComputeTarget

# Use the default configuration (you can also provide parameters to customize this)
prov_config = AksCompute.provisioning_configuration()

aks_cluster_name = “diabetes-cluster”
# Create the cluster
aks_target = ComputeTarget.create(workspace = workspace,
name = aks_cluster_name,
provisioning_configuration = prov_config)

# Wait for the create process to complete
aks_target.wait_for_completion(show_output = True)
print(aks_target.provisioning_state)
print(aks_target.provisioning_errors)

Option 2: Connect to an existing AKS cluster

If you already have an active AKS cluster running, you can add it to your Workspace using the Azure ML SDK.

from azureml.core.compute import AksCompute, ComputeTarget

# Get the resource id from https://porta..azure.com -> Find your resource group -> click on the Kubernetes service -> Properties
resource_id = “/subscriptions/<subscription-id>/resourcegroups/<resource-group>/providers/Microsoft.ContainerService/managedClusters/<aks-service-name>”

# Give the cluster a local name
cluster_name = “<cluster-name>”

# Attatch the cluster to your workgroup
aks_target = AksCompute.attach(workspace=workspace, name=cluster_name, resource_id=resource_id)

# Wait for the operation to complete
aks_target.wait_for_completion(True)
print(aks_target.provisioning_state)
print(aks_target.provisioning_errors)

Deploy to the model’s image to the specified AKS cluster

from azureml.core.webservice import Webservice, AksWebservice

# Set configuration and service name
prod_webservice_name = “diabetes-model-prod”
prod_webservice_deployment_config = AksWebservice.deploy_configuration()

# Deploy from image
prod_webservice = Webservice.deploy_from_image(workspace = workspace,
name = prod_webservice_name,
image = model_image,
deployment_config = prod_webservice_deployment_config,
deployment_target = aks_target)

# Wait for the deployment to complete
prod_webservice.wait_for_deployment(show_output = True)

Querying the deployed model in production

Evaluate the sample input vector by sending an HTTP request

Query the AKS webservice’s scoring endpoint by sending an HTTP POST request that includes the input vector. The production AKS deployment may require an authorization token (service key) for queries. Include this key in the HTTP request header.

import requests
import json
def query_endpoint_example(scoring_uri, inputs, service_key=None):
headers = {
"Content-Type": "application/json",}
if service_key is not None:
headers["Authorization"] = "Bearer {service_key}".format(service_key=service_key)print("Sending batch prediction request with inputs: {}".format(inputs))
response = requests.post(scoring_uri, data=json.dumps(inputs), headers=headers)
preds = json.loads(response.text)
print("Received response: {}".format(preds))
return preds
prod_scoring_uri = prod_webservice.scoring_uri
prod_service_key = prod_webservice.get_keys()[0] if len(prod_webservice.get_keys()) > 0 else None
prod_prediction1 = query_endpoint_example(scoring_uri=prod_scoring_uri, service_key=prod_service_key, inputs=query_input)

Updating the production deployment

Build an Azure Container Image for the new model

run_id2=”<run-id2>”

import mlflow.azureml

model_image_updated, azure_model_updated = mlflow.azureml.build_image(model_path=”model”,
workspace=workspace,
run_id=run_id2,
model_name=”model-updated”,
image_name=”model-updated”,
description=”Sklearn ElasticNet image for predicting diabetes progression”,
synchronous=False)

model_image_updated.wait_for_creation(show_output=True)

Deploy the new model’s image to the AKS cluster

Using the [`azureml.core.webservice.AksWebservice.update()`](https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.webservice.akswebservice?view=azure-ml-py#update) function, replace the deployment’s existing model image with the new model image.

prod_webservice.update(image=model_image_updated)
prod_webservice.wait_for_deployment(show_output = True)

Query the updated model

prod_prediction2 = query_endpoint_example(scoring_uri=prod_scoring_uri, service_key=prod_service_key, inputs=query_input)

Compare the predictions

print(“Run ID: {} Prediction: {}”.format(run_id1, prod_prediction1))
print(“Run ID: {} Prediction: {}”.format(run_id2, prod_prediction2))

Cleaning up the deployments

Terminate the “dev” ACI webservice

Because ACI manages compute resources on your behalf, deleting the “dev” ACI webservice will remove all resources associated with the “dev” model deployment

dev_webservice

Terminate the production AKS webservice

This terminates the real-time serving webservice running on the specified AKS cluster. It **does not** terminate the AKS cluster.

prod_webservice.delete()

Remove the AKS cluster from the Azure ML Workspace

If the cluster was created using the Azure ML SDK (see Option 1: Create a new AKS cluster), remove it from the Azure ML Workspace will terminate the cluster, including all of its compute resources and deployments.

If the cluster was created independently (see Option 2: Connect to an existing AKS cluster), it will remain active after removal from the Azure ML Workspace.

aks_target.delete()

By Anirban Das, Lead Architect — Azure Cloud- Data & AI, Kimberly Clark

Reach — anirban.das07@gmail.com