#2.3 [MUST] Inference for MLOps Pipeline

Introduction

It’s crucial to fully understand the foundations before utilizing the MLOps three-stage pipeline. If you haven’t already, I suggest the first article (introduction) to catch up.

Also, if you haven’t already setup the Data Ingestion pipeline, look at Article #2.1 for the tutorial.

If you forgot to build model for the MLOps pipeline, feel free to read our Article #2.2 for the steps.

Our model has been baked and is ready to output some results for us. From this tutorial, you will learn how to load the trained model and use it for inference.

This time, create a python file named model_inference.py and boot it up in your editor. This will be a short and easy to use file.

Loading the Model

Import the required files using:

import torch
import os
from PIL import Image

Remember the data_transform() operation from #2.2. We will be using it again, but in this case, we will load the image, transform it and then send it to model for inferencing. Here it is:

def data_tranforms():
    """
    Apply transformations on the dataset
    Args:
        None
    Returns:
        torchvision.transforms.Compose: Composed transformations
    """
    transformation = transforms.Compose([
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
    return transformation

For loading the model in PyTorch, we use torch.load() and also load its checkpoint saved in #2.2. Also, get the image using Path module. Make sure to initialize the device as well.

model = torch.load('best_model.pth')
chk = torch.load('best_checkpoint.pth.tar')
file = Path("path/to/image")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

Now, to inference from this loaded model, we need a few things first:

classes = chk['classes']    # get the classes List from the Checkpoint

image = Image.open(file)    # Open the image using path   
image = data_tranforms(image)        # Transform the image
image = image.unsqueeze(0).to(device)

Using the loaded class values and transformed image, inference the model using eval mode.

model.eval()
output = model(image)
_, pred = torch.max(output.data, 1)

print("Predicted Output is: ", classes[pred.item()])

Here is a small explanation of this code snippet:

• model.eval() — Sets the model in evaluation mode.
• output — is used to store the results for each class.
• _, pred = torch.max(output.data, 1) — Get the maximum value from all of the output classes. It returns two things — the maximum value (_) and the index of the maximum value (pred).
• classes[pred.item()] — pred.item() converts the tensor containing the predicted class index into a Python integer. classes[pred.item()] accesses the corresponding class label from a list of classes.

Now, you have successfully printed the predicted output for the given image.

Feel free to put this all code under a function to increase its modularity:

import torch
import os
from PIL import Image
from torchvision import transforms
from pathlib import Path

def data_tranforms():
    """
    Apply transformations on the dataset
    Args:
        None
    Returns:
        torchvision.transforms.Compose: Composed transformations
    """
    transformation = transforms.Compose([
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
    ])
    return transformation


def model_infer():
  model = torch.load('best_model.pth')
  chk = torch.load('best_checkpoint.pth.tar')
  file = Path("path/to/image")
  device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

  classes = chk['classes']    # get the classes List from the Checkpoint
  
  image = Image.open(file)    # Open the image using path   
  image = data_tranforms(image)        # Transform the image
  image = image.unsqueeze(0).to(device)

  model.eval()
  output = model(image)
  _, pred = torch.max(output.data, 1)
  
  print("Predicted Output is: ", classes[pred.item()])

So finally, you have inferenced a local image, which is possibly out of the dataset. Feel free to modify your code according to your needs.

Conclusion

Once you have written the above code and executed it, you will get a output on your console/terminal.

Feel free to use this code as reference for the model Inference.

That wraps up this tutorial! In the next one, we’ll combine the 3 pipeline files we have created into a single code in a clever way and try to run that. Also, I will share some tips on improving the pipelines.

Thanks for following along 👏👏. Best of luck as you continue building your MLOps pipeline!