Use Azure Custom Vision for Factory Visitor PPE check

This tutorial will guide you through the steps of setting up a visitor management system that uses Azure Custom Vision service to check if visitors are wearing the required personal protective equipment (PPE) in a factory environment. You will learn how to:

• Create a Custom Vision project and train it with images of people wearing or not wearing yellow hard hats.
• Deploy the model and use it within Power Apps based Visitor Check In app to capture visitors’ photos during the check in process.

What you will do

• Build a vision AI model with Azure Custom Vision service that detects if a person is wearing a yellow hard hat for workplace safety applications.
• Build a Power Apps Canvas app to capture visitor photo and check PPE helm equipped use the Azure Custom Vision.

What you will need

• Active Azure account (Create a free account here)
• Power Platform environment (Create and manage environments in the Power Platform admin center).
• To use the Custom Vision service, you will need to create Custom Vision Training and Prediction resources. This is done when creating a new project in customvision.ai. In case there are issues in setting up a new project, please try creating the resource manually in Azure portal.
• Yellow hard hat to test the model.

Create a vision AI model with Azure Custom Vision service

In this tutorial, you will build a vision AI model for a workplace safety scenario that detects if a person is wearing a hard hat or not. Don’t you have a yellow hard hat to be able to test the functionality after you are done? You can also take pictures and use your own data to build the Vision AI model. Take 30+ pictures for a selected object from different angles and distances instead of using the pictures for hard hat model. You can also use the camera to take pictures (you’ll need to install Python and GitHub).

Create a vision AI model with Azure Custom Vision service

There are two ways to train your model - image classification and object detection.

• Image classification: Analyzes the whole frame as a picture and doesn’t draw bounding boxes. Can only identify one object per frame. Easy to train.
• Object detection: Can identify multiple objects per image and draw bounding boxes around them. It takes a little bit more time to train as the training requires identifying the object per uploaded image. If you choose to use the object detection model with hard hat data, remember to include both the wearer’s head and hard hat as the object when applicable (not only the hat).

Steps

• Create new Azure Custom Vision resources.
• Setup a new Azure Custom Vision project.
• Create new connection for your Power Apps environment.
• Create Power Apps to test the custom vision model.