Clearsight augments electrical infrastructure inspection with AutoML for Images from Azure Machine Learning

Motivation

Visual inspection of electrical system distribution infrastructure has historically been a mostly manual process. These inspections are difficult to perform in certain environments and at desired levels of cost effectiveness and scale. The use of technology to aid the acquisition, storage, and analysis of data can greatly improve inspection safety, efficiency, and quality. Clearsight is finding Azure and Azure Machine Learning to be effective platforms upon which to build and enable several key technology solutions.

Prior to using Azure Machine Learning, the ML model development process was handled using a collection of different tools that were not well integrated. Building these ML models and tuning them for optimal performance took significant data scientist time and effort. It was recognized that a collaboration environment to manage datasets, train use case specific models and build test pipelines without writing custom code would enable a much more scalable approach.

Azure Machine Learning provided Clearsight with this unified platform. They made extensive use of Azure Machine Learning Data Labeling to label the images and leverage GPU compute that could be scaled as needed to develop their computer vision models. AutoML for Images allowed them to quickly experiment with multiple algorithms and find the model algorithms and hyperparameters that optimized model performance for their different use cases. With the responsible AI dashboard, Clearsight can also evaluate their computer vision models for accuracy and explainability to help humans in the loop evaluate each model’s robustness before deployment. The tight integration with MLOps allows them to manage the end-to-end ML life cycle of these models.

Approach

Clearsight uses cutting edge drone technology to support visual inspection of electrical infrastructure equipment. The drones obtain high resolution imagery from various offset distances and at multiple angles and perspectives, all while keeping the drone operators in a safe location. Once obtained, the imagery is uploaded to Azure storage. The images are then presented to subject matter experts for image inspection and observation logging via custom-built applications. Image attributes and related attributes identified through inspection activities are also recorded relationally in Azure storage.

Clearsight is leveraging Azure Machine Learning, including AutoML for Images, to build and deploy computer vision models (i.e., object and defect detectors) to enhance and support the inspection activities. These models are trained on datasets comprised of images and bounding boxes tagged with corresponding classes and labels that define objects and defects of note. Using AutoML for Image capabilities, Clearsight can easily try a variety of algorithms and hyperparameters for training these models (including Yolo, RetinaNet and Faster-RCNN), and subsequently choose the best performing one.

Benefits

New capabilities in Azure Machine Learning that add support for computer vision tasks in AutoML have proven to be beneficial to Clearsight’s use cases. The AutoML solution was evaluated in comparison to the previous process used for model development. They found the end-to-end solution using AutoML to be more efficient and generally easier to use. The resulting models were able to achieve performance similar to those from the legacy system, while driving significant efficiency in the model building process. Some specific features and updates of value to Clearsight include:

1. The ability to use both Azure Machine Learning Labeling for new data and to bring in datasets of images that were labeled outside of Azure.
2. The ability to manually select from a range of state-of-the-art algorithms.
3. The ability to access not only full models, but also access the scripts needed to automate the process and productionize the models.
4. The ability to assess models for explainability to aid human in the loop reviews.
5. The level of control and customization available in support of model training and management via Machine Learning Operations (MLOps).
   
   

These feature capabilities in Azure Machine Learning Data Labeling, the responsible AI dashboard, and AutoML for Images have significantly reduced Clearsight’s end to end cycle time required to build and deploy new computer vision models by an estimate of more than 50%.

You can find out more visit Constellation Clearsight at or by following them on Twitter and LinkedIn.