Bentley Systems creates breakthrough framework, drastically speeds up AI development with Azure Machine Learning

Federating critical infrastructure data

Aging physical infrastructure is a concern in many places. From roads and bridges to buildings and utility networks, these assets can take a lot of time, energy, and money to maintain. Infrastructure owners and operators need to understand and manage an asset’s condition to make necessary updates, but that information comes from many different, often paper-based sources.

Bentley Systems is there to help. The software and cloud services company provides much-needed solutions for the infrastructure markets, including the organizations that design, build, and operate the assets. Bentley offers products that federate and transform data from multiple sources so it can be used digitally. “Much of the data about the world’s infrastructure today is what we call ‘dark data,’” says Jerry King, Vice President of Strategic Alliances at Bentley Systems. “The data is locked up in opaque documents such as PDFs and drawings that computers can’t read.”

The company applies machine learning to unlock this dark data and tie it together from multiple sources. Bentley’s iTwin platform consumes data from those sources to generate a digital twin. But to create the powerful machine learning algorithms, the company’s data scientists need time to focus and freedom to experiment with code. They often write code locally with their in-house graphics processing unit (GPU) servers and then work with a machine learning operations (MLOps) engineer to move that code to Microsoft Azure. As a longtime Azure customer, Bentley turned to Microsoft to explore a way to allow data scientists to continue developing machine learning models on servers where they’re comfortable working, while automating the movement of code to the cloud to easily share data.

Creating a breakthrough framework

Bentley wanted to build a framework that developers could use to more quickly get new models and AI projects into the hands of other colleagues. The company created an MLOps framework using Azure Machine Learning that it could apply to many of its products, including the iTwin platform. “When we looked for an MLOps solution methodology, we did consider some open-source software, but Azure Machine Learning already had the infrastructure that we needed,” says Tom Chmielenski, Principal MLOps Engineer at Bentley Systems. “We liked the option of working directly with Microsoft and the Azure development teams to make sure that we were taking full advantage of Azure Machine Learning to optimize our workflows.”

As Bentley was building its framework, Microsoft announced PyTorch Enterprise support to provide long-term commercial support for the PyTorch open-source machine learning framework. The company decided to use Azure Machine Learning in combination with PyTorch, and it can use its existing Microsoft Premier Support contract to get PyTorch help if needed. Bentley gained a machine learning library of prebuilt models in PyTorch that data scientists use to jumpstart their experimentation process. “Even though PyTorch is open source, it’s like a first-class Azure service,” says Chmielenski. “We get the same level of support for PyTorch as we would for any Azure service, which creates a win-win situation for Microsoft and customers like us.”

Now, Bentley sets up the Azure Machine Learning workspace at the beginning of a project. Then, through the MLOps framework, data scientists continue their normal workflows on-premises, but their early experimentation gets logged automatically in Azure. Additionally, after an AI model is in production, data scientists easily return to the on-premises environment to experiment with new models before pushing them back to Azure. “Our MLOps engineers can easily transfer performant models to Azure to train on larger datasets or faster GPUs thanks to Azure,” says Chmielenski. “We use Azure Machine Learning and PyTorch in our new framework to develop and move AI models into production faster, in a repeatable process that allows data scientists to work both on-premises and in Azure.”

Transforming the developer experience

Bentley used configuration files that allow developers to run their code locally while automatically generating a cloud-based version of the code in Azure. Bentley built its framework by starting with two AI models in development and incorporated best practices. The company interviewed its data scientists along the way to get feedback on how the process was working and then created an environment for them to work locally and easily shift the code to the cloud. Since using those initial models to create the framework, Bentley has used it to quickly get six additional models into production, and developers easily tweak the framework as necessary when they develop new models.

The data scientists continue their work as usual and code gets automatically moved to Azure, so the lift is light for the developers. “We wanted our data scientists to feel comfortable using their local workstations while we seamlessly create an Azure Machine Learning workspace as soon as a project is started,” says Chmielenski. “That way, we connect them to Azure early in the process, but they can focus on their day-to-day work.”

In Bentley’s employee surveys, developers’ satisfaction levels have gone up. “Before, we were in the early stages with MLOps, so we didn’t have the structure to ensure proper management of all the machine learning source code,” says King. “By creating this framework with Azure and PyTorch, we’re making the work that our data scientists do around machine learning more systematized and tightly managed.”

Adds Chmielenski, “The data scientists and our release services team are happier that we’re using a consistent and reproducible environment.”

Drastically decreased development time

Introducing the Azure-based framework helped Bentley improve overall project operations because using it boosts visibility for other employees, such as the company’s MLOps team. In the past, employees had to hold a lot of one-off conversations to ask a specific colleague about a project’s progress and code status. “Now the code is just all there, and our teams know we have a repeatable process to produce machine learning models reliably without chasing down individual developers for updates on their piece of code, which saves a lot of time on everyone’s part,” says King.

That, combined with the ease of work for the developers, makes everything run faster. “After models are up in the cloud, we can scale our training with larger datasets because we’re using Azure and shorten the time to get models into production,” says Chmielenski. “We have more compute power at our fingertips, so we can compare more models than if we were keeping code on one developer’s computer. Instead of taking five days to train, we can now train in one day and do five models within the same time frame. That’s the kind of speed and scalability we’ve gained through Azure Machine Learning and PyTorch.”

Find out more about Bentley Systems on Twitter, Facebook, YouTube, and LinkedIn.