An AI view of the construction site: GOLDBECK and HoloBuilder detect defects using automated documentation with Microsoft Azure

The challenge: Developing a standardized, intelligent system for gathering and analyzing information

It takes all kinds of skills and trades to construct a building. Machines, of course, also play an important role, but considering how physical and tangible buildings are, it is difficult to imagine there being much to digitalize. Yet digitalization is advancing in this sector, too. GOLDBECK is an innovator within the building industry. Founded in Bielefeld in 1969, the family-owned company systematized the construction of commercial properties by moving the production of structural elements from the building site to its own plants. Having been industrialized already, construction is now set to go digital: the company’s more than 8,500 employees across Europe enjoy the benefits of digital solutions not just in the office, but also on the construction site. Here, it is the site managers that run the show: “They arrange the schedules for the various tasks and trades, and they coordinate countless tradespeople and laborers—always with an eye to cost and quality,” says Max Schütz, Director of the Building Information Modeling Department and Managing Director at GOLDBECK US. “The site manager always has to work on the construction site as they are also responsible for documenting the progress and quality of the work.” Digital technology has, of course, long played a role here, with traditional reports written in Word, Excel, or OneNote and supplemented with photos. 

“At GOLDBECK, we’ve systematized a large proportion of the project process; we use standardized forms for reports and we back up project data centrally in the cloud,” explains Sina Schliestedt, who started out at GOLDBECK as a site manager before switching to her current role of building information modeling specialist. “However, a site manager’s day can fill up so quickly that they often find themselves with little time for documenting the construction site as well as they should. For instance, if they fail to follow a naming convention to the letter when filing a report manually, documents will be more difficult to evaluate further down the line. This can slow down the building process and even lead to subsequent errors.” GOLDBECK needed a way of strengthening this area. And it found exactly that by drawing on 360° cameras and artificial intelligence.

The solution: Artificial intelligence analyzes and learns from photos taken on‑site

Today, GOLDBECK takes a simpler and, above all, more digital approach to its work. The company’s site managers use the HoloBuilder platform to track and document construction work with precision. The solution links 360° photos of the construction site to the digital building plans, sorts them by location and time, and stores them securely in the Microsoft Azure cloud. If there are any discrepancies between the actual and planned construction progress, the site managers can immediately identify and document defects. Contractors and tradespeople can be notified about any damage to plastering or any water damage right away. Additional or corrective work can be arranged as early as possible, preventing knock-on effects and avoiding unnecessary costs or delays to construction.

“Until now, site management had always been rather ‘hands-on’,” Schütz says. “We took photos of the site, uploaded them to the computer, evaluated them, and then summarized our evaluation in the construction progress report⁠—a slow and time‑consuming process.” HoloBuilder simplifies all this: A 360° camera is used to take immersive photos of a building. If the site manager is not around to do this in person, they can call on Spot—the four-legged, autonomously operated robot developed by Boston Dynamics. The photos are then uploaded to the HoloBuilder platform and can be accessed together with the building plans. “The 360° photos really elevate the quality of the documentation. Added to this, the solution acts as a time machine, taking users on a journey through the construction’s entire development. This makes it easy to trace defects back to their origins—to determine the cause of water penetration, for example,” Schütz says enthusiastically.

Azure Machine Learning continuously improves error detection 

The HoloBuilder solution relies on Microsoft Azure. Not content with simply saving the images and integrating them into the building plans, the developers also utilize Azure Machine Learning: “HoloBuilder can independently analyze the captured 360° photos and gain an understanding of both the construction site and construction progress,” says Mostafa Akbari-Hochberg, President & Co-Founder. “We’ve trained our software in Azure not only to differentiate between a roof and the sky on a building site, but also to recognize and distinguish between building components and materials—for example stairs, cement walls, and drywall partitions.” The information garnered from the images makes it easier to compare actual progress with the building plans: a Power BI dashboard that places construction progress alongside the plans allows the site manager to recognize discrepancies in an instant. As everything is also integrated into Microsoft Teams, the site manager can alert colleagues and business partners to inconsistencies in construction right away.

“HoloBuilder facilitates collaboration even in times of a pandemic, as it lets us take contractors on extremely realistic, virtual tours of the construction site,” Schütz says. “Our customers’ feedback has been wholly positive. They’re able to walk the site from the comfort of their own home—HoloBuilder provides real added value here.” The solution has also been well received by the site managers at GOLDBECK. So much so that after it was first trialed, the platform was incorporated into the basic IT and service package at GOLDBECK in February. Since then, HoloBuilder has already been used on around 50 GOLDBECK building sites.

“At GOLDBECK, we truly believe that digitalization, robotics, and artificial intelligence can have a disruptive effect within the construction industry, too,” Schütz asserts. HoloBuilder unites all three aspects: Construction progress is captured digitally in multiple dimensions and analyzed using artificial intelligence. In the future, the four‑legged robot will be used to move around the building site autonomously, taking photos in place of the site manager. Artificial intelligence will then come into play not only to appraise construction progress but also to manage the robot.