A secure and compliant version of ChatGPT: Otto Group builds its own in-house AI solution using Azure OpenAI

The challenge: Efficient knowledge management for 41,000 employees

For Otto Group, it all began with 300 copies of a 14-page, hand-bound catalog offering a range of 28 pairs of shoes. This first OTTO catalog appeared in 1950, just one year after Werner Otto founded the company in Hamburg, Germany. Taking as its motto “Trust with Trust,” OTTO was Germany’s first mail-order company to introduce payment by invoice, and its catalog became a runaway success. Over the more than 70 years since then, Otto Group has evolved into a retailer and service provider that employs over 41,000 people worldwide. The company is active in the fields of retail, finance, logistics, and e-commerce—and on a scale that makes efficient knowledge management a Herculean task. 

The harder a company makes it for employees to access information and utilize know-how, the slower and less efficient its processes will be. Single-handedly sifting through endless FAQ documents to find information is just one example of how time gets eaten up. Wouldn’t it be much more efficient to have someone who could always be on hand to answer questions on a topic quickly and easily?

“The technology required for such a solution suddenly appeared in the form of ChatGPT. What we had to do was to make it work for us in terms of the legal framework governing data protection and information security,” says Florian Leuerer, Expert Machine Learning & AI at Otto Group IT. “The challenge was bringing together all the different players involved: the legal and cloud compliance departments as well as the works council,” adds Anja Körber, Head of Artificial Intelligence & Automation at Otto Group IT. “With Microsoft’s help, we resolved all the issues relating to this topic and built a secure and compliant solution.” This completed one of the most complicated steps in the implementation of ogGPT. The solution is based on Azure OpenAI Service, which also provides the large language model (LLM).

The solution: ogGPT provides quick answers while meeting all legal and data protection requirements

As ogGPT is built on Azure OpenAI Service, users can upload documents and knowledge databases. “When an employee asks ogGPT a question, the tool relays this query to Microsoft’s OpenAI Service and the AI can mine these knowledge databases as needed to provide an answer,” Leuerer explains. And thanks to Microsoft Entra ID, user administration is also managed in the cloud. “An AI is complex, but implementing and using one in Azure is really simple. What’s more, we can switch to a newer model at the drop of a hat. That way, we make sure that our data doesn’t end up being used as training data in OpenAI models. And as the data is stored in Europe, we’re on the safe side with respect to GDPR compliance,” he says.

At present, 26,000 users have access to ogGPT and the next goal is to add a further 10,000. The AI makes it possible for individuals and the Group at large to tap knowledge and information faster and more easily. “For instance, it’s a vehicle for Group IT to present FAQs about IT services. Instead of users first having to rack their brains to find the right keyword and then comb through the document, they can now effectively talk to the document and ask it questions. As the AI responds right away, the entire process is quicker and easier,” Körber says.

Feedback on ogGPT ranges from merely positive to heartily enthusiastic. The AI is a creative buddy for coming up with concepts, producing test data, and generating images. It helps search for information—summarizing it on request—and links it to other content. It even cites the sources where it has culled the knowledge from. “ogGPT makes it much easier for me to do my job. Without AI, it would take me up to three hours to summarize a 20-page PowerPoint presentation. Now it’s much easier and I’m finished in a half hour, and that includes the translation into English,” says Jessica de Lafuente Torres, Project Lead Internal Communication & IT Marketing at Otto Group IT. “People all over the company are working on use cases and the solution’s potential is huge.”

By providing workshops and materials for employees to learn on their own, the company is training new and experienced users, and showing them tips and tricks that make using the tool simple and in compliance with all relevant regulations. “The tool is essentially very intuitive, but it does require a bit of practice,” Leuerer says. “We present suitable uses cases and show what ogGPT can do for people in any department or position. We’ve now gotten to the point where users are also helping each other.”

The number of users is rising all the time. “Looking ahead, we’re considering adding solutions to support comprehensive search functions, for example through Azure AI Search and AI-assisted apps,” Leuerer says. “We’re also interested in the multimodal capabilities of GPT-4, which can process images as well as text as input. This means it could answer questions about screenshots, for instance, or program the graphic interface of a computer system based on a sketch.”