Cloud9 conquers esports using Azure data analytics

Stalking an elusive target: Data

Professional gaming is a wildly successful growth industry, and Cloud9 has battled its way to the top. Based in Los Angeles, the organization boasts world-class players, coaches, and international teams across more than a dozen of the most popular games in esports, including League of Legends. Cloud9 teams hold several championships, but to stay on top, data and analytics are imperative.

The team of data scientists at Cloud9 knows that the right data can improve the way players train, strategize, and compete. Game matches generate many potential sources of information about the players and the most useful strategies. Riot Games, the developer of League of Legends, even publishes an API that gives developers access to some game parameters, such as the champion a player uses and its stats in a match.

However, games are dynamic, interactive environments. They may be a source of big data, but capturing it isn’t like ingesting sensor readings in the Internet of Things. Thousands of matches are held weekly, and the gaming telemetry of most interest to the players and coaches at Cloud9 isn’t necessarily recorded or part of any API.

So can you mine unavailable data? And even if you can, how do you capture useful information about a match when the factors that lead to success in one game may not apply in the next? To keep games challenging, manufacturers release software patches that change key variables for players. These updates change the game every few weeks, alter player strategies, and upset trends.

To solve this problem, Cloud9 consulted Microsoft. "They asked us, ’How can you help us win?’” explains Mike Downey, Director of Sports Technology at Microsoft. The question was so intriguing that Microsoft became an official technology and performance analytics partner of all Cloud9 branded teams, including the League of Legends Championship Series (LCS) team and the Counter Strike: Global Offensive team.

Engineers and data scientists from Cloud9 and Microsoft began to hunt an elusive target—data that isn’t readily available—and find a way to transform it into something useful for Cloud9 players and coaches.

Capturing the target on video

Livestreaming video is a gold mine of data that AI can capture. AI-based video analytics can respond to streaming events in real time. But first you need the video. The team began to capture matches as video and to use computer vision APIs to process images and return useful information, such as the movement and behavioral trends of the enemy.

Computer vision algorithms can analyze visual content in different ways based on user inputs. For example, the API can detect objects and describe text content in images. “We could see how the game progresses over time, specific actions that are taken in a game, and the results of an action,” explains Downey. “When you can see patterns, you can formulate strategies.”

To refine the results, the development team worked with Watch For, a group in Microsoft Research that created a breakthrough AI-based video analytics platform. Watch For grabbed headlines at a Microsoft hackathon when it built a vision algorithm that locates the most interesting parts of a video stream in real time—even if humans don’t know what will prove to be the most interesting parts. One consumer of this sophisticated technology is the Bing search engine, which uses it to identify and extract data from gaming livestreams so that fans get more exciting search results.

The development team adopted the Watch For technology and took it a step further, porting the code to .NET Core so that it could be deployed in Linux-based Docker containers. The developers built a DevOps workflow on Azure Pipelines that finds the games to monitor, captures the interesting parts, transforms the data into a usable form, and stores it in one place for easy access. A monitoring solution tracks the health of the Game Insights Platform and notifies administrators about updates and changes. Another team specializing in Microsoft Power BI created a custom dashboard for data visualization and querying.

The result is a complex, sophisticated, end-to-end solution that starts with a League of Legends game and ends with a set of strategic stats tailored to the viewer. The platform can process hundreds of videos per day and push them to Azure, where the pipeline can run, analyze, and export the interesting data for further analysis.

Mapping an architecture

The primary architecture of the Game Insights Platform is designed in four parts:

• Ingest and store: A queue records the desired games and stores the resulting videos in Azure Blob storage.
• Model runner: A machine learning model—part of the Watch For AI system—finds the interesting parts of the video. Model output is stored for later analysis in Azure Database for PostgreSQL, a fully managed service for running PostgreSQL databases with intelligent performance, scalability, and security.
• Replay: Whenever a video drops, an event-driven, container-based workflow is initiated. This process runs functions against the data to make it accessible for display and analysis.
• DevOps: Using Azure Pipelines, the team set up a continuous integration (CI) and continuous deployment (CD) workflow that deploys and tests the solution components on Azure and verifies the output.

The key to the project's success was finding the right way to orchestrate all the moving parts. "We overcame the technical limitations with a hybrid approach that integrates the on-premises solutions seamlessly into the services on Azure," says Halee Mason, Lead Data Scientist for Cloud9. 

The two environments take turns processing the data. The hybrid platform relies on Azure Queue storage and Azure Functions to prioritize and respond to events in real time. Queue storage is a service for storing large numbers of messages. In this case, it queues up prioritized lists of the videos to capture as part of the ingestion step. Azure Functions is a serverless computing service that allows the developers to run a lightning fast, event-driven data processing flow without the overhead of managing infrastructure.

Alerts and actions are written as Azure Functions code. Functions can respond to queue messages and transform the incoming data before passing it to the next queue. The automated processing starts in the cloud with a time-based function that kicks off the ingestion process, triggering the servers on-premises to start capturing video. The output is sent to Azure. When a video drops, another function is triggered to set up data processing using containers. 

"The orchestration was complex," says Mason. "The queuing system and the event-driven functions were instrumental in tracking all the different matches." 

All the functions are hosted in an Azure App Service container, which lets the team control the scaling of the function app. The team also hosts a small web server in App Service.

The model runner and replay workloads run in Azure Container Instances, a managed, serverless Azure environment. The developers can run complex tasks as self-contained units in response to events and use a container development pipeline for quick deployment.

The Container Instances service also scales dynamically to meet peak processing loads and offers per-second billing, making it more cost-effective than provisioning a static infrastructure. “The goal was to make it easy to scale,” says Downey. “With Container Instances, the platform can scale from hundreds to thousands of games per day with just a few switches in Azure portal.”

Building a pipeline

To deliver more value in less time, the team automated the build and release pipeline, including an infrastructure as code deployment of the Azure environment. “With Azure Pipelines, we were able to automate everything and get features into production quickly,” notes Downey.

Developers created an automated, repeatable process for running builds, performing tests, and deploying software releases. Terraform scripts describe the desired infrastructure on Azure that the pipeline deploys in the test and production environments. The template-based deployment also makes the platform easy for Cloud9 to maintain.

Different types of Azure Pipelines triggers are used to automate the workflow. For example, when a developer changes code and submits a pull request, a new build pipeline is triggered. Next, Docker pushes a newly built image to Azure Container Registry, a private store for container images. Azure Container Instances can then pull a new image from the registry and perform tasks on the gaming data.

With this setup, Cloud9 can manage the configuration of the code and functions completely apart from the configuration of the hosting environment. Azure includes multiple monitoring services that perform specific roles.

Azure Monitor collects baseline infrastructure metrics and logs from the telemetry emitted by various Azure services. This service includes Application Insights, an extensible Application Performance Management (APM) service that is used to monitor the health of the solution and to detect poor performance, failures, or other anomalies. The Log Analytics feature of Azure Monitor correlates the usage and performance data collected by Application Insights with configuration and performance data across the Azure resources that support the solution.

"Logs from the platform proved to be very valuable," says Danny Brady, Senior Data Engineer at Cloud9. "Application Insights gathers all the logs—from on-premises and in the cloud. We get a unified view that's really useful."

Visualizing a winning strategy

The platform’s complex pipelines ultimately pump data to a customized, real-time dashboard built using Microsoft Power BI. Coaches and players use the dashboard to filter the data by player, match, and other variables and look at trends across time. For example, Cloud9 managers can easily verify playing times to ensure that players are meeting their contracted practice hours—something they had to do manually before. Players use the dashboard to track wins, strategies, and matches over time.

“Designing a Power BI dashboard for a gamer audience presents a unique challenge,” explains Chris Hamill, a Microsoft strategist who worked on the solution. “Power BI has the features and capability to build an app-like experience that the players and coaches can navigate without any training. This allows for an extremely fast path to get from data to insights to action.”

For example, Cloud9 can use the Power BI report to strategize its in-game draft, the critical step where teams pick the champions to play, and to find performance insights about the team and individuals.

Power BI is a no-code or low-code solution for data visualization that can tap into dozens of data sources, and it even supports citizen development of reporting assets. Displaying the data is a matter of choosing a Power BI visual and setting its data source and properties.

With so many visuals to choose from and so much data to view, the developers wanted to make sure that coaches and players found the interface easy to use. Developers have a lot of control over the user interfaces they build in Power BI. For example, the development team included familiar logos to identify players and it incorporated tooltips, a programmable object that displays how-to information when a user hovers the mouse over an item on-screen.

Cloud9 also uses the Power BI interface as more than a dashboard. "People tend to think of Power BI as a tool to build reports for publication, but there is immense value in using it for ad-hoc analysis without ever publishing the results," says Hamill.

Working with the same curated dataset as the published dashboard, Cloud9 uses Power BI Desktop to quickly visualize data, find the analytical threads to pull, and answer questions easily. Players and staff can even stay connected to the same dashboard from anywhere in the world using Power BI mobile apps on their iOS and Android devices.

Next steps

In the end, it’s all about winning, and the Game Insights Platform is changing the way Cloud9 plays the game. After a recent match, even the commentators noticed and complimented Cloud9 on its new strategy.

Designed to push boundaries, the platform is providing some surprising insights that Cloud9 prefers to keep to itself. However, the team is happy to share the approach, which applies to any scenario that starts with seemingly unavailable data.

As Downey notes, “It's not custom code. It's implementing services that Azure makes available.”

The team is already planning its next steps. "The first feat was capturing the data at scale,” says Mason. “The next technical feat will be making sense of all the data we’ve collected, and we’re looking at Azure Databricks as a possibility. We’re only at the beginning.”