On Monday, Xbox suffered a major server outage that affected players worldwide. The outage took down several key functions of Xbox, including the ability to buy and download games, play them offline, and even extended to disc-based games. Services have since been restored, but the sudden and severe nature of the outage required some explanation.
Xbox chief technical officer (CTO) Scott Van Vliet did just that, offering some information on what happened and what Microsoft is doing to prevent similar outages in a lengthy post on X. Van Vliet began by saying that services were restored at 2:30pm PT (5:30pm ET, 10:30pm BST) on Monday.
The cause of the issue, according to the CTO, was “a licensing service that sits outside of Xbox, but which Xbox depends upon”. When that service began failing, it caused “some sign-in scenarios to fail, and it caused many scenarios that require an entitlement check to also fail”.
An entitlement is what Microsoft calls digital rights. When the system can’t determine which user is entitled to what, it grinds to a halt. As the post explains, this is why players haven’t been able to view their library of games or launch anything they own.
“It also affected several of our publishing and store partners who depend on those same systems which is why some of you saw issues in specific games and not others,” Van Vliet added.
Once the teams identified the outage, the failing service was isolated and engineers shifted traffic to non-affected parts of the system while the team worked on finding the root cause. Van Vliet didn’t say whether that root cause had been identified, but they did confirm that Xbox is “running a full post-incident review”.
“I care less about the one-line root cause and more about the real questions: why a failure in one service was able to take down this much, why recovery took as long as it did, and what we change so a single point of failure can’t ruin your night again,” the CTO continued.
“That means hardening the dependencies underneath sign in and game launch, improving how we detect and contain this class of failure, and being faster and clearer with you when something breaks.”
The transparency is part of a larger plan to share more about what the various Xbox teams are working on with the community. “Over the coming weeks, you’ll hear more from me about our services, our platform, and the work we’re doing to make Xbox better,” the CTO added.
The last few days have called into question the industry’s shift to an all-digital future. Seemingly the result of cosmic alignment, PlayStation also suffered its own PlayStation Network outage just days earlier. The response was especially illustrative of why players lack trust in those digital-only systems, particularly in the wake of Sony’s decision to end production of physical discs.
Many have been calling on the company to reverse its decision. Some signed a petition in the hundreds of thousands, while others are preparing to power down their PS5 consoles for a week to send Sony a message.
While Xbox avoided that backlash by simply not announcing a similar plan of its own to end discs, the company has been dealing with the resulting anger of its decision to lay off 3,200 workers across almost all of its studios.
