
@IowaStateU alum, now @Microsoft Customer Security Director. Love @CycloneATH, @RedSox, sports, running, cuisine, tech. Tweets are mine. He/him.
Clive, IA
Joined June 2008
- Tweets10.2K
- Following1.2K
- Followers1.1K
- Likes757
Adam Brewer retweeted
More details on the root cause can be found in the Microsoft 365 Admin center, and an abridged version can be found below.
We are deeply sorry for the impact to all our customers and recognize the impact that this has on their normal business & operations.
The incident on Thursday, November 12th, 2026, was caused by three main factors that together caused a global outage for the networking gateway that fronts many Microsoft 365 services.
First, a latent defect caused servers to become overwhelmed and unresponsive under certain load conditions. This was due to lock contention inside an internal process which caused the machine to become unavailable. This defect was the result of a fix that was prepared specifically for Microsoft 365 networking gateway servers; neither the underlying defect nor the fix applies to broader Windows deployments.
The initial change in behavior first started deployment in July of this year. The update went through safe deployment and saturated globally over several weeks, during which time no issues were observed. For more than three months, the change was deployed worldwide without impact.
Our normal load testing did not detect the latent issue because the different traffic and SKU mix used in our test labs. The contention issue is worsened on machines with more cores and newer hardware; whereas our load testing tended to focus on our weaker hardware SKUs with fewer cores.
Second, a surge in load increased the likelihood for the networking gateways to encounter the contention. Within Microsoft 365, we maintain two independent and heterogenous network gateway systems designed for isolation and high availability. This allows many Microsoft 365 services to fail to/from them during outages or disruptions. Approximately fifteen hours before the outage, Teams failed over from the alternative networking gateway to the ‘primary’ gateway that also supports Exchange Online. This increased the load on the networking gateway and changed its traffic patterns. This type of failover is part of normal operations and is integral to standard Microsoft resiliency efforts.
Third, at the same time, nearly 14% of server capacity for the networking gateway was out of service due to an unrelated issue from the previous day. This server capacity was on track to be returned to active service through routine maintenance and health processes. The service had sufficient capacity headroom, so the reduction in server capacity had no impact on its own.
Taken together, the increased load and the reduced server capacity, caused the network gateway to encounter the latent bug from July. This caused requests to build up until servers ultimately stopped responding. This occurred during the peak window of traffic in the United States.
When servers lose connectivity, our traffic management systems automatically route users to other healthy endpoints. This happens often and is critical to avoiding any interruption due to hardware, software, or network defects and is normally what makes the service resilient. However, the redirected traffic increased the likelihood for the same problem to occur on the remaining endpoints. This caused a cascading failure from server to server as the same amount of load was spread across fewer servers. In less than two hours, the previously unknown defect was encountered across our fleet and rendered them unresponsive. At its lowest, more than 80% of servers were unresponsive.
Normally, the network gateway is composed of thousands of ‘capacity units’ which represent shards of capacity, hardware, and networking infrastructure. These ‘capacity units’ are then organized into regions. In this case, due to a separate issue, the normal bulkheads between these different partitions did not stop the cascading failure. This is what allowed the impact to spread across our network gateway infrastructure globally.
Several services like Teams, OneDrive, SharePoint, and Office apps failed out of the faulty network gateway infrastructure. That is why those services were restored earlier, with Teams chats and meetings functioning without issue for most of the outage. But not all services, like Exchange Online, can fail out of this network gateway.
Recovery took longer than usual because as servers returned to service, they were immediately overwhelmed by waiting traffic and became unresponsive again. Our normally automated recovery processes were not designed to restore service from this broad unresponsive state. Additionally, since the main code defect was a latent bug from several months previous, our normal rollback, flighting, and feature flags were unable to quickly revert the change.
After engineers confirmed the defect as the cause, we developed an update that returned the code path to the previously known good state. To effectively apply this fix, and to slowly bring the service online again, we took groups of servers out of service, applied the change, and restarted them before returning them to service. This process was carefully applied to large quantities of infrastructure at a time (e.g. 50%+ of a region).
The final Post-Incident Review (PIR) will provide more details later this week.
Replying to @MSFT365Status
We have released the Preliminary Post-Incident Report of last week’s outage, which is accessible via the Admin center under MO1550821 and EX1550543.
Adam Brewer retweeted
Many reporters rushed to ask questions about AI and blame AI for both Microsoft and FAA (even Amtrak) outages when really what they got were more reasons why we need AI and it will be helpful to the development of software systems.
AI is better at complex systems than people.
Adam Brewer retweeted
We have no indication that the FAA’s nationwide ground stop is related to any Microsoft services.
Adam Brewer retweeted
I want to share what we know so far. This was not a cyberattack or an issue caused by AI tooling. Our amazing team was able to work through a very challenging situation to restore service.
Our investigation points to technical challenges and failure modes (cascading failures + thundering herds at our network perimeter) that made recovery more difficult than normal. I'm working closely with the engineering team to complete the full post-mortem. We will publish the post-mortem shortly covering the step-by-step timeline and our many repair items to avoid similar outages in the future.
We take our responsibility to our customers seriously and will do everything we can to learn from this and prevent it going forward.
We are deeply sorry for the disruption this incident has caused our customers and partners. We know the trust they place in our services and take that responsibility extremely seriously. We are seeing broad recovery and stability across our services.
This was a technical issue not related to cyber or AI / agents. The incident was introduced by an issue during an earlier network capacity upgrade that surfaced following a recent network change. We have addressed the underlying issue and continue to restore service to remaining customers and regions.
Given the scale of demand, we restored services gradually, validating stability at each stage. While this took longer, it helped ensure a stable restoration.
We will provide a full post-incident report once our review is complete.
Adam Brewer retweeted
We are deeply sorry for the disruption this incident has caused our customers and partners. We know the trust they place in our services and take that responsibility extremely seriously. We are seeing broad recovery and stability across our services.
This was a technical issue not related to cyber or AI / agents. The incident was introduced by an issue during an earlier network capacity upgrade that surfaced following a recent network change. We have addressed the underlying issue and continue to restore service to remaining customers and regions.
Given the scale of demand, we restored services gradually, validating stability at each stage. While this took longer, it helped ensure a stable restoration.
We will provide a full post-incident report once our review is complete.
Adam Brewer retweeted
We are deeply sorry for the disruption this incident has caused our customers and partners. We know the trust they place in our services and take that responsibility extremely seriously. We are seeing broad recovery and stability across our services.
This was a technical issue not related to cyber or AI / agents. The incident was introduced by an issue during an earlier network capacity upgrade that surfaced following a recent network change. We have addressed the underlying issue and continue to restore service to remaining customers and regions.
Given the scale of demand, we restored services gradually, validating stability at each stage. While this took longer, it helped ensure a stable restoration.
We will provide a full post-incident report once our review is complete.
Adam Brewer retweeted
Raising 3 kids may have prepared me more for managing AI agents than anything I ever did in my long career.
Adam Brewer retweeted
Brock Purdy is now at 1,495 pass attempts in his NFL career. On his 5th throw next Monday night, he will officially qualify for these all-time ranks among QBs:
#1 in passer rating (105.4)
#1 in yards/att (8.6)
#1 in adj net yards/att (8.0)
And #1 in many other advanced stats
sfstandard.com/2026/10/08/br…
Adam Brewer retweeted
JENSEN HUANG: FROM A 9-YEAR-OLD IMMIGRANT TO AMERICA’S HIGHEST SCIENCE HONOR
Nvidia co-founder Jensen Huang received the National Medal of Science today, recognizing his contributions to computing and AI.
His parents were also there to share the moment.
Born in Taiwan, Huang was just nine when his parents sent him and his brother to live with an uncle in the United States. The family later reunited in Oregon.
At 15, he worked at Denny’s washing dishes, clearing tables and serving customers. He went on to earn electrical engineering degrees from Oregon State and Stanford.
In 1993, he co-founded NVIDIA with Chris Malachowsky and Curtis Priem, developing their early plans over meetings at a Denny’s. Their starting idea was to bring better 3D graphics to personal computers.
Thirty-three years later, Huang is being honored for helping turn graphics processors into a foundation for scientific computing and artificial intelligence.
There’s something special about this beyond the company or the medal: the parents who sent their son across the world getting to stand beside him for a moment like this.
Adam Brewer retweeted
👀 Something big is coming.
Join us live @ 10AM PT tomorrow to see what’s next for Windows.
Watch here: youtube.com/live/ilmBGeGldrI
Adam Brewer retweeted
We’re building Copilot as a new OS for work that spans every model, every form factor, and every task. Today, we’re announcing our biggest update to Copilot to date, bringing four things together:
· Autopilot: proactive and long-running agent built for the enterprise
· Code: build apps with Copilot, hosted inside your company’s tenant
· Home: Chat + Cowork together
· Office: now fully embedded in Copilot (and Copilot embedded in Office, of course!)
Plus, you can invoke Copilot in Teams, and we’re introducing Today, a proactive experience that surfaces the most important information from across M365 without needing to ask for it.
The way we work is changing and so are our workflows. This update brings AI into that flow, from answering a question, to building an app, to getting work done on your behalf.
Adam Brewer retweeted
Too many features to count, but some to point out that are coming from Omar’s team:
- Local inference, file transfer and code mode on any machine connected to the OC gateway
- CLAW profiles to bootstrap new agents faster
Next to a lot of security and reliability work. Kudos!
Adam Brewer retweeted
Microsoft shipped a really compelling product on top of @OpenClaw today. We worked with them since March to make the codebase ready for large-scale deployments, they are a great partner and open-source contributor. 🙏🦞
Today we announced the preview of Microsoft Autopilot is rolling out to our first customers - "Autopilot is a persistent, proactive and personal agent that keeps working even when you’re not"
It's been a while since I updated you all on what we are building. First Scout is now Autopilot, and my team leads building this personal agent.
We are building Autopilot on @openclaw, working with @steipete and the OpenClaw Foundation to make it a fantastic enterprise grade runtime.
blogs.microsoft.com/blog/202…
Re-introducing Sebastian, my Autopilot - and he is as excited as I am!
Adam Brewer retweeted
Today we announced the preview of Microsoft Autopilot is rolling out to our first customers - "Autopilot is a persistent, proactive and personal agent that keeps working even when you’re not"
It's been a while since I updated you all on what we are building. First Scout is now Autopilot, and my team leads building this personal agent.
We are building Autopilot on @openclaw, working with @steipete and the OpenClaw Foundation to make it a fantastic enterprise grade runtime.
blogs.microsoft.com/blog/202…
Re-introducing Sebastian, my Autopilot - and he is as excited as I am!
Adam Brewer retweeted
Today we announced the preview of Microsoft Autopilot is rolling out to our first customers - "Autopilot is a persistent, proactive and personal agent that keeps working even when you’re not"
It's been a while since I updated you all on what we are building. First Scout is now Autopilot, and my team leads building this personal agent.
We are building Autopilot on @openclaw, working with @steipete and the OpenClaw Foundation to make it a fantastic enterprise grade runtime.
blogs.microsoft.com/blog/202…
Re-introducing Sebastian, my Autopilot - and he is as excited as I am!



























