Key Rotation Error Caused Large-Scale 365 Outage, Says Microsoft – Response From Industry Expert

By   ISBuzz Team
Writer , Information Security Buzz | Mar 18, 2021 04:54 am PST

The Microsoft has just confirmed that the outage which took down Teams, Exchange Online, and other 365 services was caused by a key rotation issue. More details on Microsoft’s status page here. Expert below warns that outages of this nature are likely to become more common as digital transformation accelerates as the importance of key rotation is often overlooked. 

Notify of
1 Expert Comment
Oldest Most Voted
Inline Feedbacks
View all comments
Michael Thelander
Michael Thelander , Director of Machine Identity Strategy
March 18, 2021 1:04 pm

<p style=\"font-weight: 400;\">Poorly orchestrated key rotation is the Achilles heel of modern digital transformation efforts; this oversight is capable of bringing down entire applications and services in an instant. </p> <p style=\"font-weight: 400;\"> </p> <p style=\"font-weight: 400;\">Keys and certificates have numerous ‘states’ that guide their automation and orchestration processes. They also have hard-coded expirations. ‘Retain’ is a tag that tells the system, ‘This key may be retired or expired, but the system needs to keep it to enable any overlap between dynamic processes.’ If the ‘retain’ tag is overlooked and the keys are deleted before replacements are ready – and this all happens in microseconds – systems fail. </p> <p style=\"font-weight: 400;\"> </p> <p style=\"font-weight: 400;\">Unfortunately, these kinds of outages will only continue until organisations adopt an enterprise-wide approach to managing the machine identities these keys and certificates represent. Digital transformation is not going to slow down, and this requires automation of keys and certificates found in workloads, containers, and across cloud environments as well as those in on-prem environments.</p>

Last edited 2 years ago by Michael Thelander

Recent Posts

Would love your thoughts, please comment.x