On Monday, Microsoft's DMARC report for my domain showed an SPF failure on an email I'd sent to another M365 tenant. Both ends were MS, and its own check decided its own sending server wasn't allowed to send for me.
The same day, Google's report showed SPF passing for mail from the same Microsoft servers, and Microsoft's reports showed a pass for mail I'd sent through OCI. Only MS to MS failed. My reports were clean before the outage, and they've been clean since. The email still landed, because DKIM passed and carried it through DMARC. My policy is set to quarantine, so without DKIM that email would have gone to junk and I'd have had no idea why.
That report came two days after Microsoft said M365 was fully restored.
It's been a bad week for outages. M365 went down on 12 November for about 24 hours, with users affected in 74 countries. The FAA, Chase, United and Apple all had their own incidents within days. Every organisation that has explained its outage has pointed to a technical fault, and none has called it an attack.
Microsoft's preliminary report is interesting. The bug came from a networking gateway update in July. It sat there quietly for four months, until a later network change ran into it. They've listed three factors that had to line up before anything broke. Each change on its own looked fine.
Last month I wrote about Black October and the flurry of urgent patches everyone was rushing through. Each of those patches was tested by the team that shipped it. I'd guess very few were tested against each other, because nobody owns that. How many changes are stacking up in your environment right now, waiting for the one that makes them collide?
Recovery is a change too. I don't know what shifted on the MS side to fail my email that day. My guess is something in the recovery path, where traffic runs through routes that don't normally carry it. Whatever it was, it didn't show up on any status page.
When a vendor says 'fully restored', they're talking about their own service, and that's all they can vouch for. I caught this one because I have scripts parsing my DMARC reports, like most companies do, and they alerted me. The harder part is deciding that an SPF alert in the week after a vendor outage is worth chasing.
If your organisation sends 2K external emails a day without DKIM on every system, a few days of failed SPF means a few days of mail quietly going to junk. Nobody raises a ticket for an email they never saw.
Who in your organisation owns the week after a vendor outage?
#Microsoft365 #DMARC #ITGovernance #Resilience
