{"id":26154,"date":"2022-08-11T01:40:56","date_gmt":"2022-08-10T23:40:56","guid":{"rendered":"http:\/\/159.69.82.204\/win\/?p=26154"},"modified":"2022-08-11T08:44:08","modified_gmt":"2022-08-11T06:44:08","slug":"microsoft-365-ausfall-durch-merics-firewall-10-august-2022","status":"publish","type":"post","link":"https:\/\/borncity.com\/win\/2022\/08\/11\/microsoft-365-ausfall-durch-merics-firewall-10-august-2022\/","title":{"rendered":"Microsoft 365 outage due to Cisco Meraki firewall (August 10, 2022)"},"content":{"rendered":"<p>On August 10, 2022, Microsoft 365 services experienced an outage that specifically affected North America but also EMEA. Users had problems with Office 365, Outlook and other services. It was probably due to a false alarm within the Cisco Meraki\u00a0firewall used by Microsoft.<\/p>\n<p><!--more--><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/vg05.met.vgwort.de\/na\/351d34c33ee744b786a54d7cb02a1205\" alt=\"\" width=\"1\" height=\"1\" \/>Microsoft has admitted to problems with its cloud services on <a href=\"https:\/\/twitter.com\/MSFT365Status\/status\/1557347239416176641\" target=\"_blank\" rel=\"noopener\">Twitter<\/a>, as the following tweets show. The initial message already confirmed that network traffic was blocked across various regions.<\/p>\n<p><a href=\"https:\/\/twitter.com\/MSFT365Status\/status\/1557347239416176641\" target=\"_blank\" rel=\"noopener\"><img decoding=\"async\" src=\"https:\/\/i.imgur.com\/IpSGjtM.png\" \/><\/a><\/p>\n<p>It quickly became clear that the whole thing was related to firewall solutions used by Microsoft. Blog reader Markus pointed me to <a href=\"https:\/\/www.bleepingcomputer.com\/news\/microsoft\/microsoft-365-outage-triggered-by-meraki-firewall-false-positive\/\" target=\"_blank\" rel=\"noopener\">this post<\/a> from Bleeping Computer via email (thanks for that). The outage was triggered by a false alarm in the firewall in use, which prevented users from connecting to Exchange Online, Microsoft Teams, Outlook desktop clients and OneDrive for Business.<\/p>\n<p>A Cisco employee addressed this in <a href=\"https:\/\/community.meraki.com\/t5\/Meraki-Service-Notices\/RESOLVED-Microsoft-vulnerability-and-IPS-SNORT\/ba-p\/156649\" target=\"_blank\" rel=\"noopener\">this forum post<\/a>. A vulnerability reported by Microsoft, CVE-2022-35748, triggers SNORT rule 1-60381, which caused problems with communication through the firewall.\u00a0In the meantime, however, Microsoft has probably fixed the problem (see also the notes at Bleeping Computer).<\/p>\n<p>Blog reader Andreas P. sent me the following excerpts from the status area of the Admin Center (thanks for that).<\/p>\n<blockquote><p>Published Time: 10.08.2022 19:56:28<br \/>\nThe firewall partner is currently reviewing options to remediate impact.<br \/>\nThis quick update is designed to give the latest information on this issue.<br \/>\nPublished Time: 10.08.2022 19:02:39<br \/>\nTitle: Some users may be unable to connect to multiple Microsoft 365 services.<br \/>\nUser Impact: Users may be unable to connect to multiple Microsoft 365 services.<br \/>\nMore info: Impacted services include, but are not limited to:<br \/>\n&#8211; Outlook desktop client<br \/>\n&#8211; OneDrive for Business<br \/>\n&#8211; Microsoft Teams<br \/>\nAffected customers have reported that disabling firewall rules blocking TLS 1.2 is mitigating impact. Some firewall vendors have published guidance on disabling the impacting rules, and we recommend contacting your firewall vendor for further assistance.<br \/>\nCurrent status: We continue to work with the firewall partner to investigate a Snort rule which is contributing to impact. Our focus remains on mitigation and from user reports, disabling the specific firewall rule provides immediate relief. Additionally, we continue to investigate recent changes within the Microsoft-managed environment to rule out potential causes of impact.<br \/>\nScope of impact: At this time, impact appears to be specific to some users who are served through the affected infrastructure.<br \/>\nNext update by: Wednesday, August 10, 2022, at 6:30 PM UTC<br \/>\nPublished Time: 10.08.2022 18:41:46<br \/>\nWe're continuing to work with the firewall partner to investigate the issue. Additionally, we monitoring feedback from impacted organizations that disabling a specific firewall rule, which blocks TLS 1.2, is mitigating impact.<br \/>\nThis quick update is designed to give the latest information on this issue.<br \/>\nPublished Time: 10.08.2022 17:25:42<br \/>\nTitle: Some users may be unable to connect to multiple Microsoft 365 services.<br \/>\nUser Impact: Users may be unable to connect to multiple Microsoft 365 services.<br \/>\nMore info: Impacted services include, but are not limited to:<br \/>\n&#8211; Outlook desktop client<br \/>\n&#8211; OneDrive for Business<br \/>\n&#8211; Microsoft Teams<br \/>\nAffected customers have reported that disabling firewall rules blocking TLS 1.2 is mitigating impact.<br \/>\nCurrent status: We've identified an increase in errors related to TLS 1.0 and 1.1 across Microsoft 365 services. We've confirmed that there have not been any recent changes to the service feature which is blocking the traffic. We're continuing to engage with the firewall partners to assist our investigation into the potential blocking of legitimate traffic. Additionally, we're working with impacted users to gather client logs.<br \/>\nScope of impact: At this time, impact appears to be specific to some users who are served through the affected infrastructure.<br \/>\nNext update by: Wednesday, August 10, 2022, at 5:00 PM UTC<br \/>\nPublished Time: 10.08.2022 17:01:14<br \/>\nWe're directly working with some of the affected users to aid in our investigation while continuing to engage with our firewall partners. Analysis into Microsoft 365 client endpoints is ongoing.<br \/>\nThis quick update is designed to give the latest information on this issue.<br \/>\nPublished Time: 10.08.2022 16:28:30<br \/>\nWe're looking at recent changes made within the Microsoft-managed infrastructure and reviewing endpoints that are leveraging TLS 1.2. Additionally, we're contacting firewall partners to assist our investigation.<br \/>\nThis quick update is designed to give the latest information on this issue.<br \/>\nPublished Time: 10.08.2022 15:53:29<br \/>\nTitle: Some users may be unable to connect to multiple Microsoft 365 services.<br \/>\nUser Impact: Users may be unable to connect to multiple Microsoft 365 services.<br \/>\nMore info: Impacted services include, but are not limited to:<br \/>\n&#8211; Outlook desktop client<br \/>\n&#8211; OneDrive for Business<br \/>\n&#8211; Microsoft Teams<br \/>\nCurrent status: After analyzing system telemetry and Fiddler logs from impacted users, we suspect that third-party firewall devices are potentially blocking legitimate Microsoft traffic. Affected customers have reported that disabling firewall rules blocking TLS 1.2 is mitigating impact. We're continuing our investigation into the underlying cause.<br \/>\nScope of impact: At this time, impact appears to be specific to some users who are served through the affected infrastructure.<br \/>\nNext update by: Wednesday, August 10, 2022, at 3:30 PM UTC<br \/>\nPublished Time: 10.08.2022 15:25:09<br \/>\nSome customers are able to mitigate impact by disabling a firewall rule that is blocking TLS 1.2.<br \/>\nThis quick update is designed to give the latest information on this issue.<br \/>\nPublished Time: 10.08.2022 14:57:17<br \/>\nWe're reviewing Exchange trace logs (ETL) from users who are experiencing impact. We believe the issue may be related to Active Directory (AD) services and are investigating this further.<br \/>\nThis quick update is designed to give the latest information on this issue.<br \/>\nPublished Time: 10.08.2022 14:41:52<br \/>\nTitle: Some users may be unable to connect to multiple Microsoft 365 services.<br \/>\nUser Impact: Users may be unable to connect to multiple Microsoft 365 services.<br \/>\nMore info: Impacted services include, but are not limited to:<br \/>\n&#8211; Outlook desktop client<br \/>\n&#8211; OneDrive for Business<br \/>\n&#8211; Microsoft Teams<br \/>\nCurrent status: We're reviewing system telemetry to isolate the source of the issue. Additionally, we're working with impacted users to gather network trace logs to assist our investigation.<br \/>\nScope of impact: At this time, impact appears to be specific to some users who are served through the affected infrastructure.<br \/>\nNext update by: Wednesday, August 10, 2022, at 2:00 PM UTC<\/p><\/blockquote>\n<p>Note: Wrong product name used initially has been amended.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>On August 10, 2022, Microsoft 365 services experienced an outage that specifically affected North America but also EMEA. Users had problems with Office 365, Outlook and other services. It was probably due to a false alarm within the Cisco Meraki\u00a0firewall &hellip; <a href=\"https:\/\/borncity.com\/win\/2022\/08\/11\/microsoft-365-ausfall-durch-merics-firewall-10-august-2022\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[63],"tags":[64,47],"class_list":["post-26154","post","type-post","status-publish","format-standard","hentry","category-cloud","tag-cloud","tag-issue"],"_links":{"self":[{"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/posts\/26154","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/comments?post=26154"}],"version-history":[{"count":0,"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/posts\/26154\/revisions"}],"wp:attachment":[{"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/media?parent=26154"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/categories?post=26154"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/borncity.com\/win\/wp-json\/wp\/v2\/tags?post=26154"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}