[German]A short note to the administrators whose on-premises Exchange servers are currently on strike and cannot load the FIP-FS scan engine (virus scanner) and report an error Can't Convert "2201010001" to long. You are probably not alone, as of Jan. 1, 2022 0:00 UTC on-premises Exchange servers seem to freezing transport of all emails – a date can't get converted. Here is a quick overview of what is going on.
FIP-FS Scan Engine
FIP-FS is probably the anti-malware virus scanner that has been on board since Exchange Server 2013. This is supposed to scan the on-premises Exchange Server installation for malicious content. However, this anti-malware scan engine seems to cause problems more often. Back in October 2016, Frank Zöchling published the German blog post Exchange 2016: FIPFS Event ID 6027 Filter Updates werden nicht runtergeladen, which pointed out numerous error possibilities when updating the signature files. veröffentlicht, der auf zahlreiche Fehlermöglichkeiten beim Aktualisieren der Signaturdateien hinweist.
And in early December 2021, someone on serverfault.com posted the entry Exchange 2019 Antimalware engine updates download but don't get applied. There, the FIP-FS MS Filtering Engine permanently generates entries in the Event Viewer because updates could not be installed. It has probably affected different Exchange versions and might be related to downloads of updates or virus signatures. There the error was probably corrected by Microsoft.
FIP-FS error Can't Convert "2201010001" to long
I was just alerted on Twitter by a follower to the following tweet, which briefly describes the problem, which has been occurring since January 1, 2022.
Under Exchange, the Microsoft Scan Engine FIP-FS cannot be loaded. Instead, the error Can't Convert "2201010001" to long is reported. Seems that the new date is a challenge for Exchange. In follow-up tweets, another user reports with the name Joseph Roosen:
Umm ya having issues with no mailflow because of this since that keeps crashing over and over since 0000 UTC.
Dear we have a problem with hybrid since this service keeps crashing so basically mail is down.
@MSFTExchange @Microsoft @MSFT365Status
So it fits, the Exchange doesn't let any mails through anymore. And via Facebook, someone pointed me to this tweet from Joseph Roosen, which puts it a bit in context:
Just in time for the new year, the virus scanner on Exchange Server goes on strike and scares administrators. Since March 2021, Microsoft has published the article The FIP-FS Scan Process failed initialization. Error: 0x80010105 AND Faulting application name: scanningprocess.exe, which refers to Exchange Server 2016 on Windows Server 2016. There is the current entry from 1/1/2022:
The exchange server was stuck with this error:
The FIP-FS "Microsoft" Scan Engine failed to load. PID: 39268, Error Code: 0x80004005. Error Description: Can't convert "2201010003" to long. / Event ID 5300
I have disabled filtering:
Set-MalwareFilteringServer exch-19 -BypassFiltering $true
and email is going agian … and I am looking for more info how to recover malware scanning service
happy new year exchnage
So it seems that some (all?) Exchange servers are affected.
Workaround: Disable Anti Malware Agent
The affected person from the above tweet has meanwhile posted a very simple solution via Twitter. He has deactivated the anti-malware agent on the Exchange server.
For this purpose there is the script Disable-AntiMalwareScanning.ps1. Then malware scans are no longer executed – but the mails can be sent and delivered again. Someone else who is affected by this New Year's surprise.
Addendum: Microsoft has now confimed the issue and is working on a fix – see Microsoft confirms Exchange Year 2022 problem that FIP-FS Scan Engine failed to load (Jan. 1, 2022).
Security updates for Exchange Server (July 2021)
Cumulative Exchange CUs June 2021 released
Exchange Server Security Update KB5001779 (April 13, 2021)
Exchange isues with ECP/OWA search after installing security update (March 2021)
Exchange security updates from July 2021 breaks ECP and OWA
Exchange 2016/2019: Outlook problems due to AMSI integration
Wave of attacks, almost 2,000 Exchange servers hacked via ProxyShell
Exchange Server 2016-2019: Custom attributes in ECP no longer updatable after CU installation (July 2021)
Exchange Server: Authentication bypass with ProxyToken
Exchange vulnerabilities: Will we see Hafnium II?
Exchange 2016/2019: Outlook problems due to AMSI integration
Exchange Server September 2021 CU comes Sept. 28 with Microsoft Exchange Emergency Mitigation Service
Exchange Server September 2021 CU (2021/09/28)
Security updates for Exchange Server (October 2021)
Tianfu Cup 2021: Exchange 2019 and iPhone hacked
Babuk gang uses ProxyShell vulnerability in Exchange for ransomware attacks
Exchange Server November 2021 Security Updates Close RCE Vulnerability CVE-2021-423
CERT warning: Compromised Exchange servers are misused for email attacks (Nov. 2021)
CERT-Federation, USA, GB warns about attacks on Exchange and Fortinet
ProxyNoShell: Mandiant warns of new attack methods on Exchange servers (Nov. 2021)
ProxyShell, Squirrelwaffle and a new PoC Exploit, patch your Exchange Server!
Examples of virus mails from a compromised Exchange server
German CERT-Bund warns about vulnerable Exchange Server with OWA reachable from Internet
Cookies helps to fund this blog: Cookie settings
Many thanks for your post. It came up first on Google when i searched for FIPFS 5300.
I couldn't work out why this just started failing around GMT 00:00 today but started investigations and discovered it was the antimalware service causing the issue.
I wanted to concur with someone else before i disabled the service. Looks like this is the best workaround for the time being.
should bei resolved with the last update released a couple of minutes ago – See also the comments unser my German Post, which also delivers an explanation of the error
Addendum: Microsoft has confirmed the issue – see Email Stuck in Transport Queues – they wrote: a fix will follow as early as possible.
Luckily there's solution for this and is working.
You have to disable temporarily FIP-FS service.
Please see below MS Exchange Documentation link, showing how this can be done:
thank you so much :) i resolved this problem for 10min …. now i am hero :D thank you
Simpler if you have multiple servers:
get-malwarefilteringserver | set-malwarefilteringserver -bypassfiltering $true
Huge thanks to borncity…
Microsoft do a fix ASAP!!! For Office 356 they shureley made already a fix. Because this workarround isnt really good for my oppinion, but at least its working now. Now im happy for our Appriver Filter that we have :-)
First Day in new year and already troubles with Microsoft. Happy new year to all :-)
We have faced the same issue and we executed the script Disable-AntiMalwareScanning.ps1. and restarted the Microsoft exchange transport service. then the mail flow started working again.
we executed the script Disable-AntiMalwareScanning.ps1. and restarted the Microsoft exchange transport service. then the mail flow started working again too.
My issue started from
The Microsoft Filtering Management Service service terminated unexpectedly. It has done this 1 time(s).
About 0-20 1-1-2022 UTC
then was FIP-FS issues
X exchange servers 2016/2019
I have faced the same issue and I executed the script Disable-AntiMalwareScanning.ps1. and restarted the Microsoft exchange transport service. then the mail flow started working again.
I was operating on prem EX2019 CU10 15.02.0922.007
To confirm, are we awaiting a statement from MS with regard to a resolution?
Thank you for this post. This saved me hours of troubleshooting. Send and receive logs looked all good, except from the mails that never appeared in the inbox.
Had this also on all the exchange servers I am managing.
Probably Internet Explorer is a involved and the Y2K bug came out with 22 years delay.. thank you Microsoft for the readiness test!
Happy new year to all!
If you still have issues, please check (disable) your transport rules as these also use the same engine. We have spent the last 7 hours trying all the workarounds but nothing worked. Turns out we had transport rules that scanned emails for attachments etc…
Thank you for this. Disabling the antimalware alone didn't resolve it for us either. After disabling the one transport rule we had that filtered attachments, mail started flowing for us.
Many thank's for the tricks ;)
Work very well for me !
Thanks for the post.
A friend of mine had the same problem yesterday (I'm at GMT-6). He told me that his mail server wasn't working since 18h00. I just checked out his event viewer and found multiple FIP-FS errors and after a single google search found this.
You guys are heroes!
I'm waiting for any M$ news about a critical patch
Glad Google found this top hit!
So many good comments here to, I am running again. Thanks all for the input and to borncity.com!
I used this mentioned workaround:
Get-MalwareFilteringServer|Set-MalwareFilteringServer -BypassFiltering $true
THen restarted the Transport service.
I have the same problem here and with shared solutions it's not working well.
We follow through with the incident in our environment and our email queue only grows.
Thanks for posting that fixed my issue too.
Dumbass Microsoft "agile" (translation: we never test this crap) strikes again.
Thx that I can see this post
I can fixed the problem with this workaround solution