Issues with new 365 tenant/accounts – Migration Exchange 2013 to Exchange Online

[German]This month, support for Microsoft Exchange 2013 expires, so administrators are facing a switch to Exchange 2019 or Exchange Online. A blog reader emailed me the other day because he ran into problems when migrating customers from Microsoft Exchange 2013 to Exchange Online. I'm posting the reader's experience report here – maybe it will help others who are affected.

German  blog reader Christian K. supports some customers with Exchange installations. They recognized recently that the support for Microsoft Exchange 2013 ends in April 2023. For these customers, the decision was made to switch to Exchange Online. Christian wrote me about this (I've translated his text):

Problems with new 365 Tenant/Accounts – Migration Exchange 2013 to Exchange Online

Dear Mr. Born,

I am a long time reader of your blog and your site is my primary source of information from current Microsoft issues.

I just wanted to share some recent Exchange 2013 to Exchange Online migrations in case it's interesting….

In general, from my point of view, the topic should be of interest to other readers as well – so I'm just posting it here. For motivation the reader wrote:

We have a couple of customers who noticed that Exchange 2013 may soon stop working. We have created new Microsoft 365 tenants via the Ingram CSP service.

Ingram is a vendor that offers cloud migration support and other services – see this vendor web page. This PowerPoint document, for example, deals with ordering Autopilot services through Ingram Micro Cloud Marketplace. However, I personally cannot provide any further information on the vendor itself.

Errors when creating tenants

When creating tenants for various customers via the Ingram CSP service, the reader encountered error messages, which then triggered a support request. Here is his experience.

Error when creating Tenant A

Christian created the first tenant "A" via this provider on 25.03.2023. He described the following experiences to me.

AzureAD Connect 1.6 installed on Windows Server 2012R2 and installed and synchronized normally after TLS 1.2 activation.

The Exchange Hybrid Configuration Wizard ran through to the point where the Inbound Connector is created.

Below is the error message that appears (it's in German), and it says that the Exchange hybride configuration is finished, but some items need further attention. The E-Mail-Flow configuration is wrong – Hybrid Configuration Wizard (HCW) has a problem.

The incoming connector inbound is deactivated. The following error message was then displayed in the details.

Click to zoom

Christian then opened a support case at Microsoft and wrote me about it:

The Microsoft support case has been running since Friday. So far, the support has no idea what it's about and the function for the tenant was not activated until today. Well, there is still a lot of time until 11.4.

The information "since Friday" must refer to March 31, 2023, since the mail reached me on April 5, 2023. With such an experience, joy naturally arises – and Christian seems to have received no feedback from Microsoft so far.

Error when creating tenant B and C

Following the motto "new game, new (bad) luck", Christian then had two more Exchange Online tenants created via the Ingram CSP service. The experience:

Tenant B and C were created on 3.4. and 4.4.2023.

Here, the Hybrid Configuration Wizard can no longer be executed up to the connector.

The following error message appears:

Christian then looked at the log files that are created during setup. The following appears in the log files:

2023.04.04 10:09:03.749 *ERROR* 10085 [Client=UX, Activity=Tenant Connection Validation, Thread=7] Remote server connection error with the following error message: The following error occurred while connecting to the remote server "outlook.office365.com": Bad Request For more information, see the "about_Remote_Troubleshooting" help topic.

Christian's team then tried to manually establish a connection after installing the Powershell Exchange Online Management Module. This works with the following command in the PowerShell console:

Connect-IPPSSession -ConnectionUri https://outlook.office365.com/powershell-liveid/

but then received the following error message:

The following error occurred while connecting to the remote server "outlook.office365.com": Bad Request Further

information can be found in the help topic "about_Remote_Troubleshooting". In

C:\Program Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\3.1.0\netFramework\ExchangeOnlineManagement.psm1:733 Zeichen:21 +
throw $_.Exception; +~~~~~~~~~~~~~~~~~~ + CategoryInfo : ResourceUnavailable: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : System.Management.Automation.Remoting.PSRemotingDataStructureException

We then tried to connect to the somewhat older Tenant A from the same PC, since the wizard still worked there. The command is the same, you just use different credentials in the login window that appears. This worked without any problems. Just like with our own, very old, tenant. Also in HCW we entered our own tenant data once on the same server for testing. Then it runs through:

For us it looks like Microsoft is currently constantly changing the default settings for new tenants without documentation or possibilities to fix this themselves. This is, of course, shortly before the end of support for Exchange 2013 and without announcements totally super and creates no stress at all.

Support (from Microsoft) is overwhelmed and doesn't know what to do with it at all. Even after reference to the MS page above. Instead, hundreds of questions are asked and checks are required, such as enabling TLS 1.2. The technical department takes its time as usual. Simply having it activated is therefore illusory.

If any more information is desired (log files, screenshots, remote session, etc.), I'd be happy to provide it. We are currently really frustrated that we could no longer do our work on our own. We did hundreds of these migrations last year and early this year with no issues….

So much for this blog reader's experience (my thanks for providing the information). If anyone here can provide "pertinent advice" or contribute similar experiences, comments are welcome – may help other administrators.

A post on reddit.com and an answer

In a follow-up email, the blog reader then sent me a link to the reddit.com thread Problems running Microsoft 365 Hybrid Configuration Wizard with newly created tenants/accounts, where a user writes the following:

Hi,

we have been migrating Exchange Servers to Exchange Online for many years now. Now we have the problem, that we can not do this by ourselves anymore, because microsoft seems to have changed something for newly created tenants (again).

With a tenant we created on March 25th, we can run Hybrid Configuration Wizard (HCW) up to the point where the Cloud Inbound Connector of type "On Premise" is created. Here an error appears with a link to: Inbound connector: FAQ

Q: I'm running HCW(Hybrid Configuration Wizard) and I see a warning message "Inbound Connectors created by HCW are in disabled state"? What do I need to do to enable the Inbound Connector by HCW?

In connectors created via EAC (Exchange Admin Center) or PowerShell cmdlet, customers that experience this behavior must contact Microsoft support. With a business justification to, enable an Inbound connector of OnPremises type for hybrid use within their tenant.

Microsoft Support, as usual, does not know what to do. Our case has been active since last Friday.

With tenants we created on April 3 and 4th. HCW can not even connect to the Exchange Online Service. Logfiles tell us this:

2023.04.05 09:38:44.522 *ERROR* 10085 [Client=UX, Activity=Tenant Connection Validation, Thread=20] Connecting to remote server failed with the following error message: Connecting to remote server outlook.office365.com failed with the following error message : Bad Request For more information, see the about_Remote_Troubleshooting Help topic.

This error can be reproduced by running this command in powershell with ExchangeOnlineManagement Module installed:

PS C:\Users\administrator.GARANT> Connect-IPPSSession -ConnectionUri  outlook.office365.com/powershell-liveid/

Connecting to remote server outlook.office365.com failed with the following error message : For more information, see the

about_Remote_Troubleshooting Help topic.


#At C:\Program Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\3.1.0\netFramework\ExchangeOnlineManagement.psm1:733 char:21

+ throw $_.Exception;

+ ~~~~~~~~~~~~~~~~~~

+ CategoryInfo : ResourceUnavailable: (:) [], ParentContainsErrorRecordException

+ FullyQualifiedErrorId : System.Management.Automation.Remoting.PSRemotingDataStructureException

Using the exact same command on the same machine in the same powershell session with credentials from an older tenant (like the one created on March 25th) works fine. No Problems. So there is something different on all tenants created around April 3/4.

We already checked the users, especially the Option RemotePowerShellEnabled. We also compared all properties of a working user with a non-working user: No difference in any setting but ids and names.

Has anybody experienced something like this or has successfully navigated microsoft support? We currently have six open support cases for different tenants, since you have to create one per tenant and we are currently trying to tell the engineers that there are no tls or permission issues on our side. As usual they give us 20+ questins and links to support articles, that are not relevant here. They do not even understand the Q&A from their own site…

A user named unamused443, who has the MSFT attribute, then replied (a few hours before the blog post was written by me) as follows:

We will have a fix in HCW soon to address scenarios where tenants created in April (and later) cannot connect because of RPS deprecation. Until then, this should get you unblocked:

Deprecation of Remote PowerShell in Exchange Online – Re-enabling or Extending RPS support

So the Microsoft employee refers to the issue of Remote PowerShell in Exchange Online, which I had recently covered in the blog post Deprecated Remote PowerShell gets reprieve in Exchange Online. Here I point out that administrators can only request a suspension of Remote PowerShell deactivation once a year. Blog reader Christian wrote me the following in a supplemental email about this:

Reddit was faster than Microsoft Support Cases here, by the way. There I get only constantly the reference that there is still no answer from the specialist department. We hear this in another case because of update problems twice a week since 13.1.2023.

Unfortunately, I have to realize that almost every sysadmin who has seen Windows once is more qualified or has more competences than the contact persons at the manufacturer of the product. This is getting more and more frustrating, just like the MS KB articles that more and more often describe steps on abolished administration pages or link pages that don't work at all anymore…

On the last case, the contact actually asked me how to open images, along with a screenshot of his Windows PC with the file handler error message….

Thanks again for the blog and articles. Your site has often provided us with valuable solutions and advice. You package the information infinitely more sensibly than Microsoft does.

Let's see if there are any more pointers from the readership on this topic. When I look at the topic from the outside, I can't shake the feeling that the Microsoft Cloud is slowly but surely slipping away from the administrators and also the Microsoft supporters and is mutating into a black box. The developers tinker with a change somewhere and administrators and supporters are then left in the cold to deal with the consequences.