vMotion and changing the NSX-T network

During one of my deployments I encountered something very odd; I couldn’t vMotion and at the same time change the connected network of my VM from an NSX-T Logical Segment to something else.

Wat.

Looking at the KB article mentioned, KB56991, at the time of writing made no mention of migrating from an NSX-T segment, just scenarios of migrating to an NSX-T segment. Update: there is a different KB article which mentions this issue: KB85516.

Normally this wouldn’t really be an issue. Just change the network before or after the vMotion. However, I encountered this when migrating from one VCF Workload Domain to another, so it was cross-vCenter and the source NSX-T segment did not exist on the destination.

So the options were to either;

1. Cold migrate the VM, causing more downtime;
2. Change the network to a placeholder/non-NSX-T network prior to migration, also causing downtime;
3. Configure the network to also exist on the destination, migrate, then change the network.

None of these options were suitable, but even beyond that, why is this not an option?! I couldn’t really believe this isn’t supported either, this seems like such a basic functionality that should exist.

Posting the question to the vExpert channel I was met with very confused responses; how does this not work?! And more interestingly, how did nobody notice this until now?

Going back

So I created a VMware Service Request to get to the bottom of this. Turns out, the fact that this doesn’t work has been mentioned previously, in the release notes for vSphere 6.7u2.

But then the future release notes make no mention of this anymore, neither as it being resolved or persisting. Just vanished.

Easy fix

Despite the fact that GSS was also a little taken aback, they did provide me with a very easy fix. A little too easy it seemed, but they assured me that this is safe to communicate with the outside world.

In fact, it should be in the release notes, and they have assured me that a change is underway to document this somewhere, either a KB article or in the release notes.

To be able to vMotion and change from an NSX-T Segment at the same time you have to configure the following Advanced Setting in the vCenter configuration:

config.vpxd.network.allowVmotionBetweenLogicalSwitches and set the value to true.

This immediately allows us to do what we want to do, no need to restart services or anything:

Weird.