With the R145 release of HCX on 30th October 2020, VMware Cloud on AWS customers were treated to some great new functionality at no additional cost. New features such as Replication Assisted vMotion, Application Path Resiliency, TCP Flow Conditioning, Mobility Groups and my personal favourite, Mobility Optimized Networking. I'm not going to go into too much detail around Mobility Optimized Network (MON) since Patrick Kremer (Blog | Twitter) has covered it extensively here.
On an internal slack channel the following question was asked:
When we migrate a VM from on-premises into VMC that resides on a stretched layer 2 network without enabling MON, any traffic that needs to egress that network either destined to VMs on-premises, within VMC or out to the internet will need to go via on-premises. In my lab I have two VMS currently on-premises in the same VLAN:
With MON disabled we see that when I ping 172.30.41.13 my latency is now ~100ms since traffic has to traverse the L2 extension, which is as expected:
The reason internet traffic egresses via the SDDC even if the default route of 0.0.0.0/0 is being advertised into the SDDC is because we need to avoid asymmetric routing. For stretched networks, if we need internet traffic to always egress via on-premises, maybe because we want to ensure the on-premises security posture is maintained whilst extending into the cloud then we need to configure HCX policy routing. By default, all RFC1918 addressed are configured automatically to route via the source gateway rather than the cloud gateway. In order to route internet traffic via the source gateway, we need to add the default route of 0.0.0.0/0 into the policy route. Within the HCX Network Extension Advanced menu item select Policy Routes: