VMware Transit Connect default route and the impact on VPN and HCX Connectivity

I recently had a query from a customer who was implementing intra-region peering between a VMware Transit Gateway and a native AWS Transit Gateway which would then be attached to a security VPC. Their requirement was to ensure that all VM connectivity from the SDDC would traverse the security VPC before egressing out to the internet or back to on-premises. This would require them to add a static route into the vTGW to point all traffic (0.0.0.0/0) to the peering attachment which connected the vTGW to the TGW. From there they

HCX Service Mesh and Route Based VPN with default route advertised

I've been asked a few times around whether or not HCX can be used if the customer has a route-based VPN into VMware Cloud on AWS and is advertising the default route 0.0.0.0/0 into the SDDC. The short answer is Yes, this is supported and works.

The long answer as to why this question comes up is that when we advertise the default route of 0.0.0.0/0 into the SDDC then all traffic from the SDDC will flow via on-premises, this includes traffic destined for the internet. Some customers prefer to do this to ensure all outbound internet traffic routes via their perimeter firewall so they can ensure that all security and logging policies are applied. The confusion comes around HCX. Since HCX is unable to use an existing IPSEC VPN tunnel to send traffic from on-premises into the SDDC as per the KB article it needs to establish its own. The HCX-IX Interconnect and HCX Network Extension Appliances both establish an IPSEC VPN tunnel from on-premises to their peer appliances in the SDDC using UDP/4500. So the question is, if we are advertising 0.0.0.0/0 into the SDDC so all traffic traverses the IPSEC VPN tunnel back to on-premises, then how can the HCX-IX Interconnect and HCX Network Extension Appliances communicate?

HCX Mobility Optimized Networking Policy Routes

With the R145 release of HCX on 30th October 2020, VMware Cloud on AWS customers were treated to some great new functionality at no additional cost. New features such as Replication Assisted vMotion, Application Path Resiliency, TCP Flow Conditioning, Mobility Groups and my personal favourite, Mobility Optimized Networking. I'm not going to go into too much detail around Mobility Optimized Network (MON) since Patrick Kremer (Blog | Twitter) has covered it extensively here.

On an internal slack channel the following question was asked:

When we migrate a VM from on-premises into VMC that resides on a stretched layer 2 network without enabling MON, any traffic that needs to egress that network either destined to VMs on-premises, within VMC or out to the internet

HCX Central CLI Mind Map

In a previous article I introduced the HCX Central CLI (CCLI) as a tool to help troubleshoot HCX connectivity and performance issues. I've created a quick Mind Map of all the various commands that are available through CCLI to ease the troubleshooting process.

You can download a .pdf version of the full Mind Map here

This is correct as of HCX Version R144 (Build 16989452)

Troubleshooting HCX Connectivity and performance issue into VMware Cloud on AWS

When working with customers on VMware Cloud on AWS POC's or Pilots a lot of the success criteria typically includes using Hybrid Cloud Extension (HCX) to migrate workloads from on-premises into VMware Cloud on AWS either using bulk migration or live vMotion. This sometimes involves troubleshooting connectivity and performance issue so I figured I would show the typical process that I follow to try and narrow down and identify the issue.

HCX Service Mesh Appliances Tunnel Status