Monday 18 January 2021

AWS DataSync via VPC Endpoints instead of Public Endpoints

In my previous article, I walked through the steps of replicating an on-premises file server into Amazon FSx using the AWS DataSync service. Since I didn't have a VPN between my on-premises and the VPC where the FSx service was deployed to I had to use the public endpoint whereby all communication from the DataSync agent to AWS occurs over the public internet. Within this article, I'm just going to quickly show you the process of setting this up using VPC Endpoints so communication goes over a VPN or Direct Connect directly into the VPC. This will allow for reverse replication which will be a topic for a future article. 

I currently the AWS DataSync agent deployed with a routable static IP address on-premises and a VPN established into my VPC. I first need to create a VPC endpoint in my VPC for the AWS DataSync Service. Ensure you are in the correct region and navigate to the VPC service. From within there, you will see the option to add Endpoints:

Search for the AWS DataSync service and ensure you select the correct VPC it needs to be created in. I've left the default options to ensure an interface is deployed into all subnets within that VPC:

You also have the option to enable DNS and also specify a specific security group. You may want to lock down access to this endpoint so that it can only be accessed via the DataSync agent IP address on-premises:

Once you create the endpoint it takes a few minutes to actually create it but it should eventually move from pending to available.

Once the VPC Endpoint has been created we can now activate the agent using VPC Endpoints rather than Public Endpoints. The VPC Endpoint that we just created should automatically be selected, if you have more than one ensure you select the correct one:

Now that we are communicating over VPN or Direct Connect we can automatically retrieve the key from the agent (Provided your on-premises firewall and AWS security groups allow communication). The only information that we need to add is the DataSync agent private IP address: 

If all is well then the agent will successfully activate:

We can now see that the agent can successfully communicate with the VPC Endpoint and we can continue to define our on-premises and FSx locations ready for replication:

If you receive the following error during activation ensure that your security groups allow traffic inbound from the on-premises agent IP address:

In order to resolve this quickly, I simply ensured that my VPC default security group allowed traffic from the on-premises network CIDR range:

No comments:

Post a Comment