AWS Consulting Sydney

Securing Matillion ETL with AWS Simple AD

If you are using  Matillion ETL/ELT tool for Redshift and want to take advantage of AWS Simple AD, the short answer is yes, you can.

The Matillion marketplace AMI works with AWS Simple AD LDAP.

Here are the complete step-by-step instructions to help you get started. You’ll need to be running an AWS Simple AD, one Windows instance (for AD configuration) and one Matillion server in your VPC. We assume you have an existing Simple AD in your account already. If you need a brief of how to create a new AWS Simple AD, please follow this AWS document.

Once the status of your new directory is “Active”, we can start.

To secure Matillion ETL with AWS Simple AD we need to do following:

  1. Configure the Windows box to manage the Active Directory (Optional)

    If you already have a windows server that can manage your Active Directory already, please start from the next step.

  1. Create Matillion users and user groups in Active Directory
  2. Configure the Matillion server to connect Active Directory

 

 

Configure the Windows box to manage the Active Directory

  1. Join the Windows server to the Domain

    1. Change the Windows server DNS setup

      Option A: (Recommended) Update VPC DHCP Options set with the Simple AD DNS addresses. Restart the Windows instance after the updated DHCP is in place.

      Option B: Setup DNS server addresses in the Windows instance directly. (Note: Restarting the Windows instance will reset the DNS setup to VPC DHCP options)

      Open Network and Sharing Center -> Ethernet -> Properties -> IPv4 -> Properties -> Put the Simple AD both addresses in “Use the following DNS server addresses” section -> OK

    1. Joining the Windows instance to the domain.

      This PC -> Right Click -> Properties -> Change settings

Open “Computer Name/Domain Changes”, set it as member of your domain.

 

 

In this demo, our test domain is matillion.ad.test.  The system will pop up a window require your domain login followed by a welcome message.  Then restart the server as required. Login the server again with your domain login.

 

 

Create Matillion users and user groups in Active Directory

After the server restart, you can confirm the Windows instance has become a domain member by checking the computer properties again

 

 

  1. Install AD Role Administration Tools. (If not installed on the server)
  • Start “Server Manager”
  • Choose “Add roles and features”
  • Click through the wizard until “Features”
  • Go to “Remote Server Administration Tools” and expand it
  • Go to “Role Administration Tools” and expand it
  • Select “AD DS and AD LDS Tools” (as the screenshot below)
  • Click through the rest of wizard.
  • Restart the server if required.

  1. Create/manage users and user groups for Matillion in Active Directory

Go to Control Panel -> System and Security\Administrative Tools -> Active Directory Users and Computers

Create or pick (if you have existing groups) 3 User groups under your Active Directory -> Users. By default, the Matillion has 3 user roles “Admin”, “Emerald” and “API”. However, you can create these groups in any name you prefer, then map the group name with user role properly. In this demonstration, I use the user role as group name directly.

Then create/choose user(s) and assign them to required groups.  In here, since it’s test, I created only one user “matillion test” and assigned it to all three groups.

Configure the Matillion server to connect Active Directory

Before you start, please make sure you have the ssh access to the Matillion server or you have the latest AMI/snapshot that you can recover from. Incorrect configuration may block you out from logging in the server.

  1. Logon the Matillion server admin portal,
  2. Go in to “Security” section
  3. Select “External” tab
  4. Fill in the Simple AD details as per your AD setup.
  5. Save configuration
  6. Restart server

The details of Matillion supported 3 user-roles are

  1. Emerald – Gives you access to the Matillion ETL interface to build/test/run jobs.
  2. Admin – Access to Admin portal to manage users, configure LDAP, etc.
  3. API – allows a user to be used for issuing API calls to Matillion REST API.

Before restarting, make sure you’ve reviewed the configuration properly and mapped the AD user group with Matillion user role correctly.

Also, the screenshot below is my setup.

 

After restarting, you should be able to login Matillion platform with domain user.

 

Note: one thing I didn’t test is managing users in Matillion after connecting it with Simple AD. I assume domain user management mainly done by the domain administrator. Yet, in some user cases, client may prefer to use Matillion admin user to manage the external LDAP. To do so, I assume you may need to assign the admin user to domain administrator group, or grant the Matillion admin group with domain user management privileges.