One of the great features of Redshift is it’s ability to leverage its MPP (massively parallel processing) ability to suck data from S3 using the inbuilt Redshift COPY command. Similarly the UNLOAD command allows data to be extracted from a Redshift database by way of a DB query to one or more S3 files. To protect your data in transit within AWS Redshift uses hardware accelerated SSL to communicate with Amazon S3 for COPY and UNLOAD operations.
However, until now the Redshift COPY and UNLOAD commands required you to embed access keys into the command line to enable access to S3 resources. This was awkward and care needed to be taken around how to include these access keys and where they should be stored if for example the COPY or UNLOAD operations were to be run from a script.
AWS have now announced that IAM roles can be used within the COPY or UNLOAD commands when they are called by quoting the role’s ARN. This removes the need to embed any access keys into the command itself. The steps required to use and IAM role with these commands are:
- Create an IAM role for use with the Amazon Redshift cluster
- Associate the IAM role with the Redshift cluster
- Include the IAM role’s ARN when you call the COPY or UNLOAD command.
More details on this new feature are available here
Cloudten specialise in designing, building and managing AWS environments according to security best practice please contact us for any questions around this new Redshift feature.