Bastion hosts have long been a fixture in secure infrastructure implementations. Their role as a secure, hardened sentinel allowing access to your underlying servers has become yet more indispensable in the cloud era. As always in the cloud, we have been looking for abstractions that can simplify and give us this much need functionality without the overhead of managing the jumphosts themselves.
Lately with the advent of SSM, AWS had come up with some alternative mechanisms which provide similar functionality to jumphosts.
The challenge with this approach is the lack of session logs and also real time debugging can be difficult. AWS has yet again proved that it listens to its customers with another service that fills this gap.
So what is AWS Session Manager? It is a managed service that can provide interactive browser based shell access to your instances in the cloud. Some of its impressive features:
- Access without opening any SSH port. Session Manager interacts using the SSM agent
- No more keys to manage
- It removes all the complications of certificate and SSH key rotation/management in the concept of ephemeral SSH.
- Auditing capability – All the screen commands are copied to S3 AND/OR Cloudwatch logs which can be used for security audit
- Its one solution for Windows and Linux
Now lets jump into a real world scenario and see its applicability. Many enterprises have a requirement for their developers/operations team to login to EC2 instances and skim through application/infra logs to debug an error or gauge health of the server.
There could be yet another group of people who would need the access to make any configuration changes to the server in the way of configurations or hardening. And then there is the super admin category mostly security/superadmin team who has near root level access to the server to change credentials/introduce new groups etc.
Let’s walkthrough this for a linux server as an example, this is achieved through user creation groups and permissions. Though privileged access management (PAM) is widely practiced, potential issues can arise due to elevated access. What I am going to discuss below is a simple implementation of AWS Sessions Manager which will help arrest misuse of elevated access and we will see together how it can make the environment more secure.
So as explained above, the below table, gives the list of users that is allowed in my fictional company.
|Linux user||Used By||Used For|
|ops||Operations Team||Read Only to examine logs for troubleshooting|
|infadmin||App Team/ AWS Admin||Any installation, Infra hardening, Configuration of files|
|supadmin||Super Admin/Sec||Creating new user and setting group policy, Change Sudoers file, Setting/Changing passwords|
With root disabled (default in most organisations) a user who signs in as ssm-user has to first switch role as “ops” user and then elevate to other users “infradmin” and “supadmin” as per his role and task he intends to do. This elevation of access is something that needs to be tracked and audited to understand what the user is supposed to do on the server so lets put our controls on top of that.
A simple implementation as below will help the whistle blowers in the company to take necessary action to thwart spurious activity on the servers. So first thing for us is to enable the Cloudwatch logging capability for AWS Session Manager. I have defined a Cloudwatch log group “MySessionsLogs” which will record all the user activity in my servers by other users.
Below is a sample of how it looks:
Now if we set up Metric filters for the commands that we need to monitor most, we have achieved our objective of being able to track who’s firing the most privileged commands.
So lets see that in action, in our case elevating the user is the key trigger point for us so we will set up the filters around monitoring “su infradmin”.
The metric filter will be tied to a Cloudwatch alarm which will alert the concerned team of any suspicious activity. Here’s how it looks on screen.
This is a very simple but potent way of tracking the commands that matter most to you from a security and auditability perspective. It is completely serverless and works at scale. Configurations are easy and expandable. Response to any security event can be easily done through SSM as it has reach to all your servers which can include password resets or terminating user sessions. Lets try to embrace this service and keep getting more secure in the cloud…!