AWS Consulting Sydney

AWS GuardDuty – Intelligent Threat Detection

AWS GuardDuty

Amazon GuardDuty is an intelligent cloud scale managed threat detection service, but the easiest way to think of it is as an AWS cloud infrastructure focused intrusion detection system. There is no prevention and protection – it is continuous security monitoring and threat detection focused on cloud infrastructure.  

GuardDuty does not replace your IDS (HIDS, NIDS), IPS, or SIEM but enriches them with heavy uplifting of log analysis and threat intelligence and provides an optional mechanism (Cloudwatch) to take action.  

In this post I share a summary of what you need to know about this service – whether you’re a technical expert or managing the process – based on studying public AWS documents and practical experience using AWS GuardDuty. 

Why GuardDuty?

It is one of the completely managed security services provided by AWS. AWS is proactively communicating with customers, sharing with them the actionable intelligence about their cloud infrastructure. Some highlights of the service are: 

  • Zero operational impact: there is no extra infrastructure required to deploy for the service, and no need to think about sensors, network taps, threat intelligence, etc.  
  • No planning before buying: you get a 30-day free trial and an estimate of the cost per month. AWS has provided a cost-effective service to enrich your cloud infrastructure security monitoring.  
  • Well defined scope: GuardDuty does not interact with any of your cloud resources and does not profile any customer. It also does not monitor data access or user behavior. It purely focuses on cloud infrastructure. 
  • No configuration required: GuardDuty is saying what AWS thinks about your cloud infrastructure security – you either enable it or not. It gives you the security alerts and mechanism to automate, with no configurations needed. 
  • Known/Unknown threats (Zero-days): GuardDuty has integrated threat intelligence from leading providers , anomaly detection based on historical baselines, and behavioral analytics (machine learning engine) that applies a global baseline of known patterns in AWS cloud. 

In summary, there is no reason not to turn it on. 

How does GuardDuty work?

Just like every other IDS, it needs to collect and analyse logs and events. GuardDuty looks into three specific log sources: 

  • VPC Flow Logs: The netflow information including source/destination ip addresses, port number and protocol from private/public VPC traffic.  
  • Cloudtrail Logs: It includes everything that makes an API request inside the AWS platform, both management and detailed data logs. 
  • DNS Logs: It includes logs from Route 53 and Amazon provided DNS servers in your VPCs. 

You do not need to enable those log sources and services in your AWS environment or pay for them. AWS owns the infrastructure and has first-hand access to all the required logs; you just permit GuardDuty to analyse the logs for you.  

GuardDuty uses machine learning algorithms to analyse the logs and detect threats. We can categorize threats in two groups: 

Known patterns: (Rule based) 

  • Signatures, traffic patterns, actors from AWS threat feeds (e.g. connecting to malware server, sending spam, and brute forcing) 
  • Patterns that AWS has already determined through analysis of malicious activities in AWS cloud ( e.g. An IAM User disables CloudTrail, or changes password policy) 

Unknown patterns: (Heuristic based through machine learning) 

  • Anomalies or deviations from the historical baseline of a resource. GuardDuty is trained through machine learning algorithms to detect any activity that falls out of normal activity for a resource. (e.g. An EC2 instance starts communicating with remote server on an unusual server port with no prior history of such communications) 
  • Behavioral deviations from global AWS usage baseline statistics. Looking into service usage as a statistical sample of the whole AWS cloud. (e.g. your service usage shows anomalies compared to normalized patterns for VPCs) 

For known patterns, GuardDuty has integrated threat intelligence from AWS Security team and multiple external sources: 

  • Proofpoint ET intelligenceGuardDuty has seamlessly integrated this product to detect any surface threats hidden in traffic and activity between customer AWS instances to or from malicious sites and bad actors.
  • Crowdstrike Falcon:GuardDuty has seamlessly integrated the threat intelligence real time feed from Crowdstrike.
  • Custom Threat lists and White lists:Custom threat lists of known malicious IP addresses or CIDR lists can be added. It can be taken from alternate threat feed subscriptions (AlienVault, FireEye, OTX, etc) or from your own lists. Whitelists are your trusted IP lists that you exclude.  

Note: At the time of writing GuardDuty’s custom threat lists accepts IP addresses rather than domains or URLs

Whichever method is used, GuardDuty will create a very specific cloud related alert. These alerts, also known as GuardDuty Findings, are exported to AWS Cloudwatch service every 5 minutes (and updates to existing findings every 6 hours) which is the mechanism to trigger other automated actions based on it.  

Another important aspect of GuardDuty is that it is a regional service and respects data sovereignty laws and leaves all the security findings inside a region. To get the best out of GuardDuty you should enable it in all the regions. GuardDuty has also provided a mechanism to aggregate findings so you do not need to export findings from each region or account individually .  

Each GuardDuty account can have up to 100 member accounts and become a master to them. Using this hierarchical mechanism, an organization can create a single point of aggregation which is important for larger AWS deployments. 

GuardDuty findings 

GurdDuty currently has 33 findings that we can categorise  based on threat type (Known patterns vs Behavioral/Heuristic, purpose, and variant), log source (VFC Flow log, CloudTrail log, and DNS log) and affected resource type. 

I’ll provide some examples below to get better insights into what is found by GuardDuty. 

  • Example 1: Known threat, CloudTrail, IAM User 

Multiple logins for the same IAM User at the same time from different geographical locations imply potential unauthorised access to your AWS account. 

  • Example 2: Known threat, DNS Log, EC2 Instance 

Your EC2 instance communicates with a bitcoin mining pool. As far as you are not using your EC2 for mining, it shows a compromised EC2 instance.  

  • Example 3: Unknown Threat, VFC Flow Log, EC2 Instance 

EC2 instance start sending much traffic to a remote host and deviates from its historical baseline. It implies that EC2 instance might be compromised. 

  • Example 4: Unknown Threat, CloudTrail Log, IAM User 

An IAM User starts launching EC2 instances of a specific type and there is no history of such activity for that user. It implies that IAM user credentials might be compromised. 

You can find the link to all the finding types  of AWS GuardDuty at the related links part at the end of this post. 

Auto Remediation 

You can configure your SIEM security system to receive and categorise your GuardDuty through CloudWatch or create your own automation.

As an example, I have used a lambda function to automatically deactivate any access key associated with a high severity GuardDuty alert.

You can use GuardDuty’s SAMPLE findings feature to test the execution of lambda function. Create a rule in CloudWatch for AWS GuardDuty Findings and put the Target as this lambda function as target.  

Related Links:

GuardDuty Finding Types:
https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_finding-types.html 

  

Python Source:

#!/usr/bin/env python
from __future__ import print_function
import boto3
import json

def lambda_handler(event, context):
    print("Received event: " + json.dumps(event))

    try:
        if (     event['service']['serviceName'] == 'guardduty' 
             and event['resource']['resourceType'] == "AccessKey"
             and event['resource']['resourceRole'] == "TARGET"
             and event['severity'] >= 5 
            ):
            iam= boto3.resource('iam')
            access_details = event['detail']['resource']['accessKeyDetails']
            access_key = iam.AccessKey(access_details['userName'],access_details['accessKeyId'])
            response_status = access_key.deactivate()
            status_code = response_status['ResponseMetadata']['HTTPStatusCode']
            if status_code == 200:
                print("Key Disabled Successfully")
            else:
                print("Key deactivation failed")

            return event['title'] 
    except:
        pass
    return 'AccessKey not found'

Disclaimer

All data and information provided on this site is for informational purposes only. The Cloudten Blog makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information on this site & will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.