Posting Cloudwatch metrics on Slack with Lambda

In this post I will take you through the general steps and high level explanation on whats needed to automatically post messages on Slack based on AWS Cloudwatch metrics, using Lambda function, Cloudwatch event rules and Slack api.

Assumed Knowledge:

General use case

  1. You want to be automatically notified on your team’s Slack channel, the state of your current EC2 instances. i.e You want to quickly take action if your servers are being stopped or terminated.
  2. You want to be automatically notified if the CPU/Memory of a particular instance has passed your given threshold.
    EC2 state notification

    EC2 state notification

    Clopudwatch metric graph on Slack

    Cloudwatch metric graph on Slack (Cpu Utilization)

     

     

     

     

 

Architecture

Below I’ve drawn a digram to indicate the different components within the architecture and the flow of metrics from EC2 instances to Slack Channel

Architecture Diagram

 

 

 

 

 

 

 

 

 

 

Step 1  –  Create Slack App and Generate Slack Api token

Step 2  –  Implement node.js function

Step 3  –  Upload package to AWS Lambda

Step 4  –  Create Cloudwatch alarm on AWS and assign to our lambda function

Step 5  –  Brace yourself, for the incoming notifications on Slack

 

Step 1

Before implementing our our nodeJs function, we first need to create a slack app in order to retrieve a slack api token to allow us to post messages on our slack channels through their api

PS: You can also generate a legacy token for development / testing purposes (Quicker way)

So first sign in to your slack account then proceed to Slack-Apps

Next click on Create Slack App and then type a name for your app, choose your team from the dropdown and click on Create App

 

 

 

 

 

Proceed on to basic Information of your app and under credentials you will find your api token we will be storing this as our environment variable for our nodeJs function

 

 

 

 

 

Step 2

We start off by first creating a fresh node.js project and installing the required dependencies based on our package.json and creating .env file to store our slack api token and aws credentials, for testing locally

For this example we will be mainly using four node modules:

Once you run npm install, you will have node_modules folder created on your project folder and you can follow my sample project structure and code as seen below

First, I require the modules that I have installed through node package manager and I then initialise my configurations for AWS, Slack and Cloudwatch Chart, such as access keys, tokens, channel name, chart width, height etc.

Next I write a simple wrapper function for the slack api module which I then execute in my main function to post messages on slack based on the input channel, message, icon and options (attachment for sending images)

The message that is given as input will be that of Cloudwatch event details as shown below

Documentation on chat post slack message can be found here

Code Sample

Sample project structure and code

Function wrapper for slack api

Main Function

 

Before adding features, always remember to start off  simply by first testing locally with a simple post message to slack.

i.e postToSlack(‘yourslackchannel’,’testing’,’:bell:’) 

Executing this function, should post the message testing on yourslackchannel with a bell icon.

 

Step 3

Now that our function is completed we just need to zip up our index.js and node_modules folder and upload it to AWS lambda

on terminal you can simply run `zip -r aws-slack.zip index.js node_modules/` within your project folder

Next we login to our AWS console and upload the code that we just packaged to Lambda.

 

Step 4

You will notice nothing will be happening after you upload the package because your lambda function needs a trigger.

For our example we will be triggering our lambda function thorugh cloudwatch event.

So first, from your AWS console search Cloudwatch and click on rules on the left, then click on Create Rule

Next, under Event Source, we will be selecting Event Pattern and select EC2 from the drop down list since our goal is to check on ec2 instance state.

Now select EC2 Instance state change notification for the event type.

Once selected, leave the next radio boxes Any state and Any Event selected unless you want to narrow down to a specific instance and a specific state.

Next click on Add Target, choose Lambda function and the name of your lambda function from the dropdowns, in my case its aws-slack

Finally, click on Configure details, give your rule a name and description and click finish off by clicking on Create Rule.

If you go back to the Cloudwatch rules page, you will now see your rule listed there. You have now successfully created your Cloudwatch rule.

Cloudwatch Event Pattern

Target Lambda Function

 

 

 

 

 

 

 

You can then check that this event trigger is added, by navigating to your lambda function in your AWS console as shown below

Event Trigger

Event Trigger

 

 

 

 

 

 

Step 5

To recap, our cloudwatch rule was created on ec2 instance state pattern. So our lambda function will be triggered if any of our ec2 instances are in either of these states [pending, running, stopped, terminated].

A simple way to test is to stop a running instance that you have been testing on or launch a new instance and you should see the messages flow onto your slack channel as per your configuration in your lambda function.

I hope this gives you a basic idea on how you can make use of lambda function, cloudwatch event rules and slack api to automate posts onto your team’s Slack channel on critical information about your AWS infrastructure.

Disclaimer

All data and information provided on this site is for informational purposes only. The Cloudten Blog makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information on this site & will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.