Auto tag EC2 instances using Amazon EventBridge and deploy Infrastructure with Github Actions

By

 

Use Case

Tagging AWS resources is very important when you want to automate some tasks based on the tags, like you want to install patch automatically based on the tags from SSM association or you want to get a list of resources based on the tags. There are many aspects where tagging the resources will be required. 

In this blog, I will be showing on automatically adding a tag to EC2 instances with platform OS version and Launched By details. This use case will be helpful when we want to target any API actions to multiple EC2 instances of particular OS type. Example, if you have multiple different flavors of Operating systems in your Organization and you want to apply emergency patch to a windows instance and if you don't have the tags then grouping and applying the patches would be tedious task. In this blog we will be focusing on the how to automatically tag an instances with OS tag. Let’s get started.

Amazon EventBridge

Amazon EventBridge is a serverless event bus service that makes it easy to connect your applications with data from a variety of sources. EventBridge delivers a stream of real-time data from your own applications, Software-as-a-Service (SaaS) applications, and AWS services and routes that data to targets such as AWS Lambda. You can set up routing rules to determine where to send your data to build application architectures that react in real time to all of your data sources.

Design Overview


Implementation

I have packaged all the above resources with Infrastructure as a code using CloudFormation to create AWS EventBridge, AWS Lambda and necessary IAM roles. Also in this blog, I will show you on how you can deploy this package using Github Actions. Below is the deployment process for our implementation.

In the deployment package, as a pre-requisite, CloudTrail Trail should be created. For more details on creating the Trail, please refer AWS official doc

I will walk you through the each steps on the implementation. Below is the high-level details on the IaaC and workflow:
  • CloudFormation template will create below AWS resources:
    • AWS EventBridge
    • IAM Role
    • Lambda
  • GitHub workflow job will run below steps
    • Validate the syntax of CloudFormation template
    • Static Analysis of the CloudFormation template
    • Create CloudFormation Stack
Lets begin in detail on the implementation with CloudFormation and workflow.

CloudFormation resources to create AutoTag EC2 workflow

CloudFormation template is packaged to implement the Autotag EC2 instances workflow, lets look into the each resources which we are creating from the template
  • In below resource, RuleForEC2tagging is the logical ID to create EventBridge rule and source is "aws.ec2" and event name is "RunInstances" and target will be Lambda which we I will be showing up in further resource section. This is pretty simple, in layman term we are creating an event bridge rule to trigger the lambda whenever an EC2 instance is launched 
    RuleForEC2tagging:
      Type: "AWS::Events::Rule"
      Properties:
        Name: !Ref EventBridgeRuleName
        Description: "While EC2 instance is launched, This rule will trigger"
        EventPattern:
          source:
            - aws.ec2
          detail-type:
            - AWS API Call via CloudTrail
          detail:
            eventName:
            - RunInstances
        State: "ENABLED"
        Targets:
          - Arn: !GetAtt AutotaggingLambda.Arn
            Id: "TargetFunctionV1"
  • In below resource, PermissionForEventsToInvokeLambda is the logical ID to provide the permission for lambda to invoke a function for EventBridge source. This is mandatory resource when you are creating via CloudFormation template 
    PermissionForEventsToInvokeLambda:
      Type: "AWS::Lambda::Permission"
      Properties:
        FunctionName: !GetAtt AutotaggingLambda.Arn
        Action: "lambda:InvokeFunction"
        Principal: "events.amazonaws.com"
        SourceArn: !GetAtt RuleForEC2tagging.Arn
  • In below resource, LambdaRole is the logical ID to create a IAM role for lambda to call the certain API actions. Here, we are using ec2 create tags and describe images API calls in lambda function and AWSLambdaBasicExecutionRole managed policy is required to put the logs to CloudWatch log stream 
    LambdaRole:
      Type: 'AWS::IAM::Role'
      Properties:
        RoleName: !Ref LambdaRoleName
        AssumeRolePolicyDocument:
          Version: 2012-10-17
          Statement:
            - Effect: Allow
              Principal:
                Service:
                  - lambda.amazonaws.com
              Action:
                - 'sts:AssumeRole'
        Policies:
          - PolicyName: root
            PolicyDocument:
              Version: 2012-10-17
              Statement:
                - Effect: Allow
                  Action:
                    - "ec2:CreateTags"
                    - "ec2:DescribeImages"
                  Resource: "*"
        ManagedPolicyArns:
          - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
  • Final resource is lambda creation part, where AutotaggingLambda is the logical resource to create a lambda function. Function is very simple, where I am getting the RunInstances event from EventBridge and I am further tagging the launched resource with below provided logic in the lambda handler 
    AutotaggingLambda:
      Type: AWS::Lambda::Function
      Properties:
        FunctionName: !Ref LambdaFunctionName
        Code:
          ZipFile: |
                    import json
                    import boto3

                    def lambda_handler(event, context):
                        try:
                            print(event)
                            items = event["detail"]["responseElements"]["instancesSet"]["items"]
                            region = event["detail"]["awsRegion"]
                            LaunchedBy = event["detail"]['userIdentity']['arn']
                            for item in items:
                                instanceid= item['instanceId']
                                amiid = item['imageId']
                                ec2client = boto3.client('ec2', region_name = region)
                                response= ec2client.describe_images( ImageIds=[amiid])
                                AMI_Name= response['Images'][0]['Name']
                                ec2client.create_tags(
                                    Resources=[
                                        instanceid,
                                    ],
                                    Tags=[
                                        {
                                            'Key': 'OS_Name',
                                            'Value': AMI_Name
                                        },
                                        {
                                            'Key': 'LaunchedBy',
                                            'Value': LaunchedBy
                                        },
                                    ]
                                )
                        except Exception as e:
                            print(e)
        Description: Custome AMI Tagging
        Handler: index.lambda_handler
        MemorySize: 128
        Role: !GetAtt LambdaRole.Arn
        Runtime: python3.7
        Timeout: 900


Github Workflow to deploy the CloudFormation template using Github Actions

As a pre-requisites, in order to deploy your resources in AWS, you will have to pass the user credential in Github secrets as shown below

Create a programmatical IAM user from AWS console, please refer AWS official doc for the steps to create an IAM user. Add the secret keys and access secrets keys in GitHub secrets.
  • Lets look into the GitHub workflow. As per below workflow, GitHub actions will automatically execute when you push or create the pull request to main branch 
    name: 'Deploy to AWS CloudFormation'
    on:
      push:
        branches: [ main ]
      pull_request:
        branches: [ main ]
  • actions/checkout@v2 action checks-out your repository under $GITHUB_WORKSPACE, so your workflow can access it. For more details on checkout action visit GitHub official doc 
    steps:
      - name: Checkout
        uses: actions/checkout@v2
  • ScottBrenner/cfn-lint-action@1.6.1 GitHub Action used for CloudFormation Linter. More details can be found in GitHub page 
    - name: cfn-lint-action
      uses: ScottBrenner/cfn-lint-action@1.6.1
      with:
        args: "cloudformation/**/*.yaml"
  • minchao/cfn-nag-action@v0.1 is cfn-nag tool which looks for patterns in CloudFormation templates that may indicate insecure infrastructure.  
    - name: cfn-nag-action
      uses: minchao/cfn-nag-action@v0.1
      with:
        args: "--input-path cloudformation/"

In Layman terms, it will look for:

    • IAM rules that are too permissive (wildcards)
    • Security group rules that are too permissive (wildcards)
    • Access logs that aren't enabled
    • Encryption that isn't enabled
    • Password literals
  • actions/upload-artifact@v1 uploads artifacts from your workflow allowing you to share data between jobs and store data once a workflow is complete. Here, we are using to keep the artifacts which is CloudFormation in Github Artifacts. More details can be looked in GitHub page 
    - name: cfn-artifacts
      uses: actions/upload-artifact@v1
      with:
        name: cloudformation-artifacts
        path: cloudformation/
  • aws-actions/configure-aws-credentials@v1 is used to configure AWS credential and region environment variables. More details can be looked in GitHub page 
    - name: Configure AWS Credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.ACCESS_KEY }}
        aws-secret-access-key: ${{ secrets.ACCESS_KEY_SECRET }}
        aws-region: ap-south-1
  • aws-actions/aws-cloudformation-github-deploy@v1 is used to deploy the AWS CloudFormation Stacks. More details can be looked in GitHub page 
    - name: Deploy to AWS CloudFormation
      uses: aws-actions/aws-cloudformation-github-deploy@v1
      with:
        name: ec2-auto-tag-stack
        template: cloudformation/ec2-auto-tag.yaml
        no-fail-on-empty-changeset: "1"
        parameter-overrides: >-
          LambdaRoleName=ec2TaggingRole,
          EventBridgeRuleName=ec2TaggingRule,
          LambdaFunctionName=ec2TaggingFunction
        Capabilities: CAPABILITY_NAMED_IAM
You can clone my GitHub repo  and deploy this setup in your environment 

Now it's time to test the deployment. Lets push the code to the repo and check the workflow. I pushed the code and created the pull request from feature branch to main branch, my workflow automatically triggered
You can see, workflow triggered and each step executed sequentially one after the other, if syntax is wrong in CloudFormation then cfn-lint-action will fail and trigger an email to you and on the other step if you have given over-permissive in your role then cfn-nag-action step will fail. After validation, we can see stack executed in the given AWS account and region. you can take this workflow as a reference and build your infrastructure.

Conclusion

In this blog post, I have shown you how to tag your EC2 instances with "OS version" and "Launched By" key with its respective values using Lambda and Event bridge. I also showed you on how the whole design can be packaged in CloudFormation. Following this, I showed you how to deploy the CloudFormation using Github Action by using steps with cfn-lint, cfn-nag and cfn-deploy. Hope this blog helped you in your similar use case.

Thank you for reading!

Comments

Post a Comment

Popular posts from this blog

Connect to Linux EC2 Instance if Key pair is lost after Initial Launch

By
Use Case We know that EC2 Linux instances are accessible through the private keys by default. However, SSH is allowed but you cannot use SSH password authentication to access Linux instance as it is disabled by default. So, what would happen if you lose the private key of your Linux instance? In this blog, I will walk you through the easiest ways to login when you lost the Key pair What is a key-pair file? Key-pair is a combination of a public and private key. Amazon EC2 uses public-key cryptography to encrypt and decrypt login information. Private key is only the way to get access to the instance, what if you lost the key? Is your servers lost in black hole? Thankfully nothing is lost you still can access your server, let me show you how can you solve this problem. Things that you should know before to proceed to this topic This procedure will requires s
Read more

Setup Grafana on AWS EKS and integrate with AWS Cloudwatch

By
What is Grafana? Grafana is open-source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations. Setup Grafana on Amazon Elastic Kubernetes Service This blog explains on how to run Grafana on Amazon Elastic Kubernetes cluster and adding cloudwatch as datasource to Grafana. Using Grafana you can simplify Kubernetes monitoring dashboards from Clo
Read more

Start or Stop services in multiple Windows EC2 Instances using AWS Systems Manager

By
Use Case You are the admin over a large amount of EC2 instances and managing different AWS accounts under Organization and you have to start/stop/disable a service for all Windows EC2 Instances in the Organization. Rather than wasting labor hours to perform this activity in each instance manually, you want to automate the process using AWS Systems Manager. Recently Windows had a flaw in Print Spooler Remote Code Execution Vulnerability and Microsoft recommended to stop until the patch is applied, in order to apply this immediately, we can make use of this design Systems Manager Systems Manager (SSM) gives you visibility and control over your AWS infrastructure. Automation, a capability of AWS Systems Manager, simplifies common maintenance and deployment tasks of Amazon Elastic Compute Cloud (Amazon EC2) instances and other AWS resources. Automation helps you to do the following: Build automations to configure and manage instances and
Read more

Concourse CI Installation and Configuration in Windows

By
Introduction to Concourse Concourse is an open source CI/CD system with integration supported to Resource types in the outside world. Concourse's principles reduce the risk of switching to and from Concourse, by encouraging practices that decouple your project from your CI's little details, and keeping all configuration in declarative files that can be checked into version control. Install and Configure Concourse in Windows Follow the below Steps to Install Installing Docker Desktop is pretty simple, download the exe file and install the docker desktop Modify Docker daemon configuration file as below
Read more

Install SSM Agent in Amazon EC2 Instance

By
What is SSM Agent AWS Systems Manager Agent is a Amazon software which can be installed on Amazon EC2 Instances. If SSM Agent is Installed in the servers then it will be easy to manage the EC2 Instances through AWS Systems manager.  By default SSM agent is pre installed for below Amazon Machine Images Amazon Linux Amazon Linux 2 Amazon Linux 2 ECS-Optimized Base AMIs macOS 10.14.x (Mojave), 10.15.x (Catalina), and 11.x (Big Sur) SUSE Linux Enterprise Server (SLES) 12 and 15 Ubuntu Server 16.04, 18.04, and 20.04 Windows Server 2008-2012 R2 AMIs published in November 2016 or later Windows Server 2016 and 2019 In this blog, I will walk you through on the ways to Install SSM Agent in the Instances which will not get SSM Agent by default. Steps to Install S
Read more

Deploy AWS infrastructure using Terraform and GitHub Actions

By
Terraform being an Infrastructure as a Code helps us to manage a lot of infrastructure for several platforms in a consistent manner.  In this blog, I will walkthrough on deploying a terraform code using  Github actions Terraform code to create S3 bucket In this blog, I will walkthrough each blocks in terraform to create an S3 bucket in AWS in automated workflow using GitHub Actions. Lets look into the Terraform files. To understand easily, I have created separate tf files for each purpose, but you can still use all in one tf file as per your convenient. You can use any name with tf extension, for better understanding I used following names. versions.tf
Read more

Automate Permission Boundary Attachment to IAM roles and Users

By
When we are managing multiple AWS Accounts in an Organization and each accounts will have their own requirement but as a centralized admin, we should be having a certain governance and compliance framework. IAM is a widely used AWS service and IAM roles/users are common used entity in AWS. As a centralized Admin, when we provide access to create IAM roles, cloud consumers in the landing zone accounts would create Users and Roles and attach Administrative policy and bypass the governance framework. In this blog, I will be briefing on restricting this scenario using IAM permission boundary and also deploying this setup through automation. AWS supports permissions boundaries for IAM entities (users or roles). A permissions boundary is an advanced feature for using a managed policy to set the maximum permissions that an identity-based policy can grant to an IAM entity. An entity's permissions boundary allows
Read more

AWS Route 53 Inbound Resolver to resolve DNS for Multi Account Organization

By
Amazon Route53 Resolver The route 53 resolvers are contactable IP addresses (endpoints) where the DNS queries from different sources can be directed. There are two types of resolvers that administrators can deploy. Inbound Resolver As the name says inbound (DNS Resolutions coming in) which will enable your resources in the on-prem network to resolve AWS resource domain names or records in a Route 53 private hosted zone by allowing your on-prem network DNS resolvers to forward queries to Route 53 Resolver. Outbound Resolver As the name says Outbound (DNS Resolutions going out) which will enable your AWS resources to resolve the domain names of your on-prem network resources using resolver rules, which would forward selected queries to the on-prem network DNS resolvers On-prem to AWS DNS Resolution for
Read more

Import Existing Resources using Terraform

By
In this blog, I will walk you through the ways to import pre-existing cloud resources in terraform. Why Import is required? There will be a scenario, where you will have a stringent timeline to setup a cloud Infrastructure and it can lead to situations where infrastructure needs to be created manually due to time pressures, emergency releases or just the fact that the infrastructure exists, and terraform was never used in the first instance. In a worst case scenario, you can lose the terraform.tfstate file. Terraform import will be essential in most of all these scenarios. Let's Import With an understanding of why Import is required, let us begin by importing a simple AWS resource S3. Firstly, we need to setup the Terraform and AWS credentials locally, we will not go into the details of installing and setting AWS credentials in
Read more

Hosting AWS VPC Interface Endpoints in Shared Model

By
What is VPC Endpoint A VPC endpoint enables private connections between your VPC and supported AWS services. Endpoints are classified as Interface and Gateways endpoints. In this blog, I will be showing how to host interface endpoints in shared model Do you have Multi AWS Accounts in Organization? When you have multi accounts structure in your Organization then creating endpoints in each accounts will increase the costs and IPs, in this blog I will explain how you can deploy endpoints in Shared Services Account and associate the member accounts VPCs with Private Hosted zone. Below is the Architectural diagram which I will be showing in detail on how to setup and also how the flow works Architecture Design AWS Services/Components required VPC Subnets Endpoints Hosted zone and Record set Transit Gateway I
Read more