Configure AWS VPC Flow logs to CloudWatch Log group

In this short tutorial we will be discussing how you can enable VPC flow logs in your AWS Account. VPC Flow Logs is an AWS feature which makes it possible to capture IP traffic information traversing the network interfaces in the VPC. We will configure publishing of the collected data to Amazon CloudWatch Logs group but S3 can also be used as destination. Since flow log data is collected outside of the path of your network traffic, it does not affect network throughput or latency.

With access to VPC flow logs you can:

• Monitor the traffic that is reaching your instance
• Determine the direction of the traffic to and from the network interfaces
• Diagnose overly restrictive security group rules

Flow logs can be created or deleted without any risk of impact to network performance. This can be done on a VPC, a subnet, or a network interface. When enabled at VPC or Subnet level each network interface in that subnet or VPC is monitored.

Before you begin you need an installed and configure AWS CLI. Refer to our article below for complete how to article.

Install and Use AWS CLI on Linux – Ubuntu / Debian / CentOS

Confirm your AWS CLI is working by getting identity.

aws sts get-caller-identity

You also need Administrative privileges in your AWS Account to complete this tutorial. Access to AWS Console is also important for data visualization.

Step 1: Create IAM Policy and Role

The first step is creation of an IAM role that will enable service to act on our behalf for logs forwarding. This action can be done on AWS console or from CLI. This is an IAM role to publish flow logs to the CloudWatch log group.

Before Role creation you need an IAM Policy which can be created on IAM > Policies > Create policy > JSON. Paste below data.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents",
                "logs:DescribeLogGroups",
                "logs:DescribeLogStreams"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Give a policy a name and complete creation.

To create IAM role go to Identity and Access Management (IAM) > Roles > Create role. For trusted entity type choose “AWS service” and “EC2”

Click Next to attach Permissions. Under “Attach permissions policies” page, select the policy you created before.

Give a role a name to complete creation. I’ll name mine IAM-PublishFlowLogs, same name as policy.

Copy the Role ARN and save it somewhere.

Step 2: Edit Trust Relationships

You will also need to Enable trust relationship so that the IAM role can access the CloudWatch Log group. Go to AWS Roles and click on the role you created in Step 1.

Click “Edit Trust Policy” to update the policy.

Add below contents:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
         "Service": "vpc-flow-logs.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Click “Update Trust Policy” button to update the relationship.

Step 3: Create CloudWatch Log group

The CloudWatch log group defines where the log streams are recorded. It is created on Services > CloudWatch > Logs > Actions> Create log group.

Hit the Create button to action.

Step 4: Enable AWS VPC Flow Logs

To enable flow logs select Services > VPC > Your VPCs > YourVPCName > Create flow log

Fill all required information:

• Give flow log a name
• Select traffic filter type
• Aggregation interval
• Destination – Can be CloudWatch Logs or Amazon S3 bucket
• Destination Log group in CloudWatch
• IAM role with permissions to publish to selected Log group
• Log Format

My settings are as shown in the screenshot below.

Hit the Create flow log button to complete the setup. You can confirm if creation was successful by listing available Flow Logs.

Clicking on Destination name link should take you to the Log group where you can filter log streams. You can also enable from the CLI:

aws ec2 create-flow-logs --resource-type VPC --resource-ids  --traffic-type ALL --log-group-name  --deliver-logs-permission-arn 

More Articles on AWS:

Easily Setup Kubernetes Cluster on AWS with EKS

Create AWS IAM Users & Groups with AWS CLI

How To Rename IAM User name on AWS

Learning materials: