Amazon EKS Autoscaling Based on Cluster Metrics

One of the many advantages that Kubernetes has to offer is auto-scaling. This enables the automatic provisioning of resources based on traffic/demand. As the traffic demand increases more resources are provisioned and vice versa. Kubernetes offers three types of scaling:

• Vertical Pod autoscaler: Scaling that increases/decreases the CPU or memory allocated to Pods.
• Horizontal Pod autoscaler: Increase/Decreases the number of pods per service in response to traffic/demand.
• Cluster autoscaler: Scales your cluster nodes based on the number of pending pods.

Running Kubernetes on AWS (EKS) offers the option to scale your cluster nodes based on metrics from your cluster resource utilization. This article guides the reader on configuring auto-scaling based on cluster metrics. We will use the Cluster Node Memory utilization metric to scale our nodes.

N/B: This is only possible on an EKS cluster with a managed node group (Provisioned Worker Nodes) and not an EKS Fargate cluster. With Fargate there is no need to provision instances for your cluster because it provides compute capacity for your containers. It also automatically scales for you, resources based on traffic demands.

Requirements/Prerequisites

• An AWS Account.
• Created a User on the account with Permissions to provision resources on the account and Access to CloudWatch.
• Provisioned an AWS EKS Cluster with a managed Node Group.
• Enabled Container Insights for your EKS Cluster by deploying CloudWatch Agent on your cluster.

Part 1: Create the AWS EKS SNS Topic & Subscriptions

We will create two SNS Topics, The Scale Up and Scale Down Topics and add email subscriptions to the topics. The below CloudFormation Templates can be used for this.

The Scale Down Topic and Subscriptions Template.

AWSTemplateFormatVersion: "2010-09-09"
Description: "Template to create SNS Topic and Subscriptions"

Parameters:
    Name:
        Type: String
        Description: The Topic Name
        Default: EKS_CLUSTER_MEMORY_LESS_THAN_50

Resources:
    Email1Notification:
        Type: AWS::SNS::Subscription
        Properties:
           Endpoint: [email protected]
           Protocol: email
           TopicArn: !Ref ScaleDownTopic

    Email2Notification:
        Type: AWS::SNS::Subscription
        Properties:
           Endpoint: [email protected]
           Protocol: email
           TopicArn: !Ref ScaleDownTopic

    ScaleDownTopic:   
        Type: AWS::SNS::Topic
        Properties: 
           Tags: 
              - Key: Name
                Value: !Ref Name
              - Key: createdBy
                Value: Maureen Barasa
              - Key: Project
                Value: test-blog
           TopicName: !Ref Name

Outputs:    
    ScaleDown:
        Description: The scale down topic
        Value: !Ref ScaleDownTopic
        Export:
          Name: "EKSScaleDown"
          Value: !Ref ScaleDownTopic

N/B: The user should customize the template. The names of Resources, Email endpoints and Tags should match the user’s specific requirements.

The Scale Up Topic and Subscriptions Template

AWSTemplateFormatVersion: "2010-09-09"
Description: "Template to create SNS Topic and Subscriptions"
Parameters:
    Name:
        Type: String
        Description: The Topic Name
        Default: EKS_CLUSTER_MEMORY_MORE_THAN_70

Resources:
    Email1Notification:
        Type: AWS::SNS::Subscription
        Properties:
           Endpoint: 
           Protocol: email
           TopicArn: !Ref ScaleUpTopic

    Email2Notification:
        Type: AWS::SNS::Subscription
        Properties:
           Endpoint: 
           Protocol: email
           TopicArn: !Ref ScaleUpTopic

    ScaleUpTopic:   
        Type: AWS::SNS::Topic
        Properties: 
           Tags: 
              - Key: Name
                Value: !Ref Name
              - Key: createdBy
                Value: 
              - Key: Project
                Value: 
           TopicName: !Ref Name

Outputs:
    ScaleUp:
        Description: The scale up topic
        Value: !Ref ScaleUpTopic
        Export:
          Name: "EKSScaleUp"
          Value: !Ref ScaleUpTopic

N/B: The user should customize the template. The names of Resources, Email endpoints and Tags should match the user’s specific requirements.

• is your first email address e.g [email protected], [email protected]
• is the second email address e.g [email protected]
• is the name of a Project to which the Topic applies

Under Outputs we have allowed our values to be exported so we can use them in the template that will create the CloudWatch Alarms.

Once the SNS Topics and Subscriptions are created, go to the given email endpoints and confirm the subscription. We are now ready to create the Scaling Policies and CloudWatch Alarms.

Part 2: Create Scaling Policies and Attach to your AWS EKS NodeGroup ASG

Next, we will create two policies and attach them to the EKS nodegroup autoscaling group. Use the below Cloud formation Template.

AWSTemplateFormatVersion: "2010-09-09"
Description: "create a scaling policy and apply it to EKS ASG Group"

Resources:
  ScaleDownPolicy: 
    Type: AWS::AutoScaling::ScalingPolicy
    Properties: 
      AdjustmentType: "ChangeInCapacity"
      PolicyType: "SimpleScaling"
      Cooldown: "300"
      AutoScalingGroupName: "eks-ASG-GroupName"
      ScalingAdjustment: -2

  ScaleUpPolicy: 
    Type: AWS::AutoScaling::ScalingPolicy
    Properties: 
      AdjustmentType: "ChangeInCapacity"
      PolicyType: "SimpleScaling"
      Cooldown: "300"
      AutoScalingGroupName: "eks-ASG-GroupName"
      ScalingAdjustment: 2

Outputs:
    ScaleUp:
        Description: The scale up policy
        Value: !Ref ScaleUpPolicy
        Export:
          Name: "EKSScaleUpPolicy"
          Value: !Ref ScaleUpPolicy

    ScaleDown:
        Description: The scale down policy
        Value: !Ref ScaleDownPolicy
        Export:
          Name: "EKSScaleDownPolicy"
          Value: !Ref ScaleDownPolicy

Part 3: Create the AWS EKS CloudWatch Alarms

Finally, we will create the CloudWatch Alarms. Here, we will use values created from the above templates. i.e. The SNS Topics and the Scaling Policies.

On the CloudWatch console, select alarms. Then click create alarms.

Under Select Metric, choose “Container Insights” Namespace then select by Cluster name. Under your EKS Cluster name, choose node_memory_utilization metric and click select metric. Next, specify Metric Conditions. For Period one can choose a period from a minute to 1 day. It is the time taken to analyze the metric or expression for creating individual data points for an alarm.

Next, we define the conditions for the alarm. For the first case, we will create an Alarm, where node memory utilization is greater than 70. Click next.

Then we configure actions to take when the alarm is triggered. For, our case our first action is to send a notification to the respective SNS topic created in part 1 above. For example, when the memory utilization average is greater than 70, it should send a notification to EKS_CLUSTER_MEMORY_MORE_THAN_70 SNS topic.

The 2nd action to take will be to scale up based on the scaling policies we defined in part 2 of this article. Ensure to select the EKS cluster node, scaling group. For the high memory utilization alarm, choose the scale-up policy created and click next.

Finally, you create a Name and Description for the Alarms as below and click next. After carefully evaluating your configurations, you can now click, create alarm.

Equally important is to create the alarm for low memory utilization, follow the same steps as above. This time ensure that for the alarm, under conditions, we should select the relevant SNS topic and scaling policies. For example, use the EKS_CLUSTER_MEMORY_LESS_THAN_50 SNS topic and the scale-down policy for scaling actions.

You have now successfully configured EKS cluster auto-scaling using CloudWatch Alarms and SNS topics.

Important Links:

• AWS Fargate Documentation
• AWS EKS Documentation

Other Kubernetes Guides:

• How To Configure Kubernetes Dynamic Volume Provisioning With Heketi & GlusterFS
• Install Kubernetes Metrics Server on Amazon EKS Cluster
• Grant Developers Access to EKS Kubernetes Cluster
• Kubectl Cheat Sheet for Kubernetes Admins & CKA Exam Prep

Happy Building!!!