Grant Developers Access to EKS Kubernetes Cluster

In the previous article we covered installation steps of an EKS Cluster in Amazon Cloud platform. Amazon Elastic Kubernetes Service (Amazon EKS) is a fully managed Kubernetes service which takes away the headaches of control plane management including etcd from SysAdmin. Upgrading a cluster can be easily done with a click of a button. In this short guide I’ll walk you through the process of granting Developers access to Kubernetes namespace using IAM policies and Kubernetes native RBAC.

For this guide to favor you some pre-requisites which should be ticked are:

• A Working EKS Cluster: Check installation of an EKS Cluster
• Working AWS CLI configuration: Install and Use AWS CLI on Linux
• IAM User with required administrative permissions
• Access to AWS Web Console for management. The same operations can be done fully in CLI but we’ll use both.

Before you get started

Confirm you can list you cluster from you local machine or Bastion server which can access EKS Control Plane.

$ eksctl get cluster
NAME			REGION
prod-eks-cluster	eu-west-1

Or with aws command.

$ aws eks list-clusters

If your command doesn’t return any output check if you’re using correct credentials and region.

From Web Console:

By default only the creator of the Amazon EKS cluster has system:masters permissions which unlocks all Kubernetes cluster operations to be executed from kubectl. To extend the functionality so other users can access the cluster, aws-auth ConfigMap will be modified.

We’ll need to create an IAM role with AWS Security Token Service (STS) permissions which allows users to request temporary, limited-privilege credentials.

Step 1: Create an IAM role

Next is to create a role on IAM > Roles > Create role.

Select “Another AWS Account” as the type of trusted entity:

Don’t attach any permission

Add appropriate tags.

Give a role a name and create it using the “Create role” button at the bottom. Mine will be called k8s-devs-role, this is the same name set in Policy creation.

Step 2: Create IAM Policy with STS assume role permissions

Login to your AWS web console and navigate to IAM > Policies > Create policy > JSON and paste below json contents replacing with your AWS Account ID and k8s-devs-role with the name of the role you created.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam:::role/k8s-devs-role"
        },
        {
            "Effect": "Allow",
            "Action": [
                "eks:DescribeCluster",
                "eks:ListClusters"
            ],
            "Resource": "*"
        }
    ]
}

Example for account ID 293759796572

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sts:AssumeRole",
            "Resource": "arn:aws:iam::293759796572:role/k8s-devs-role"
        },
        {
            "Effect": "Allow",
            "Action": [
                "eks:DescribeCluster",
                "eks:ListClusters"
            ],
            "Resource": "*"
        }
    ]
}

Give the policy a name and hit the create button.

Step 3: Create an IAM user group

Create an IAM group named k8s-devs

Attach the policy created in Step 2.

Review the policy and complete creation process.

Step 4: Add IAM Users to the group

Our last step is adding IAM users who need access to Kubernetes Cluster to the group we just created.

Go to IAM > Groups > k8s-devs > Add Users to Group to add users to the group.

Add all other users you want to grant access to.

Step 5: Create Kubernetes RBAC for Developers

Role-based access control (RBAC) is a method of regulating access to computer or network resources based on the roles of individual users within your organization. The RBAC API declares four kinds of Kubernetes object: Role, ClusterRole, RoleBinding and ClusterRoleBinding.

I’ll start by creating three namespaces that I’ll give developers access to: prod, uat and dev

$ kubectl create namespace prod
namespace/prod created

$ kubectl create namespace uat
namespace/uat created

$ kubectl create namespace dev
namespace/dev created

List all namespaces to confirm creations.

$ kubectl get ns
NAME              STATUS   AGE
default           Active   25d
dev               Active   9s
istio-operator    Active   14d
istio-system      Active   14d
kube-node-lease   Active   25d
kube-public       Active   25d
kube-system       Active   25d
monitoring        Active   17d
prod              Active   40s
uat               Active   26s

Creating Cluster Role

I’ll create a cluster role called dev-full-access whose manifest file looks like below:

kubectl apply  -f - <<EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-full-access
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: ["batch"]
  resources:
  - jobs
  - cronjobs
  verbs: ["*"]
- apiGroups: ["autoscaling"]
  resources: ["*"]
  verbs: ["*"]
EOF

Expected output:

clusterrole.rbac.authorization.k8s.io/dev-full-access created

Confirm settings:

$ kubectl get clusterroles dev-full-access -o yaml

Creating RoleBindings

Next is creation of group role binding for our Developers. The group is called k8s-devs

# Access to Prod namespace
NAMESPACE="prod"
CLUSTERROLE="dev-full-access"

kubectl apply  -f - <<EOF
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-full-access
  namespace: $NAMESPACE
subjects:
- kind: Group
  name: "k8s-devs"
  apiGroup: rbac.authorization.k8s.io
  namespace: $NAMESPACE
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: $CLUSTERROLE
EOF

# Access to UAT namespace
NAMESPACE="uat"
CLUSTERROLE="dev-full-access"

kubectl apply  -f - <<EOF
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-full-access
  namespace: $NAMESPACE
subjects:
- kind: Group
  name: "k8s-devs"
  apiGroup: rbac.authorization.k8s.io
  namespace: $NAMESPACE
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: $CLUSTERROLE
EOF

# Access to dev namespace
NAMESPACE="dev"
CLUSTERROLE="dev-full-access"

kubectl apply  -f - <<EOF
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: dev-full-access
  namespace: $NAMESPACE
subjects:
- kind: Group
  name: "k8s-devs"
  apiGroup: rbac.authorization.k8s.io
  namespace: $NAMESPACE
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: $CLUSTERROLE
EOF

Step 6: Edit Kubernetes aws-auth ConfigMap

To grant additional AWS users or roles the ability to interact with your cluster, you must edit the aws-auth ConfigMap within Kubernetes.

Check to see if you have already applied the aws-auth ConfigMap.

$ kubectl describe configmap -n kube-system aws-auth

Adding IAM user or role to an Amazon EKS cluster

We’ll be adding our IAM role created in Step 1 to an EKS cluster by editing the aws-auth ConfigMap.

$ kubectl edit -n kube-system configmap/aws-auth

Update the data.mapRoles section to authorize access with IAM role and Kubernetes RBAC group.

apiVersion: v1
data:
  mapRoles: |
    - groups:
      - k8s-devs
      rolearn: arn:aws:iam:::role/k8s-devs-role
      username: developer

Replace:

• with the account ID of your AWS Account.

Here is a screenshot of my configuration:

If you want to give a specific user Master admin access, add like below.

mapUsers: |
    - groups:
      - system:masters
      userarn: arn:aws:iam:::user/
      username: arn:aws:iam:::user/

Or just access to particular namespaces as set in Role bindings earlier:

mapUsers: |
    - groups:
      - k8s-devs
      userarn: arn:aws:iam:::user/
      username: arn:aws:iam:::user/

Step 7: Create a kubeconfig for Amazon EKS (As Developer)

Developers will need to first install AWS CLI and configure their credentials.

$ aws configure

Once credentials are set confirm with the command below:

$ aws sts get-caller-identity

Expected output:

{
    "UserId": "",
    "Account": "",
    "Arn": "arn:aws:iam:::user/"
}

Developers will use AWS CLI update-kubeconfig command to create or update kubeconfig for the cluster:

aws eks --region  update-kubeconfig --name  --role-arn arn:aws:iam:::role/k8s-devs-role

The example below is for cluster called prod-eks-cluster created in eu-west-1 region on account ID 293759796572

$ aws eks --region eu-west-1 update-kubeconfig --name prod-eks-cluster --role-arn arn:aws:iam::293759796572:role/k8s-devs-role

Command output:

Added new context arn:aws:eks:eu-west-1:293759796572:cluster/prod-eks-cluster to /var/root/.kube/config

Try listing Cluster scope resources, you should get Forbidden error message:

$ kubectl get nodes
Error from server (Forbidden): nodes is forbidden: User "developer" cannot list resource "nodes" in API group "" at the cluster scope

But you should be able to list namespace scope resources:

$ kubectl get all -n prod
No resources found in prod namespace.

$ kubectl get all -n uat
No resources found in prod namespace.

$ kubectl get all -n dev
No resources found in prod namespace.

Deploy test application:

$ kubectl run nginx-example --image=nginx --replicas=2  -n dev
deployment.apps/nginx-example created

$ kubectl get pods -n dev
NAME                             READY   STATUS    RESTARTS   AGE
nginx-example-79c476f965-jqm6f   1/1     Running   0          29s
nginx-example-79c476f965-lkzll   1/1     Running   0          29s

$ kubectl delete deploy nginx-example -n dev
deployment.apps "nginx-example" deleted

Enjoy your Development on Kubernetes powered by Amazon EKS. Below are some Video Classes you can try on Kubernetes and Microservices.