In this tutorial we will discuss on how to configure EKS Persistent Storage with EFS Amazon service for your Kubernetes cluster to use. The storage backend service we’ll be using is EFS, this will be our default persistent storage for volume claims used by stateful applications. A StorageClass provides a way for administrators to describe the “classes” of storage they offer to allow for dynamic provisioning of persistent volumes.

In Kubernetes, a A PersistentVolume (PV) is a piece of storage in the cluster while PersistentVolumeClaim (PVC) is a request for storage by a user, usually Pod. You need a working EKS cluster before you can use this guide to setup persistent storage for your containerized workloads.

Setup Pre-requisites:

This is the name of the EKS cluster I’ll be working with in this tutorial.

$ eksctl get cluster
NAME			REGION
prod-eks-cluster	eu-west-1

Save the cluster name as a variable that will be used in the remaining steps.

EKS_CLUSTER="prod-eks-cluster"

Using EFS CSI driver to create Persistent Volumes

The Amazon Elastic File System Container Storage Interface (CSI) Driver implements the CSI specification for container orchestrators to manage the lifecycle of Amazon EFS file systems.

Step 1: Create an Amazon EFS File System

The Amazon EFS CSI driver supports Amazon EFS access points, which are application-specific entry points into an Amazon EFS file system that make it easier to share a file system between multiple pods.

You can perform these operations from Amazon console or from the terminal. I’ll be using AWS CLI interface in all my operations.

Locate the VPC ID of your Amazon EKS cluster:

EKS_CLUSTER="prod-eks-cluster"
EKS_VPC_ID=$(aws eks describe-cluster --name $EKS_CLUSTER --query "cluster.resourcesVpcConfig.vpcId" --output text)

Confirm VPC ID if valid.

$ echo $EKS_VPC_ID
vpc-019a6458a973ace2b

Locate the CIDR range for your cluster’s VPC:

EKS_VPC_CIDR=$(aws ec2 describe-vpcs --vpc-ids $EKS_VPC_ID --query "Vpcs[].CidrBlock" --output text)

Confirm VPC CIDR:

$ echo $EKS_VPC_CIDR
192.168.0.0/16

Create a security group that allows inbound NFS traffic for your Amazon EFS mount points:

aws ec2 create-security-group --group-name efs-nfs-sg --description "Allow NFS traffic for EFS" --vpc-id $EKS_VPC_ID

Take note of Security group ID. Mine is:

{
    "GroupId": "sg-0fac73a0d7d943862"
}
# You can check with
$ aws ec2 describe-security-groups --query "SecurityGroups[*].{Name:GroupName,ID:GroupId}"

Add rules to your security group:

SG_ID="sg-0fac73a0d7d943862"
aws ec2 authorize-security-group-ingress --group-id $SG_ID --protocol tcp --port 2049 --cidr $EKS_VPC_CIDR

To view the changes to the security group, run the describe-security-groups command:

$ aws ec2 describe-security-groups --group-ids $SG_ID
{
    "SecurityGroups": [
        {
            "Description": "Allow NFS traffic for EFS",
            "GroupName": "efs-nfs-sg",
            "IpPermissions": [
                {
                    "FromPort": 2049,
                    "IpProtocol": "tcp",
                    "IpRanges": [
                        {
                            "CidrIp": "192.168.0.0/16"
                        }
                    ],
                    "Ipv6Ranges": [],
                    "PrefixListIds": [],
                    "ToPort": 2049,
                    "UserIdGroupPairs": []
                }
            ],
            "OwnerId": "253859766502",
            "GroupId": "sg-0fac73a0d7d943862",
            "IpPermissionsEgress": [
                {
                    "IpProtocol": "-1",
                    "IpRanges": [
                        {
                            "CidrIp": "0.0.0.0/0"
                        }
                    ],
                    "Ipv6Ranges": [],
                    "PrefixListIds": [],
                    "UserIdGroupPairs": []
                }
            ],
            "VpcId": "vpc-019a6458a973ace2b"
        }
    ]
}

Create the Amazon EFS file system for your Amazon EKS cluster:

# Not encrypted
$ aws efs create-file-system --region eu-west-1

# Encrypted EFS file system
$ aws efs create-file-system --encrypted --region eu-west-1

Note the file system ID:

{
    "OwnerId": "253759766542",
    "CreationToken": "c16c4603-c7ac-408f-ac4a-75a683ed2a29",
    "FileSystemId": "fs-22ac06e8",
    "FileSystemArn": "arn:aws:elasticfilesystem:eu-west-1:253759766542:file-system/fs-22ac06e8",
    "CreationTime": "2020-08-16T15:17:18 03:00",
    "LifeCycleState": "creating",
    "NumberOfMountTargets": 0,
    "SizeInBytes": {
        "Value": 0,
        "ValueInIA": 0,
        "ValueInStandard": 0
    },
    "PerformanceMode": "generalPurpose",
    "Encrypted": true,
    "KmsKeyId": "arn:aws:kms:eu-west-1:253759766542:key/6c9b725f-b86d-41c2-b804-1685ef43f620",
    "ThroughputMode": "bursting",
    "Tags": []
}

UI view:

<img alt="" data-ezsrc="https://kirelos.com/wp-content/uploads/2020/08/echo/efs-kubernetes-02-1024×559.png" data-ez ezimgfmt="rs rscb8 src ng ngcb8 srcset" height="559" loading="lazy" src="data:image/svg xml,” width=”1024″>

Step 2: Create EFS Mount Target

Get Subnets in your VPC where EC2 instances run. In my case all EKS instances run in private subnets.

EKS_VPC_ID=$(aws eks describe-cluster --name $EKS_CLUSTER --query "cluster.resourcesVpcConfig.vpcId" --output text)
aws ec2 describe-subnets --filter Name=vpc-id,Values=$EKS_VPC_ID --query 'Subnets[?MapPublicIpOnLaunch==`false`].SubnetId'

My output:

[
    "subnet-0977bbaf236bd952f",
    "subnet-0df8523ca39f63938",
    "subnet-0a4a22d25f36c4124"
]

Create Mount Targets.

# File system ID
EFS_ID="fs-22ac06e8"

# Create mount targets for the subnets - Three subnets in my case
for subnet in subnet-0977bbaf236bd952f subnet-0df8523ca39f63938 subnet-0a4a22d25f36c4124; do
  aws efs create-mount-target 
    --file-system-id $EFS_ID 
    --security-group  $SG_ID 
    --subnet-id $subnet 
    --region eu-west-1
done

Step 2: Using EFS CSI driver

After creation of an EFS file system and and Mount Target we can test EFS CSI driver by creating static persistent volume and claiming it through a test container.

Deploy EFS CSI provisioner.

$ kubectl apply -k "github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/dev/?ref=master"

daemonset.apps/efs-csi-node created
csidriver.storage.k8s.io/efs.csi.aws.com created

List available CSI drivers:

$ kubectl get csidrivers.storage.k8s.io
NAME              CREATED AT
efs.csi.aws.com   2020-08-16T19:10:35Z

First get the EFS filesystem ID:

$ aws efs describe-file-systems --query "FileSystems[*].FileSystemId"
[
    "fs-22ac06e8"
]

Create storage class:

kubectl apply -f - <<EOF
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
EOF

List Storage Classes available:

$ kubectl get sc
NAME            PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
efs-sc          efs.csi.aws.com         Delete          Immediate              false                  28s
gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  4d21h

Create and modify following manifest file to set correct File system ID:

$ vim efs-pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: efs-pv
spec:
  capacity:
    storage: 1Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs-sc
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-22ac06e8 # Your EFS file system ID

Apply the file to create resources:

$ kubectl apply -f efs-pv.yml
persistentvolume/efs-pv created

$ kubectl get pv
NAME     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
efs-pv   1Gi        RWO            Retain           Available           efs-sc                  19s

Create claim resource:

kubectl apply -f - <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: efs-sc
  resources:
    requests:
      storage: 1Gi
EOF

List claims to confirm it was created and status is Bound:

$ kubectl get pvc
NAME        STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
efs-claim   Bound    efs-pv   1Gi        RWO            efs-sc         7s

Create test pod which uses the volume claim.

kubectl apply -f - <> /data/out.txt; sleep 5; done"]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim
EOF

Verify if the Pod is running:

$ kubectl get pod
NAME      READY   STATUS    RESTARTS   AGE
efs-app   1/1     Running   0          34s

Inside container check mount points.

$ kubectl exec -ti efs-app -- bash
[[email protected] /]# df -hT
Filesystem     Type     Size  Used Avail Use% Mounted on
overlay        overlay   80G  3.4G   77G   5% /
tmpfs          tmpfs     64M     0   64M   0% /dev
tmpfs          tmpfs    1.9G     0  1.9G   0% /sys/fs/cgroup
127.0.0.1:/    nfs4     8.0E     0  8.0E   0% /data
/dev/nvme0n1p1 xfs       80G  3.4G   77G   5% /etc/hosts
shm            tmpfs     64M     0   64M   0% /dev/shm
tmpfs          tmpfs    1.9G   12K  1.9G   1% /run/secrets/kubernetes.io/serviceaccount
tmpfs          tmpfs    1.9G     0  1.9G   0% /proc/acpi
tmpfs          tmpfs    1.9G     0  1.9G   0% /sys/firmware

Write some test files to /data where EFS file system is mounted.

[[email protected] /]# touch /data/testfile1
[[email protected] /]# touch /data/testfile2
[[email protected] /]#
[[email protected] /]# ls /data/
out.txt  testfile1  testfile2
[[email protected] /]# exit
exit

Clean your test data.

kubectl delete pod efs-app
kubectl delete pvc efs-claim
kubectl delete pv efs-pv

Related articles:

Ceph Persistent Storage for Kubernetes with Cephfs

Persistent Storage for Kubernetes with Ceph RBD

Configure Kubernetes Dynamic Volume Provisioning With Heketi & GlusterFS

, , , ,
Show Comments