How To Configure AWS S3 CLI for Ceph Object Gateway Storage

The Ceph Object Gateway is an object storage interface built on top of librados to provide applications with a RESTful gateway to Ceph Storage Clusters. The Ceph Object Gateway daemon (radosgw) is an HTTP server for interacting with a Ceph Storage Cluster. It provides interfaces compatible with both OpenStack Swift and Amazon S3 and has embedded user management.

Ceph Object Storage has support for two interfaces.

1. S3-compatible: Provides object storage functionality with an interface that is compatible with a large subset of the Amazon S3 RESTful API.
2. Swift-compatible: Provides object storage functionality with an interface that is compatible with a large subset of the OpenStack Swift API.

In this guide, we’ll focus on configuring Amazon S3 CLI to work with Ceph Object Storage cluster. This will be helpful for automated personal backups and pushing your Server data & configurations to Ceph Object store.

Step 1: Install AWS CLI

We need to install the AWS CLI on the server or machine where access to Ceph Object Gateway will be done.

Follow our guide below to install AWS CLI:

Install and Use AWS CLI on Linux

Verify installation:

$ aws --version
 aws-cli/1.17.10 Python/2.7.5 Linux/3.10.0-1062.el7.x86_64 botocore/1.14.10

Step 2: Create Object Store User for S3 Access

A user should be created on Ceph Object Store backend. This will generate S3 API credentials that we’ll configure AWS S3 CLI to use.

Run the commands in one of your Ceph cluster nodes with access to cluster for administration.

sudo radosgw-admin user create --uid="computingforgeeks" --display-name="Computingforgeeks S3User"

Where:

• computingforgeeks is the name of the user to be created.
• Computingforgeeks S3User is the user display names

This is the sample output from the command.

{
    "user_id": "computingforgeeks",
    "display_name": "Computingforgeeks S3User",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "subusers": [],
    "keys": [
        {
            "user": "computingforgeeks",
            "access_key": "J18YW5BHJIVF69Y57IIT",
            "secret_key": "JthVuEmluDqMuAZyc1oA0abaquc1U0WfemmzL5XS"
        }
    ],
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "default_storage_class": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "check_on_raw": false,
        "max_size": -1,
        "max_size_kb": 0,
        "max_objects": -1
    },
    "temp_url_keys": [],
    "type": "rgw",
    "mfa_ids": []
}

Take note of the access_key and the secret_key.

"user": "computingforgeeks"
"access_key": "J18YW5BHJIVF69Y57IIT"
"secret_key": "JthVuEmluDqMuAZyc1oA0abaquc1U0WfemmzL5XS"

Step 3: Using AWS CLI for Accessing Ceph Object Storage

Before you can start using AWS CLI tool to interact with AWS services, you need to configure it by running the aws configure command.

$ aws configure --profile=ceph
AWS Access Key ID [None]: J18YW5BHJIVF69Y57IIT
AWS Secret Access Key [None]: JthVuEmluDqMuAZyc1oA0abaquc1U0WfemmzL5XS
Default region name [None]:
Default output format [None]: json

Paste the access key and Secret key you copied in previous step. The credentials are written to file ~/.aws/credentials.

$ cat ~/.aws/credentials
[ceph]
aws_access_key_id = J18YW5BHJIVF69Y57IIT
aws_secret_access_key = JthVuEmluDqMuAZyc1oA0abaquc1U0WfemmzL5XS

Configuration file is located in ~/.aws/config.

$ cat ~/.aws/config
[profile ceph]
output = json

You need to have the IP address of one of your Rados Gateway nodes or equivalent DNS name configured. Get a list of rgw servers using the command:

$ sudo ceph -s | grep rgw
    rgw: 3 daemons active (ceph-rgw-01, ceph-rgw-02, ceph-rgw-03)

In my setup, I have three rgw servers. I’ll use one of them – ceph-rgw-01, whose URL is http://172.21.148.53

$ aws --profile=ceph --endpoint=http://172.21.148.53 s3 mb s3://test

The command will create a bucket called test. The creation can be confirmed using the command radosgw-admin.

$ sudo radosgw-admin bucket list
[
    "jkmutai-bucket",
    "test"
]

You can list the buckets created with the command:

$ aws --profile=ceph --endpoint=http://172.21.148.53 s3 ls
2020-02-13 15:17:13 test

Copy a test file to the bucket:

$ aws --profile=ceph --endpoint=http://172.21.148.53  s3 cp release.asc s3://test/
upload: ./release.asc to s3://test/release.asc

Confirm if the file has been uploaded.

$ aws --profile=ceph --endpoint=http://172.21.148.53 s3 ls s3://test/
2020-02-13 16:04:17       1645 release.asc

You can confirm the same from the Ceph Dashboard under Object Gateway > Buckets section.

Reference: Ceph Object Gateway

Related guides:

Ceph vs GlusterFS vs MooseFS vs HDFS vs DRBD

How To Install Ceph Storage Cluster on Ubuntu 18.04 LTS

Monitoring Ceph Cluster with Prometheus and Grafana