Installing Ceph

Here we present one way to install a rudimentary Ceph cluster with Juju backed by an AWS cloud. See Ceph documentation to find out more about Ceph.

Below, we install three Ceph monitors and three OSD systems. Each OSD system has two 32 GiB OSDs and one 8 GiB volume dedicated to the OSD journal. All volumes are EBS-based and magnetic (i.e. not SSD).

juju deploy -n 3 ceph-mon
juju deploy -n 3 ceph-osd --storage osd-devices=ebs,32G,2 --storage osd-journals=ebs,8G,1
juju add-relation ceph-mon ceph-osd

This will take a while. Presented below is the final output to the juju status command for a model called 'ceph':

Model  Controller  Cloud/Region   Version  SLA
ceph   aws         aws/us-east-1  2.3.1    unsupported

App       Version  Status  Scale  Charm     Store       Rev  OS      Notes
ceph-mon  10.2.9   active      3  ceph-mon  jujucharms   18  ubuntu
ceph-osd  10.2.9   active      3  ceph-osd  jujucharms  252  ubuntu

Unit         Workload  Agent  Machine  Public address  Ports  Message
ceph-mon/0   active    idle   0        54.157.253.136         Unit is ready and clustered
ceph-mon/1*  active    idle   1        54.166.249.23          Unit is ready and clustered
ceph-mon/2   active    idle   2        54.221.92.67           Unit is ready and clustered
ceph-osd/0   active    idle   3        54.156.62.76           Unit is ready (2 OSD)
ceph-osd/1   active    idle   4        54.167.125.232         Unit is ready (2 OSD)
ceph-osd/2*  active    idle   5        54.159.206.197         Unit is ready (2 OSD)

Machine  State    DNS             Inst id              Series  AZ          Message
0        started  54.157.253.136  i-02969e42b6654c4cb  xenial  us-east-1c  running
1        started  54.166.249.23   i-09a55570b2c47159f  xenial  us-east-1a  running
2        started  54.221.92.67    i-04467e3dce94a0795  xenial  us-east-1d  running
3        started  54.156.62.76    i-0ee866a3b9a8f2b02  xenial  us-east-1a  running
4        started  54.167.125.232  i-0610ebe802907fc3e  xenial  us-east-1d  running
5        started  54.159.206.197  i-0b9fbb073810df44c  xenial  us-east-1c  running

Relation provider  Requirer      Interface  Type     Message
ceph-mon:mon       ceph-mon:mon  ceph       peer
ceph-mon:osd       ceph-osd:mon  ceph-osd   regular

Now view the status of the cluster from Ceph itself by running the ceph status command on one of the monitors:

juju ssh ceph-mon/0 sudo ceph status

Sample output:

   cluster 316b16da-f02a-11e7-9315-22000b270782
     health HEALTH_OK
     monmap e2: 3 mons at {ip-10-111-155-227=10.111.155.227:6789/0,ip-10-153-224-190=10.153.224.190:6789/0,ip-10-232-134-102=10.232.134.102:6789/0}
            election epoch 12, quorum 0,1,2 ip-10-111-155-227,ip-10-153-224-190,ip-10-232-134-102
     osdmap e27: 6 osds: 6 up, 6 in
            flags sortbitwise,require_jewel_osds
      pgmap v62: 64 pgs, 1 pools, 0 bytes data, 0 objects
            202 MB used, 191 GB / 191 GB avail
                  64 active+clean

This tells us that there are six OSDs, 191 GiB of available space, and all Placement Groups are 'active+clean'.

Now list the storage from Juju:

juju list-storage

Output:

[Storage]
Unit        Id              Type   Pool  Provider id            Size    Status    Message
ceph-osd/0  osd-devices/0   block  ebs   vol-078db09cf280baa61  32GiB   attached
ceph-osd/0  osd-devices/1   block  ebs   vol-04ef7df7ae5ef688b  32GiB   attached
ceph-osd/0  osd-journals/2  block  ebs   vol-056e0f1ae9ef3ee35  8.0GiB  attached
ceph-osd/1  osd-devices/3   block  ebs   vol-06a0a30d8bb069be5  32GiB   attached
ceph-osd/1  osd-devices/4   block  ebs   vol-021706e37f90eb5c1  32GiB   attached
ceph-osd/1  osd-journals/5  block  ebs   vol-02bfd75cae548800c  8.0GiB  attached
ceph-osd/2  osd-devices/6   block  ebs   vol-0321404bf7050c1b3  32GiB   attached
ceph-osd/2  osd-devices/7   block  ebs   vol-0cfc7464af10aebd2  32GiB   attached
ceph-osd/2  osd-journals/8  block  ebs   vol-0236aed9919b0ec8e  8.0GiB  attached

Here we see each Ceph OSD unit has two OSD volumes and one journal volume, for a total of nine volumes.

This Ceph cluster is used as a basis for some examples provided in Using Juju Storage.