As the title suggests, this article aims to build __ceph (octopus) __, which is the latest at the moment (2021.01). Also, since it also serves as a memorandum of the procedure you built yourself, there may be mistakes in the details.
Basically, the official DOCUMENTATION covers everything you need, so it's a good idea to read it. Since some knowledge of Linux is required, it is advisable to grasp the basics in advance on other sites and books.
Please refer to the related article for the preparation.
Building Ceph (Octopus) (Preparation Hardware) Construction of Ceph (Octopus) (Preparation software) Construction of Ceph (Octopus) (common to settings) This article
Divide 15 nodes into 3 groups and configure them. L2SW was prepared for each group and aggregated by L3SW. The details are as follows. From now on, we will proceed on the premise of this configuration.
| group | hostname | role |
|---|---|---|
| 0 | ceph-mon01 | mon/mgr/osd |
| 0 | ceph-mds01 | mds/osd |
| 0 | ceph-osd01 | mds(※)/osd |
| 0 | ceph-osd02 | osd |
| 0 | ceph-osd03 | osd |
| 1 | ceph-mon11 | mon/mgr/osd |
| 1 | ceph-mds11 | mds/osd |
| 1 | ceph-osd11 | mds(※)/osd |
| 1 | ceph-osd12 | osd |
| 1 | ceph-osd13 | osd |
| 2 | ceph-mon21 | mon/mgr/osd |
| 2 | ceph-mds21 | mds/osd |
| 2 | ceph-osd21 | mds(※)/osd |
| 2 | ceph-osd22 | osd |
| 2 | ceph-osd23 | osd |
Regarding OS installation, I think it is better to refer to other sites than I explain, so I will omit it here. This time I used CentOS7. I used Minimal because I don't need a desktop environment to run ceph.
Select a suitable mirror from the official Download and download it. CentOS Project
Use chrony to set the time synchronization. Specify the server that can be referenced. This time, ceph-monXX of each group is synchronized with the existing NTP server, and other nodes are synchronized with the mon node of each group.
vi /etc/chrony.conf
For mon node, add the following
server [NTP server IP] iburst allow [Network segment of other node]
For other than mon node, add the following
server [IP of mon node] iburst
SSH In the case of the setting that uses DNS (default), it may be slow to connect by name, so disable it.
vi /etc/ssh/sshd_config
UseDNS no
FW I will stop FW because it is troublesome to isolate the cause when it does not work well.
systemctl stop firewalld
systemctl disable firewalld
SELinux Disable it for the same reason as FW.
vi /etc/selinux/config
SELINUX=disabled
Add the host name of the node to hosts.
vi /etc/hosts
192.168.1.101 ceph-mon01 192.168.1.102 ceph-mds01 192.168.1.103 ceph-osd01 ・ ・ ・ 192.168.1.111 ceph-mon11 ・ ・ ・ 192.168.1.121 ceph-mon21 ・ ・ ・ 192.168.1.125 ceph-osd23
I will update to the latest version for the time being.
yum -y update
I installed docker because container software is required for octopus. Image pulling is done automatically when you bootstrap and when you add a node to the cluster. Therefore, it is okay to put only the package here.
yum -y install yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum -y install docker-ce docker-ce-cli containerd.io
systemctl start docker
systemctl enable docker
I feel like it was needed for cephadm. If you didn't put it in, you wouldn't be able to use any important commands. (I'm sorry I forgot.)
yum -y install python36
That's all the work required for all nodes in common. You can do it efficiently by installing only the OS first and working with TeraTerm etc. It is unclear whether it is necessary to set the same hosts (15 units listed) for all nodes. Maybe you don't need it. (I think it's probably needed for the mon node.)
I will post about individual settings of ceph in the next article.
Recommended Posts