Construction of Ceph (Octopus) (common to settings)

__ Introduction __

As the title suggests, this article aims to build __ceph (octopus) __, which is the latest at the moment (2021.01). Also, since it also serves as a memorandum of the procedure you built yourself, there may be mistakes in the details.

Basically, the official DOCUMENTATION covers everything you need, so it's a good idea to read it. Since some knowledge of Linux is required, it is advisable to grasp the basics in advance on other sites and books.

Please refer to the related article for the preparation.

Related article

Building Ceph (Octopus) (Preparation Hardware) Construction of Ceph (Octopus) (Preparation software) Construction of Ceph (Octopus) (common to settings) This article

__ Cluster configuration __

Divide 15 nodes into 3 groups and configure them. L2SW was prepared for each group and aggregated by L3SW. The details are as follows. From now on, we will proceed on the premise of this configuration.

group hostname role
0 ceph-mon01 mon/mgr/osd
0 ceph-mds01 mds/osd
0 ceph-osd01 mds(※)/osd
0 ceph-osd02 osd
0 ceph-osd03 osd
1 ceph-mon11 mon/mgr/osd
1 ceph-mds11 mds/osd
1 ceph-osd11 mds(※)/osd
1 ceph-osd12 osd
1 ceph-osd13 osd
2 ceph-mon21 mon/mgr/osd
2 ceph-mds21 mds/osd
2 ceph-osd21 mds(※)/osd
2 ceph-osd22 osd
2 ceph-osd23 osd

__OS preparation __

Regarding OS installation, I think it is better to refer to other sites than I explain, so I will omit it here. This time I used CentOS7. I used Minimal because I don't need a desktop environment to run ceph.

Select a suitable mirror from the official Download and download it. CentOS Project

__OS standard function settings __

Time synchronization

Use chrony to set the time synchronization. Specify the server that can be referenced. This time, ceph-monXX of each group is synchronized with the existing NTP server, and other nodes are synchronized with the mon node of each group.

vi /etc/chrony.conf

For mon node, add the following

server [NTP server IP] iburst allow [Network segment of other node]

For other than mon node, add the following

server [IP of mon node] iburst

SSH In the case of the setting that uses DNS (default), it may be slow to connect by name, so disable it.

vi /etc/ssh/sshd_config

UseDNS no

FW I will stop FW because it is troublesome to isolate the cause when it does not work well.

systemctl stop firewalld
systemctl disable firewalld

SELinux Disable it for the same reason as FW.

vi /etc/selinux/config

SELINUX=disabled

Edit hosts

Add the host name of the node to hosts.

vi /etc/hosts

192.168.1.101 ceph-mon01 192.168.1.102 ceph-mds01 192.168.1.103 ceph-osd01 ・ ・ ・ 192.168.1.111 ceph-mon11 ・ ・ ・ 192.168.1.121 ceph-mon21 ・ ・ ・ 192.168.1.125 ceph-osd23

Package update

I will update to the latest version for the time being.

yum -y update

__ Setting additional functions __

Add docker

I installed docker because container software is required for octopus. Image pulling is done automatically when you bootstrap and when you add a node to the cluster. Therefore, it is okay to put only the package here.

yum -y install yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum -y install docker-ce docker-ce-cli containerd.io
systemctl start docker
systemctl enable docker
Addition of python3

I feel like it was needed for cephadm. If you didn't put it in, you wouldn't be able to use any important commands. (I'm sorry I forgot.)

yum -y install python36

in conclusion

That's all the work required for all nodes in common. You can do it efficiently by installing only the OS first and working with TeraTerm etc. It is unclear whether it is necessary to set the same hosts (15 units listed) for all nodes. Maybe you don't need it. (I think it's probably needed for the mon node.)

I will post about individual settings of ceph in the next article.

Recommended Posts

Construction of Ceph (Octopus) (common to settings)
Construction of Ceph (Octopus) (Preparation hardware)
Construction of Ceph (Octopus) (preparation software)
Construction of Ceph (Octopus) (setting mon node)
Make common settings with subplot of matplotlib
Change IP settings to ACL of conoha with python
Settings to debug the contents of the library with VS Code