Kubernetes Scheduler Introduction to Homebrew

Introduction

The scheduling framework, which is a framework for creating your own Kubernetes scheduler, has been put in place. However, it is still in the alpha stage, so there is not enough information to actually make and try it, and it is a confusing situation. So, in this article, I will introduce a sample of a simple scheduling framework and how to run it. Furthermore, as an application example, we will explain the Coscheduling plug-in developed by kubernetes scheduler sigs.

What is a scheduler in the first place? What is a scheduling framework? What are the specifications?

I will explain only roughly. For more information, please refer to the good commentary already available.

Scheduler

A k8s management component that controls which node the pod starts on. By default, one default-scheduler is running.

scheduling framework There were some cases where people who were not satisfied with the default-scheduler webhook extension developed a scheduler with full scratch. However, full scratching seems excessive, as some parts behave the same as the default-scheduler. Therefore, a framework was created that allows scheduler development by describing only the behavior of the part that you want to change based on default-scheduler. That is the scheduling framework. (maybe)

scheduling framework specifications

The behavior of default schdeuler is roughly divided into the following phases

Code only the phase part you want to replace for these By compiling in combination with the default scheduler You can create a scheduler that says, "Change only the part you want to change and the others are the same as the default."

Actually, there are more detailed phases. There are small extension points such as PreFilter that run before Filter. See Kubernetes: Understanding kube-scheduler at the source code level and Official Manual for more information.

Actually make and move

The sample source can be found on github. In addition, since I touched Go for the first time this time, please point out any strange points.

Implementation details

Create extension plugin structure

sample-scheduler.go


import(
    "k8s.io/kubernetes/cmd/kube-scheduler/app"
    //(Abbreviation)
    framework "k8s.io/kubernetes/pkg/scheduler/framework/v1alpha1"
)

type SampleScheduler struct {
	framework.PreFilterPlugin
	framework.PostBindPlugin
}

Create extension plugin method

sample-scheduler.go


func (cs *SampleScheduler) PostBind(ctx context.Context, state *framework.CycleState, pod *v1.Pod, nodeName string) {
	klog.Infof("pod %v is binded to %v", pod.Name, nodeName)
}

Create a plugin generation function

sample-scheduler.go


func New(_ runtime.Object, _ framework.FrameworkHandle) (framework.Plugin, error) {
	return &SampleScheduler{}, nil
}

Create an entry point by embedding a plugin in the scheduler

sample-scheduler.go


func main() {
	command := app.NewSchedulerCommand(
		app.WithPlugin(Name, New),
	)

	logs.InitLogs()
	defer logs.FlushLogs()

	if err := command.Execute(); err != nil {
		os.Exit(1)
	}
}

compile

go build sample-scheduler.go

Prepare the scheduler start settings

scheduler-config.yaml


apiVersion: kubescheduler.config.k8s.io/v1beta1
kind: KubeSchedulerConfiguration
leaderElection:
  leaderElect: false
clientConnection:
  kubeconfig: "/etc/kubernetes/admin.conf"
profiles:
- schedulerName: sample-scheduler
  plugins:
    preFilter:
      enabled:
        - name: SampleScheduler
    postBind:
      enabled:
        - name: SampleScheduler

Start the scheduler

sudo ./sample-scheduler --authentication-kubeconfig=/etc/kubernetes/scheduler.conf --authorization-kubeconfig=/etc/kubernetes/scheduler.conf --config=scheduler-config.yaml --secure-port=10260

Pod creation for operation check yaml

apiVersion: v1
kind: Pod
metadata:
  name: sample-scheduler-pod
  labels:
    name: scheduler-example
spec:
  schedulerName: sample-scheduler
  containers:
  - name: container1
    image: k8s.gcr.io/pause:2.0
I1225 07:09:30.512923   10010 sample-scheduler.go:27] pre filter called for pod sample-scheduler-pod
I1225 07:09:30.516019   10010 sample-scheduler.go:34] pod sample-scheduler-pod is binded to node-hoge-hoge

More advanced examples

The above sample has too little processing, and I think there is a big gap with the practical level. So, as a reference, I will briefly introduce how the k8s official extension plugin coscheduler uses the scheduling framework. The repository is kubernetes-sigs/scheduler-plugins

What is coscheduling?

The default scheduler schedules pods one by one. However, if you want to launch multiple linked pods, The following deadlocks can occur:

Mechanism of deadlock

Two groups of pods linked by three pods are in the Pending state

image.png

Two pods are launched from the blue group

image.png

Two pods are launched from the red group

image.png

If one more pod is started in both red and blue, the process can be started, but it cannot be started because the resources are full.

image.png

Avoiding deadlocks with coscheduling

The above deadlock occurs because you start the pods one by one. Therefore, you can avoid it by starting them all at once. This is called coschedule (or gang scheduling).

A rough description of the implementation of the Coscheduling plugin

By using a combination of custom resources and scheduler plug-ins in this way, advanced scheduling is achieved.

Remarks

How to make the created scheduler the default scheduler?

Recommended Posts

Kubernetes Scheduler Introduction to Homebrew
Introduction to MQTT (Introduction)
Introduction to Scrapy (1)
Introduction to Scrapy (3)
Introduction to Supervisor
Introduction to Tkinter 1: Introduction
Introduction to PyQt
Introduction to Scrapy (2)
[Linux] Introduction to Linux
Introduction to Scrapy (4)
Introduction to discord.py (2)
Introduction to discord.py
Introduction to Lightning pytorch
Introduction to Web Scraping
Introduction to Nonparametric Bayes
Introduction to EV3 / MicroPython
Introduction to Python language
Introduction to TensorFlow-Image Recognition
Introduction to OpenCV (python)-(2)
Introduction to PyQt4 Part 1
Introduction to Dependency Injection
Introduction to Private Chainer
Introduction to machine learning
AOJ Introduction to Programming Topic # 1, Topic # 2, Topic # 3, Topic # 4
Introduction to electronic paper modules
A quick introduction to pytest-mock
Introduction to dictionary lookup algorithm
Introduction to Monte Carlo Method
[Learning memorandum] Introduction to vim
Introduction to PyTorch (1) Automatic differentiation
opencv-python Introduction to image processing
Introduction to Cython Writing [Notes]
An introduction to private TensorFlow
An introduction to machine learning
[Introduction to cx_Oracle] Overview of cx_Oracle
A super introduction to Linux
AOJ Introduction to Programming Topic # 7, Topic # 8
[Introduction to pytorch-lightning] First Lit ♬
Introduction to Anomaly Detection 1 Basics
Introduction to RDB with sqlalchemy Ⅰ
[Introduction to Systre] Fibonacci Retracement ♬
Introduction to Nonlinear Optimization (I)
Introduction to serial communication [Python]
AOJ Introduction to Programming Topic # 5, Topic # 6
Introduction to Deep Learning ~ Learning Rules ~
[Introduction to Python] <list> [edit: 2020/02/22]
Introduction to Python (Python version APG4b)
An introduction to Python Programming
[Introduction to cx_Oracle] (8th) cx_Oracle 8.0 release
Introduction to discord.py (3) Using voice
An introduction to Bayesian optimization
Deep Reinforcement Learning 1 Introduction to Reinforcement Learning
Super introduction to machine learning
Introduction to Ansible Part ③'Inventory'
Series: Introduction to cx_Oracle Contents
[Introduction] How to use open3d
Introduction to Python For, While
Introduction to Deep Learning ~ Backpropagation ~
Introduction to Ansible Part ④'Variable'
Introduction to vi command (memorandum)
Introduction to Linux Commands ~ LS-DYNA Edition ~