Create a cluster configuration file to build MirantisKubernetes. In the previous two times, I created a cluster configuration file in advance, but this time I would like to confirm the setting items.
-Mirantis Kubernetes construction (formerly Docker Enterprise) -Istio Ingress on Mirantis Kubernetes
launchpad.You can check the yaml setting items from the following.
> Mirantis Launchpad Configuration File
> https://github.com/Mirantis/launchpad/blob/master/docs/configuration-file.md
I will pick up and explain the parameters that are likely to be used frequently.
* This time, it will be the setting item in launchpad 1.1.1.
** Common **: Common settings
--kind: Set with ``` mke``` if only MKE (k8s), and with ``` mke + msr``` if you also use MSR (Registory).
- metadata
--name: The name of the cluster. For now, it is used internally for logs, client settings, etc.
** hosts **: Settings for each host
--address: IP address when connecting to each node
--privateInterface: NIC can be specified when configuring the network (initial value: `` `eth0```)
--role: You can specify the role in the cluster (`` `manager```, `` `worker```, `` `msr```)
--engineConfig: Mapped as a Docker Engine config file in the lower hierarchy (converted to daemon.json) <br> The above URL example specifies debug mode and log options (file size and rotation).
--ssh: This is the setting when the connection method is SSH.
--user: User name for SSH connection (initial value: ``` root```)
--port: Port number for SSH connection (initial value: 22)
--keypath: Private key path for SSH connection (initial value: `` `~/.ssh/id_rsa```)
** MKE (mke) **: MKE settings
* version: Version of MKE to be installed (initial value: 3.3.3)
* adminUsername: Administrator account name (initial value: admin)
* adminPassword: Administrator account password (default: generate a random character string without specifying)
* installFlags: Explained in InstallFlags
* configFile: Explained in config.toml
* configData: Explained in config.toml
** DockerEngine (engine) **: DockerEngine settings
* version: Docker Engine version
InstallFlags
You can make detailed settings by setting the installFlags item.
You can check the items by running the following on the VM where docker is installed.
$ docker run -t -i --rm docker/ucp:3.3.0 install --help Docker Universal Control Plane Tool
install - Install UCP on this node
USAGE:
docker run --rm -it
-v /var/run/docker.sock:/var/run/docker.sock
docker/ucp
install [command options]
DESCRIPTION:
This command initializes a new swarm, turns this node into a manager, and installs Docker Universal Control Plane (UCP).
When installing UCP you can customize:
The certificates used by the UCP web server. Create a volume named 'ucp-controller-server-certs' and copy the ca.pem, cert.pem, and key.pem files to the root directory. Then run the install command with the '--external-server-cert' flag.
The license used by UCP, by bind-mounting the file at '/config/docker_subscription.lic' in the tool. E.g. -v /path/to/my/config/docker_subscription.lic:/config/docker_subscription.lic or by specifying with '--license "$(cat license.lic)"
If you're joining more nodes to this swarm, open the following ports in your firewall:
If you have SELinux policies enabled for your Docker install, you will need to use 'docker run --rm -it --security-opt label=disable ...' when running this command.
OPTIONS:
--debug, -D Enable debug mode
--jsonlog Produce json formatted output for easier parsing
--interactive, -i Run in interactive mode and prompt for configuration values
--admin-username value The UCP administrator username [
config.toml
You can also pass the MKE installation settings as a configuration file.
Create a sample file with the following command and specify the path in launchpad.yaml.
* You can also pass the value in the configuration file directly to "configData".
* Item explanations are also explained on the following pages.
https://docs.mirantis.com/docker-enterprise/v3.1/dockeree-products/mke/mke-configure/mke-configuration-file.html
#### **`shell-sessionconfig.toml creation`**
```toml creation
$ docker container run --rm mirantis/ucp:3.3.2 example-config
config.toml
# Example UCP configuration file
[auth]
# The name of the auth backend to use, either "managed" or "ldap".
backend = "managed"
# NOTE: LDAP options may only be configured via the UCP UI or API.
# Indicates if SAML is enabled on this UCP.
samlEnabled = false
# The customized SAML login button text
samlLoginText = ""
# Indicates if SCIM is enabled on this UCP.
scimEnabled = false
# Indicates if managed password is disabled.
managedPasswordDisabled = false
# The fallback user when the managed password authentication is disabled.
managedPasswordFallbackUser = ""
# The role that new users will get for their private collections. Can be
# "admin", "viewonly", "scheduler", "restrictedcontrol", or "fullcontrol".
default_new_user_role="restrictedcontrol"
# JSON Web Keys url of external OpenID provider.
jwkUrl="http:///www.example.service/openid-connect/certs"
[auth.sessions]
# Specifies the initial lifetime (in minutes) of a session from the moment
# it is generated.
lifetime_minutes = 60
# Indicates a period of time (in minutes) before the expiration of a session
# where, if used, a session will be extended by the current configured lifetime
# from then, a zero value disables session extension.
renewal_threshold_minutes = 20
# Indicates the maximum number of sessions that any user can have active at any
# given time. If creating a new session would put a user over this limit then
# the least recently used session will be deleted. A value of zero disables
# limiting the number of sessions that users may have.
per_user_limit = 10
# Set token to be stored in sessionStorage instead of localStorage
store_token_per_session = false
# SAML related settings
[auth.saml]
# Service Provider Host
spHost = "52.38.7.31"
# Identity Provider URL
idpMetadataURL = "https://www.testshib.org/metadata/testshib-providers.xml"
# Root Certs for IdP metadata
rootCerts = ""
# TLS Skip verify for IdP Metadata
tlsSkipVerify = false
# SCIM related settings
[auth.scim]
# SCIM API token
token = "4e102f46-6258-4501-9a54-4e5993fabcf5"
# Organization name
orgName = "docker-datacenter"
[[registries]]
# The address used to connect to the DTR instance tied to this UCP cluster
host_address = ""
# The DTR instance's OpenID Connect Client ID, as registered with our auth provider.
service_id = ""
# The root CA bundle for the DTR instance (if using a custom CA).
ca_bundle = ""
# Whether or not this DTR can look up image scanning data in batches
batch_scanning_data_enabled = true
[scheduling_configuration]
# Allow admins to schedule containers on managers
# Set to true to allow admins to schedule on manager
enable_admin_ucp_scheduling = true
# The type of orchestrator to use for new nodes that are joined to the
# cluster. Can be either "swarm" or "kubernetes".
default_node_orchestrator = "swarm"
[tracking_configuration]
# Disable analytics of usage information
# Set to true to disable analytics
disable_usageinfo = false
# Disable analytics of API call information
# Set to true to disable analytics
disable_tracking = false
# Anonymize analytic data
# Set to true to hide your license ID
anonymize_tracking = false
# Set an optional label to be included with analytics that are sent to
# Docker indicating whether this UCP cluster is a production cluster or a
# development cluster. This allows Docker to analyze the data in UCP
# analytics more accurately.
cluster_label = "prod"
[trust_configuration]
# Require images be signed by content trust
require_content_trust = false
# Specify users or teams which must sign images
require_signature_from = ["team1", "team2"]
[log_configuration]
# Set the logging level for UCP components - uses syslog levels
level = "DEBUG"
[audit_log_configuration]
# Specify the audit logging level. Leave empty for disabling audit logs. Other
# legal values are "metadata" and "request". TODO: add link to level descriptions
level = ""
# When set to true, support dumps will include audit logs in the logs
# of the ucp-controller container of each manager node.
support_dump_include_audit_logs = false
[license_configuration]
# Enable attempted automatic license renewal when the license nears expiration
# If disabled, you must manually upload renewed licesnse after expiration.
auto_refresh = true
[[custom_api_server_headers]]
# The name of the HTTP header.
name = "X-Some-Custom-Name"
# The value of the HTTP header.
value = "Some Custom Value"
[pki]
# A custom Root CA bundle to use for client TLS certificate authentication in
# addition to UCP's internal cluster CAs.
client_ca_bundle = ""
[user_workload_defaults]
# A map describing default values to set on Swarm services at creation time
# if those fields are not explicitly set in the service spec.
[user_workload_defaults.swarm_defaults]
[user_workload_defaults.swarm_defaults."tasktemplate.restartpolicy.delay"]
value = "5m"
[user_workload_defaults.swarm_defaults."tasktemplate.restartpolicy.maxattempts"]
value = "3"
[cluster_config]
# Configures the port the ucp-controller listens to
controller_port = 443
# Configures the port the Kubernetes API server listens to
kube_apiserver_port = 6443
# Configures the port the ucp-swarm-manager listens to
swarm_port = 2376
# Configures Swarm scheduler strategy for container scheduling
# This does not affect swarm-mode services
swarm_strategy = "spread"
# Configures DNS settings for the UCP components
dns = []
dns_opt = []
dns_search = []
# Turn on specialized debugging endpoints for profiling UCP performance
profiling_enabled = false
# Turn off Swarm API polling to increase performance of UCP
swarm_polling_disabled = false
# Enable aggregation of DTR image vulnerability scans
image_scan_aggregation_enabled = false
# Tune the KV store timeout and snapshot settings
kv_timeout = 5000
kv_snapshot_count = 20000
# Specify an optional external LB for default links to services with expose ports in the UI
external_service_lb = ""
# Specify the URL of a kubernetes YAML file to be used for installing a CNI
# plugin. Only applies during initial installation. If empty, the default CNI
# plugin will be used
cni_installer_url = ""
# Set to true if ucp should not manage the cni provider lifecycle.
# UCP only manages Calico plugin if the unmanaged-flag is set to false.
unmanaged_cni = false
# Set to true if ucp should deploy secure-overlay which enables IPsec
# Network Encryption in Kubernetes
secure_overlay = false
# Set to true if cluster should use vxlan as dataplane.
calico_vxlan = false
# Set VXLAN VNI id for vxlan data plane.
vxlan_vni = 10000
# Adjust the metrics retention time
metrics_retention_time = ""
# Set the interval for how frequently managers gather metrics from nodes in the cluster
metrics_scrape_interval = "1m"
# Set the size of the cache used by UCP's RethinkDB servers. The installed
# default is 1GB, but leaving this field empty or specifying the special
# value "auto" will instruct RethinkDB to determine a cache size
# automatically (about half of currently available memory) which ensures a
# reliable operation in most scenarios, but the cache size can be tuned
# manually to accommodate increased memory usage by other processes or to
# maximize query performance. A larger cache improves the database’s
# performance, but you must consider other sources of memory consumption to
# avoid out of memory conditions.
# https://www.rethinkdb.com/docs/memory-usage/#cache-size
rethinkdb_cache_size = "16GB"
# If true, disable headers in UCP API responses that indicate which
# UCP controller is returning the response
exclude_server_identity_headers = true
# Set the cloud provider for the kubernetes cluster. Depending on the cloud provider
# different configurations to kubernetes plugin might be need.For eg This could be used
# for cases where we need specific CNI plugin depending on the cloud provider to support
# pod networking
cloud_provider = ""
# Enable the cluster's ISCSI storage backend. Enabling this prepares the
# Kubernetes worker nodes for creation and management of ISCSI based
# persistent volumes.
iscsi_enabled = false
# iscsiadm is the absolute path to the host's iscsiadm binary. This is used when
# iscsi_enabled is set to true. This is necessary for the containerized
# kubelet to act as a iscsi initiator. Defaults to "/usr/sbin/iscsiadm"
# Symlinks not allowed.
iscsiadm_path = ""
# iscsidb is the absolute path to the host's iscsi database. This is used when
# iscsi_enabled is set to true. This is necessary for the containerized
# kubelet to act as a iscsi initiator. Defaults to "/etc/iscsi". Symlinks not allowed.
iscsidb_path = ""
# Set cluster Pod subnet pool. This will be the subnet pool from which the IP for the pod
# would be allocated by the CNI ipam plugin
pod_cidr = "192.168.0.0/16"
# storage_expt_enabled flag is used to enable experimental storage features in the cluster.
storage_expt_enabled = false
# Set the MTU (maximum transmission unit) size for the Calico plugin.
calico_mtu = "1480"
# Set the VXLAN MTU (maximum transmission unit) size for the Calico plugin.
calico_vxlan_mtu = "1450"
# Set the VXLAN Port for the Calico plugin.
calico_vxlan_port = "4789"
# Set the IPIP MTU size for the calico IPIP tunnel interface
ipip_mtu = "1480"
# Set the IP count for azure allocator to allocate IPs per azure VM
azure_ip_count = "64"
# Set the ServiceClusterIPRange used by Kubernates for Services
service_cluster_ip_range = "10.96.0.0/16"
# Set the port range that for Kubernetes services of type NodePort can be
# exposed in.
nodeport_range = "32768-35535"
# Enable the use of a KMS (Key Management Service) plugin. The KMS plugin
# must be deployed on every manager node. UCP will configure the API server
# with the specified endpoint socket.
kms_enabled = false
# Name of the KMS resource
kms_name = ""
# Host path of the KMS endpoint socket
kms_endpoint = ""
# Number of data encryption keys to be cached in the clear by the KMS.
kms_cachesize = 0
# ADVANCED: Set any additional flags on the Kubernetes system components that
# you may require. You must ensure that any custom flags do not conflict with
# configuration options already set by UCP.
custom_kube_api_server_flags = ["--event-ttl=1h0m0s", "--service-node-port-range=30000-32767"]
custom_kube_controller_manager_flags = ["--service-sync-period=5m0s"]
custom_kubelet_flags = ["--http-check-frequency=20s", "--kube-api-qps=200", "--kube-api-burst=200"]
custom_kube_scheduler_flags = ["--algorithm-provider=DefaultProvider", "--kube-api-qps=200", "--kube-api-burst=200"]
custom_kube_proxy_flags = ["--conntrack-min=300000", "--conntrack-max-per-core=200000"]
# Store data about collections for volumes in UCP's local KV store instead of
# on the volume labels. This is used for enforcing access control on volumes.
# Most users will not need to turn this on; this is only necessary if you
# use third-party volume drivers.
local_volume_collection_mapping = false
# Reserve resources for Docker UCP and Kubernetes components which are
# running on manager nodes.
# For more details, see:
# https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#kube-reserved
manager_kube_reserved_resources = "cpu=1,memory=2Gi,ephemeral-storage=4Gi"
# Reserve resources for Docker UCP and Kubernetes components which are
# running on worker nodes.
# For more details, see:
# https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#kube-reserved
worker_kube_reserved_resources = "cpu=100m,memory=300Mi,ephemeral-storage=500Mi"
# Set Number of Pods that can run on this Kubelet.
kubelet_max_pods = 110
# Set pre logon message to alert users before they proceed with login.
pre_logon_message = ""
# Enable Windows GMSA support in Kubernetes. Enabling this deploys the webhook
# admission controller, and creates the Kubernetes resources, required for GMSA
# workloads in Kubernetes.
windows_gmsa = false
[cluster_config.service_mesh]
# Disable Istio Ingress and Service Mesh
enabled = false
# Set the Number of Ingress Replicas
ingress_num_replicas = 2
# Set the List of External IPs for Ingress Gateway Service
ingress_external_ips = []
# Enable external load balancer
ingress_enable_lb = false
# Enable preserving inbound traffic source IP
ingress_preserve_client_ip = false
# Set exposed ports
[[cluster_config.service_mesh.ingress_exposed_ports]]
name = "name"
port = 1
target_port = 2
node_port = 3
# Set node affinity
[cluster_config.service_mesh.ingress_node_affinity]
nodetype = "ingress"
# Set node toleration
[[cluster_config.service_mesh.ingress_node_toleration]]
key = "com.docker.ucp.manager"
value = "dummy"
operator = "Exists"
effect = "NoSchedule"
This time, I checked the setting method and setting values at the time of installation. Since there are core settings from the required items, I think it is better to set from the required place and apply the settings as appropriate.