[DOCKER] Differences in default behavior of memory and CPU recognition on kubernetes (GKE) for each Java version

Introduction

Even in Java 8 [Oracle Blogs Japanese Summary-[Java] Java SE support for Docker CPU and memory limits](https://orablogs-jp.blogspot.com/2017/05/java-se-support-for-docker-cpu- and.html) From the article, after 8u131, can you see container.resource.limit.cpu and container.resource.limit.memory to some extent and do well? I thought that it was not good behavior, so I investigated the CPU recognition by java8, java10, java11 (ea) when running with kubernetes, and the part to secure the memory size.

In particular, does the behavior of Runtime # availableProcessors, which affects the parallel execution of GC (number of threads) and the number of pools of various parallel threads, change depending on the version? Is the part https://bugs.openjdk.java.net/browse/JDK-8140793 Even if you look at it, as of July 31, 2018, the fixed Version has not been released yet, so I was wondering if any version would work properly.

Conclusion

--When running a Java application with kubernetes, if you want to recognize the CPU and memory correctly, java10 or later will recognize it properly. --Even with java8, memory can be controlled by Xmx, Xms, etc. --In java8, Runtime # availableProcessors cannot be restricted correctly, but GCThread etc. can be controlled.

Prerequisite Java application

Check the metrics information when you run the appropriate Spring Boot application and the information when you run java -XX: + PrintGCDetails -XX: + PrintFlagsFinal -XX: + UnlockExperimentalVMOptions on the container.

Spring Boot application https://github.com/h-r-k-matsumoto/spring-boot-sample Based on, each has been changed as follows. In addition, the resource allocation to the cointainer is as follows.

020_deployments.yml

        resources:
          requests:
            cpu: 150m
            memory: 512Mi
          limits:
            cpu: 900m
            memory: 512Mi

The full text of deployments https://github.com/h-r-k-matsumoto/spring-boot-sample/blob/master/kubernetes/020_deployments.yml is.

java8 (no cgroup option) Change the from image of pom.xml to the following. In particular, Java options do not control anything. Leave the default.

pom.xml

  <image>openjdk:8u171-jre-alpine</image>

java8 Change the from image of pom.xml to the following.

pom.xml

  <image>openjdk:8u171-jre-alpine</image>

In addition, add the following jvmFlag option. The point is ʻUseCGroupMemoryLimitForHeap`. The rest ... is what I always wear when running in a docker environment.

pom.xml

  <jvmFlag>-XX:+UnlockExperimentalVMOptions</jvmFlag>
  <jvmFlag>-XX:+UseCGroupMemoryLimitForHeap</jvmFlag>
  <jvmFlag>-XX:ParallelGCThreads=1</jvmFlag>
  <jvmFlag>-XX:CICompilerCount=2</jvmFlag>
  <jvmFlag>-Djava.util.concurrent.ForkJoinPool.common.parallelism=1</jvmFlag>

java10 Change the from image of pom.xml to the following. No options are specified.

pom.xml

  <image>openjdk:10-jre-slim</image>

java11 (ea) Change the from image of pom.xml to the following. No options are specified.

pom.xml

  <image>openjdk:11-jre-slim</image>

Measurement contents

Spring Boot Actuator metrics

PrintFlagsFinal

 kubectl exec {pod-name} -- java -XX:+PrintGCDetails -XX:+PrintFlagsFinal -XX:+UnlockExperimentalVMOptions

Use the information obtained by executing.

Spring Boot Actuator - metrics

system.cpu.count http://pod-ip:port/actuator/metrics/system.cpu.count Get it with. This is the execution result of Runtime # availableProcessors. The source is [micrometer --ProcessorMetrics.java](https://github.com/micrometer-metrics/micrometer/blob/35890bdc64614c24a8117099b0dcdaa003eab798/micrometer-core/src/main/java/io/micrometer/core/instrument/binder/system/ ProcessorMetrics.java # L83).

jvm.memory.max (heap) http://pod-ip-port/actuator/metrics/jvm.memory.max?tag=area:heap Get it with. The maximum size of the heap area. The source is [micrometer --JvmMemoryMetrics.java](https://github.com/micrometer-metrics/micrometer/blob/cff2e23445812852b5fa97b414958dbe595f0ae2/micrometer-core/src/main/java/io/micrometer/core/instrument/binder/jvm /JvmMemoryMetrics.java#L77-L96). This is the result of MemoryUsage # getMax.

PrintFlagsFinal result

MaxHeapSize The size of the maximum heap management area. It should be the same as [jvm.memory.max](# jvm.memory.max? Tag = area: heap) ...

UseParallelGC Whether to run GC in parallel. Reference: https://docs.oracle.com/javase/jp/8/docs/technotes/guides/vm/gctuning/collectors.html

ParallelGCThreads The number of threads for parallel GC.

Measurement result

The spec of the node to execute is vCPUx2 and memory 7.5GB.

• What is parallel in java 8 can be changed to serial with the option of -XX: + UseSerialGC. Or rather, -XX: ParallelGCThreads = 1 isn't working ...

Unless any option is specified, java11 and java10 are calculated correctly from the memory size 512Mi allocated to the containers, and MaxRAMFraction = 4, so 512/4 = 128. Also, the CPU is set to 900m, but it is judged as one core. java11 and java10 seem to work correctly on the container without specifying Xmx or Xms.

In the case of java8, you can control the memory. However, it seems that the number of CPUs can not be dealt with well ... It seems better to set the part that is calculated from Runtime # availableProcessors and secures resources in the system properties etc. each time.

I don't care if there is a slight error between jvm.memory.max (heap) and MaxHeapSize! Please point out if you say "It is better to specify such an option" or "This is wrong".

reference

--FookJoinPool and pool number specifications: https://docs.oracle.com/javase/jp/8/docs/api/java/util/concurrent/ForkJoinPool.html#ForkJoinPool-- --Java docker environment support story: https://orablogs-jp.blogspot.com/2017/05/java-se-support-for-docker-cpu-and.html