[2018 definitive edition] [Java 11 containerization] After downsizing the Spring Boot application to the limit, build it with Jib and push it to ECR

The world is an unprecedented container boom, and when I went to the conference this year, there was no day when I didn't hear the k8s story.

Under such circumstances, Java has a large footprint and is incompatible with Cloud Functions such as Lambda, so it tends to be shunned in various ways. Even though we use Java steadily, in our Future Advent Calendar 2018, everyone writes Java material. I don't. .. ..

I love Java, Python, Ruby, Go, and ES6, and I hope I can use them in the right place, but this time I'll write an article that pushes Java back on Christmas Eve.

JDK Warring States Period-Which JDK should we use?

Java, which has been used free of charge by Oracle from Sun Microsystems, has finally been paid for from Java 11 in September 2018, and the world has been rushed.

However, if you organize the information properly, in addition to the Oracle JDK that Oracle supports for a fee, the community and vendors will release the LTS version of the JDK based on the OpenJDK that Oracle has led, so it is not a fuss. I think.

But in the enterprise world, who is responsible for timely patching when vulnerabilities are discovered? However, the point is that security is defense in depth in principle, and vulnerabilities such as library frameworks such as Struts have a greater impact than Java itself, so a purely Japanese mind is a normal judgment. I think it's hindering, but I think that Oracle paid support is also useful for such companies.

In addition, it is said that Amazon / Azure also provides OpenJDK and supports LTS, which is exactly the Warring States period.

It's ironic because this is unlikely to happen with other language runtimes, but you can see how many Java systems are used ^^;

Furthermore, the existence of GraalVM cannot be overlooked. There are various restrictions such as not being able to support Dynamic Loading in native image conversion yet, so it can not be used with Spring Boot etc., but it seems to be a matter of time, so I have to look forward to it.

reference

Comparison of vendors providing long-term commercial support (LTS) for JDK (also mentioning free use) --Qiita https://qiita.com/u-tanick/items/bb166929a58a4c20bb88

I tried GraalVM-Qiita https://qiita.com/sonodar/items/dcfafdfba8af2db53b16

Try to make Spring Boot Application native-image with Micronaut for Spring --Qiita https://qiita.com/h-r-k-matsumoto/items/5b82177294cd71df5024

Java11 SpringBoot containerization-Downsizing is not limited to hosts-

From here, let's think about running on Java 11 while using a Web application by SpringBoot as a subject.

If you don't want to make it a container, you can use a single startup Jar (fat-jar) and it will be very easy because you can just deploy it to the environment where the JDK is installed. I think that there are many cases where the size of the jar is only a few tens of MB.

When it comes to containerization, it is OK to copy the above jar to the container image containing the JDK and start it, but the problem here is the image size. The container is small but justice, including PUSH / PULL to the registry and the startup of the container itself.

OpenJDK (Official)

Since the dokcer image (1GB) of OpenJDK11 is large, create a small image (85MB) with alpine linux + jlink --Qiita https://qiita.com/h-r-k-matsumoto/items/294eeb838cfd062d75b6

As introduced here, the official image of OpenJDK is 1GB. Besides, there is no LTS ... so it is out of the options.  Distroless by Google

Create a runtime Docker image using a distroless image --Qiita https://qiita.com/some-nyan/items/90b624b0f148231748f0)

I investigated that Distoless, which is released as an image for runtime by Google, seems to be good, but at the moment it only supports Java 8 and it seems that I have to wait for Java 11 support.

AdoptOpenJDK

Looking at adoptopenjdk / openjdk11: alpine published by AdoptOpenJDK, it is about 200MB, which is quite slim, so use this It seems that going is a better option as of 2018.

However, if you put an application on it, it will be heavier, so as introduced in the article above, Java11 also has the benefit of the module system, so we will create a custom JRE using jlink and further downsize it.

Therefore, the flow is to first create a JDK that is slimmed down to the minimum necessary to operate Spring Boot with the Alpine image of AdoptOpenJDK, and then copy it to the original image of Alpine.

The caveat here is that just copying a custom JRE into plain Alpine won't work. Therefore, add glibc as shown below.

Dockerfile

###########################################################
# spring-boot-jre-min-11
# Custom JRE from AdobtOpenJDK11 for spring-boot
# 
###########################################################
FROM adoptopenjdk/openjdk11:alpine AS builder

# create custom jre
RUN jlink \
    --module-path="${JAVA_HOME}/jmods" \
    --compress=2 \
    --add-modules=java.base,java.logging,java.xml,jdk.unsupported,java.sql,java.naming,java.desktop,java.management,java.security.jgss,java.instrument,jdk.charsets \
    --no-header-files \
    --no-man-pages \
    --verbose \
    --output=/opt/jre-min

# pull plane alpine
FROM alpine:3.8

ENV JAVA_HOME="/opt/jre-min"
ENV PATH="$PATH:/opt/jre-min/bin"
ENV JAVA_VERSION="jdk-11.0.1+13"
ENV JAVA_TOOL_OPTIONS="-XX:+UseContainerSupport"

# add glibc-compat
RUN apk --update add --no-cache ca-certificates curl openssl binutils xz \
    && GLIBC_VER="2.28-r0" \
    && ALPINE_GLIBC_REPO="https://github.com/sgerrand/alpine-pkg-glibc/releases/download" \
    && GCC_LIBS_URL="https://archive.archlinux.org/packages/g/gcc-libs/gcc-libs-8.2.1%2B20180831-1-x86_64.pkg.tar.xz" \
    && GCC_LIBS_SHA256=e4b39fb1f5957c5aab5c2ce0c46e03d30426f3b94b9992b009d417ff2d56af4d \
    && ZLIB_URL="https://archive.archlinux.org/packages/z/zlib/zlib-1%3A1.2.9-1-x86_64.pkg.tar.xz" \
    && ZLIB_SHA256=bb0959c08c1735de27abf01440a6f8a17c5c51e61c3b4c707e988c906d3b7f67 \
    && curl -Ls https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -Ls ${ALPINE_GLIBC_REPO}/${GLIBC_VER}/glibc-${GLIBC_VER}.apk > /tmp/${GLIBC_VER}.apk \
    && apk add /tmp/${GLIBC_VER}.apk \
    && curl -Ls ${GCC_LIBS_URL} -o /tmp/gcc-libs.tar.xz \
    && echo "${GCC_LIBS_SHA256}  /tmp/gcc-libs.tar.xz" | sha256sum -c - \
    && mkdir /tmp/gcc \
    && tar -xf /tmp/gcc-libs.tar.xz -C /tmp/gcc \
    && mv /tmp/gcc/usr/lib/libgcc* /tmp/gcc/usr/lib/libstdc++* /usr/glibc-compat/lib \
    && strip /usr/glibc-compat/lib/libgcc_s.so.* /usr/glibc-compat/lib/libstdc++.so* \
    && curl -Ls ${ZLIB_URL} -o /tmp/libz.tar.xz \
    && echo "${ZLIB_SHA256}  /tmp/libz.tar.xz" | sha256sum -c - \
    && mkdir /tmp/libz \
    && tar -xf /tmp/libz.tar.xz -C /tmp/libz \
    && mv /tmp/libz/usr/lib/libz.so* /usr/glibc-compat/lib \
    && apk del binutils \
    && rm -rf /tmp/${GLIBC_VER}.apk /tmp/gcc /tmp/gcc-libs.tar.xz /tmp/libz /tmp/libz.tar.xz /var/cache/apk/*

COPY --from=builder /opt/jre-min /opt/jre-min

Here's what I pushed to Docker Hub.

shoutstar/spring-boot-jre-min-11 - Docker Hub https://hub.docker.com/r/shoutstar/spring-boot-jre-min-11

I managed to downsize to 48MB: smile:

Containerization at build ~ Do you know Jib? ~

GoogleContainerTools/jib: Build container images for your Java applications. https://github.com/GoogleContainerTools/jib

Jib is a tool that makes the Java application released by Google OSS in July this year into a container image when building Maven / Gradle and even registers it in the registry. The following is detailed.

"Jib" that automatically builds Java applications into Docker images, released as open source by Google-Publickey https://www.publickey1.jp/blog/18/javadockerjibgoogle.html

Jib provides a mechanism to automatically push not only DockerHub but also GCP GCR and AWS ECR. It is extremely hot that it can be automated so far without writing a Dockerfile and without typing a docker command: fire :.

Well, like I wrote the Dockerfile earlier ... Actually, there is a reason for this. In Jib, the from image is specified in the settings, but it is not possible to customize this from image by executing the above jlink. So, if you want jlink to run using a custom JRE, you need to create a base image in advance and publish it to the registry.

https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#i-need-to-run-commands-like-apt-get

Of course, the resources in the classpath are automatically copied, but it is possible to copy any file that is not in the classpath by setting.

https://github.com/GoogleContainerTools/jib/blob/master/docs/faq.md#i-need-to-run-commands-like-apt-get

In addition to PUSH to the registry, Jib supports registering with the local Docker daemon, creating tars, and more.

#PUSH to the registry
$ mvn compile jib:build

#Registration to local Docker daemon (Docker start required)
$ mvn compile jib:dockerBuild

#Create tar
$ mvn compile jib:buildTar

Now, let's set up to push to ECR with Jib's Maven plugin.

pom.xml

<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>jib-maven-plugin</artifactId>
    <version>0.10.0</version>
    <configuration>
        <!--In some cases it is convenient to put it in when using a proxy-->
        <allowInsecureRegistries>true</allowInsecureRegistries>
        <from>
            <!--PULL a lightweight image with only a custom JRE from Docker Hub-->
            <image>shoutstar/spring-boot-jre-min-11</image>
        </from>
        <to>
            <!--At build time-Djib.to.Replace with image-->
            <image>future/sprinb-boot-app</image>
            <!--At build time-Djib.to.Replace with credHelpr
            <credHelper>ecr-login</credHelper>
            -->
        </to>
        <container>
            <!--I will pass the profile of spring here-->
            <jvmFlags>
                <jvmFlag>-Dadd-opens=java.base/java.lang=ALL-UNNAMED</jvmFlag>
                <jvmFlag>-Dadd-opens=java.base/java.lang.invoke=ALL-UNNAMED</jvmFlag>
                <jvmFlag>-Dspring.profiles.active=${spring.profiles.active}</jvmFlag>
            </jvmFlags>
            <!--Set image creation time at build time-->
            <useCurrentTimestamp>true</useCurrentTimestamp>
        </container>
    </configuration>
</plugin>

supplement to pom.xml

jib.from.image

The custom JRE mentioned above is specified. It is shoutstar / spring-boot-jre-min-11 of Docker Hub.

jib.to.image, jib.to.credHelpr

Since there are cases where you want to place it in local Docker during development, basically, I think it is better to replace it with -Djib.to.image and -Djib.to.credHelper at build time.

PUSH to ECR

Originally, the docker login command is issued with the $ (aws ecr get-login --no-include-email --region ap-northeast-1) command to log in, and then PUSH is performed, but in jib, the following You can push it in cooperation with the command line tool.

awslabs/amazon-ecr-credential-helper: Automatically gets credentials for Amazon ECR on docker push/docker pull https://github.com/awslabs/amazon-ecr-credential-helper

Follow the README above to install. Installation of golang 1.6 or above is required.

$ go get -u github.com/awslabs/amazon-ecr-credential-helper/ecr-login/cli/docker-credential-ecr-login

Put a symbolic link in $ GOPATH / bin / docker-credential-ecr-login and put it in your PATH.

Then create the following config.json to make all ECR registry logins OK.

json:~/.docker/config.json

{
    "credsStore": "ecr-login"
}

If the build environment is EC2, if you assign IAM Role, no other settings are required. If that is not possible, set the access key secret in ~ / .aws / credentials, or specify ʻAWS_ACCESS_KEY_ID and ʻAWS_SECRET_ACCESS_KEY in the environment variables, respectively.

When pushing to ECR, the command is as follows.

$ mvn clean compile jib:build \
-Djib.to.image=xxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/future/spring-boot-app \
-Djib.to.credHelper=ecr-login

...Omission

[INFO] --- jib-maven-plugin:0.10.0:build (default-cli) @ spring-boot-app ---
[WARNING] Setting image creation time to current time; your image may not be reproducible.
[INFO] 
[INFO] Containerizing application to xxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/future/spring-boot-app ...
[WARNING] Base image 'shoutstar/spring-boot-jre-min-11' does not use a specific image digest - build may not be reproducible
[INFO] Retrieving registry credentials for xxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com...
[INFO] Getting base image shoutstar/spring-boot-jre-min-11...
[INFO] Building snapshot dependencies layer...
[INFO] Building resources layer...
[INFO] Building dependencies layer...
[INFO] Building classes layer...
[INFO] The base image requires auth. Trying again for shoutstar/spring-boot-jre-min-11...
[INFO] Retrieving registry credentials for registry.hub.docker.com...
[INFO] 
[INFO] Container entrypoint set to [java, -Dadd-opens=java.base/java.lang=ALL-UNNAMED, -Dadd-opens=java.base/java.lang.invoke=ALL-UNNAMED, -Dspring.profiles.active=container, -cp, /app/resources:/app/classes:/app/libs/*, jp.co.future.sample.SpringBootApplication]
[INFO] Finalizing...
[INFO] 
[INFO] Built and pushed image as xxxxxx.dkr.ecr.ap-northeast-1.amazonaws.com/future/spring-boot-app
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

I was able to push to ECR safely.

By the way, it is an application like spring-petclinic clone that I created for the purpose of checking the operation of the in-house framework, and the image size is about 95MB. At the end of 2018, I wonder if this is the limit: sweat_smile:

At the end

Jib has made containers familiar and easy for Java engineers, so if you haven't touched them yet, please give it a try. After all, I've been hitting docker commands through trial and error, so I'm not saying that even Java users who don't know Docker can use it. .. ..

In the future, if the startup speed becomes faster with GraalVM, it seems that it can be used with Cloud Function, and Micronaut, which is inspired by Spring Boot but has a high affinity with GraalVM, is also noteworthy.

Just a few days ago, Amazon EKS came to the Tokyo region, so I'd like to write about deploying the image pushed to ECR to EKS.

Amazon EKS now supports the Tokyo region. | Amazon Web Services Blog https://aws.amazon.com/jp/blogs/news/amazon-eks-tokyo-region/