In short (TL; DR;)

--For those who think that Maven / Java builds are slow every time in DevOps pipeline under OpenShift environment. --If you deploy OSS repository manager ** Nexus3 ** on OpenShift, it will become a Maven proxy and cache will work, so the build will be faster!

trouble

Hello! This is Ishida. I'm currently creating a DevOps demo environment for OpenShift, but Java / Maven builds are slow. .. Specifically, when Maven builds a Java application in a container on OpenShift triggered by a source change, ** the container is newly created each time it is built, so there is no Maven cache. As a result, both the first build and the second and subsequent builds will be downloaded from the Maven repository on the Internet, which will take a considerable amount of time. ** **

This is a super simple Servlet build, but it takes about 60 seconds each time

`SimpleServlet.java`


package my;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
 * Servlet implementation class SimpleServlet
 */
@WebServlet("/*")
public class SimpleServlet extends HttpServlet {
    private static final long serialVersionUID = 1L;

    /**
     * @see HttpServlet#doGet(HttpServletRequest request, HttpServletResponse response)
     */
    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
        response.getWriter().print("<h1><font color=green>Simple Servlet ran successfully</font></h1>"
                                   + "Powered by WebSphere Application Server Liberty Profile");
    }

}

Since there is no cache ($ home / .m2 / repository) when the new container (Pod) is started, it is downloaded endlessly from repo.maven.apache.org as follows every time.

`Execution log example`


[INFO] ------------------------< SimpleApp:SimpleApp >-------------------------
[INFO] Building SimpleApp 1.0
[INFO] --------------------------------[ war ]---------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom 
Progress (1): 2.2/8.1 kBProgress (1): 5.0/8.1 kBProgress (1): 7.8/8.1 kBProgress (1): 8.1 kB                        Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom  (8.1 kB at 4.4 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/23/maven-plugins-23.pom 
Progress (1): 2.2/9.2 kBProgress (1): 5.0/9.2 kBProgress (1): 7.8/9.2 kBProgress (1): 9.2 kB                        Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-plugins/23/maven-plugins-23.pom  (9.2 kB at 48 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-parent/22/maven-parent-22.pom 
.....

: warning: Just in case, this article is about ** building Maven over OpenShift ** in the DevOps pipeline. In other words, when s2i is not used = Maven build from source is executed outside OpenShift (= in short, command environment of normal OS etc.) to create a war / ear file, and war / ear is placed (copy) in OpenShift If you just do), the normal Maven cache will work, so there is no problem.

A plan to reduce the execution time of maven build

First of all, generally speaking, in the normal case (= not in the container environment), the content is cached under $ home / .m2 / repository when the first mvn build is executed, and the cache is referenced from the second time onward. Build time is reduced.

1. Store and restore .m2 on your own using s2i's incremental build mechanism

If you replace the same thing with a container environment, first save .m2 in save-artifacts using s2i build --incremental). It is possible to restore it from the second time onward. An example is here. However, this method is not good because you have to implement the s2i script yourself, and if you have multiple builds, you need to implement each individually.

2. Deploy Nexus 3 as Maven Proxy on OpenShift

An easier and more flexible way is to use the repository manager Nexus3 provided by Sonatype on OSS. Nexus3 is a repository manager and can be used for various purposes such as replacing Docker registry as well as Maven, but it has Maven proxy function by default. The container image is published on Dockerhub, so it's very easy to deploy on OpenShift. If you specify Nexus3 as the reference repository when building Java / Maven on OpenShift, the Maven repository on the Internet will be accessed through the Maven proxy of Nexus3 for the first time, but the resources will be cached on the Nexus3 side. Therefore, in the second and subsequent builds, Maven resources will hit the Nexus 3 cache, which will reduce the build time.

In short, when running mvn package in a new container (Pod) in OpenShift

① By default, download occurs every time ② When Nexus3 is operated, it will be downloaded for the first time, but it will be cached. ③ The cache will be hit in the second and subsequent builds when Nexus3 is running (so it will not be downloaded).

I tried it

The procedure for setting up and using Nexus 3 is shown below.

Notice

--If you search the net at the time of writing this article (January 2020), you will find many articles on how to deploy Nexus on OpenShift. I tried several, but it seems that many of the articles are old and do not work as they are. In addition, there are some articles that describe different methods, such as how to specify the Nexus3 repository. This article describes the procedure that the author actually tried and worked at the time of writing, but please be aware that there are other methods as well. --In particular, refer to the official OpenShift 3.11 documentation: newspaper: "Setting Up a Nexus Mirror for Maven" did

environment

--OKD 3.11 (on ubuntu 18.04) running on IBM Cloud (IaaS)-all-in-one configuration started with oc cluster up

Deploying Nexus 3 images to OpenShift

The above image is published on Dockerhub, so you can easily deploy it to OpenShift with the oc command.

`How to deploy nexus3`


oc login -u developer -p dummy
oc new-project nexus3
oc new-app sonatype/nexus3
oc expose svc/nexus3

It is okay if the route (http://nexus3-nexus3.162.133.79.188.nip.io in the above example) is created in oc status.

`oc_status`


handson01@devops01:~$ oc status
In project nexus3 on server https://162.133.79.188:8443

http://nexus3-nexus3.162.133.79.188.nip.io to pod port 8081-tcp (svc/nexus3)
  dc/nexus3 deploys istag/nexus3:latest
    deployment #1 deployed 2 minutes ago - 1 pod

2 infos identified, use 'oc status --suggest' to see details.

Pod is running on the management console, isn't it?

Next3 setup

Obtaining an administrator password

Next, log in to the Nexus 3 console as an administrator (admin) to set up, but before that, you need to obtain the administrator password.

According to the site description of sonatype / nexus3, the password is written in the file ** admin.password **.

Notes: Default user is admin and the uniquely generated password can be found in the admin.password file inside the volume. See Persistent Data for information about the volume.

Use the above procedure to identify the pod name, log in to the pod with oc rsh, and then cat the contents of ** / nexus-data / admin.password ** to get the password. [^ 1]

handson01@devops01:~$ oc get pods
NAME             READY     STATUS    RESTARTS   AGE
nexus3-1-45rpz   1/1       Running   0          2m
handson01@devops01:~$ oc rsh nexus3-1-45rpz
sh-4.4$ find / -name admin.password
find: '/var/cache/ldconfig': Permission denied
find: '/var/lib/private': Permission denied
find: '/var/lib/portables': Permission denied
find: '/lost+found': Permission denied
find: '/proc/tty/driver': Permission denied
/nexus-data/admin.password
sh-4.4$ cat /nexus-data/admin.password
6a37dec6-fc57-41c8-87ac-***********sh-4.4$

[^ 1]: admin.password does not seem to be generated until about 5-10 seconds after starting the pod, so if you can not find it, please wait for a while and try again.

You can do the same with Pods-Terminal from the OpenShift management console.

Login to the Nexus 3 management console

Go to the URL shown in the root (eg http://nexus3-nexus3.162.133.79.188.nip.io), open the Nexus 3 admin console, then click Sign In in the upper right.

Enter admin as the user and the password you obtained earlier, and click "Sign In".

Setup Wizard

The initial setup wizard will start, so click "Next".

First, change the initial password and click "Next"

Specify whether to allow anonymous access. This time the purpose is to cache the public maven2 repository, so it is not confidential, so check "Enable anonymous access" and click "Next"

That's it.

Get Maven Proxy URL

Get the URL of the repository that should be specified when executing the maven package.

It is displayed by "Gear mark (Server Administration and Configulation)"-"Repository"-"Repositories" at the top of the console, and click ** maven-central ** from the list of repositories.

The URL is the URL that Maven should refer to.

specification at the time of maven build

Now that Nexus 3 is ready to use, specify the URL to refer to the Nexus 3 Maven proxy (instead of the default repo.maven.apache.org) as the repository when building with Maven. Specifically, specify the user setting file with -s nexus-settings.xml [^ 2] when building with mvn package as shown below.

[^ 2]: -s, --settings Meaning of Alternate path for the user settings file

`assemble specification example in s2i`


mvn package -s nexus-settings.xml
cp -f target/*.war /config/dropins/

The contents of the user setting file are as follows, and the URL is rewritten in 3 places. [^ 3] For the description, I referred to the article: newspaper: OpenShift Nexus.

[^ 3]: Anonymous access is enabled in the Nexus3 settings wizard, so you don't need to specify the username and password.

`nexus-settings.xml`


<settings>
    <servers>
        <server>
            <id>nexus</id>
            <!-- username>admin</username -->
            <!-- password>password</password -->
        </server>
    </servers>
    <mirrors>
        <mirror>
            <id>nexus</id>
            <mirrorOf>*</mirrorOf>
            <url>http://nexus3-nexus3.162.133.79.188.nip.io/repository/maven-central/</url>
        </mirror>
    </mirrors>
    <profiles>
        <profile>
            <id>nexus</id>
            <!--Enable snapshots for the built in central repo to direct -->
            <!--all requests to nexus via the mirror -->
            <repositories>
                <repository>
                    <id>central</id>
                    <url>http://nexus3-nexus3.162.133.79.188.nip.io/repository/maven-central/</url>
                    <releases><enabled>true</enabled></releases>
                    <snapshots><enabled>true</enabled></snapshots>
                </repository>
            </repositories>
            <pluginRepositories>
                <pluginRepository>
                    <id>central</id>
                    <url>http://nexus3-nexus3.162.133.79.188.nip.io/repository/maven-central/</url>
                    <releases><enabled>true</enabled></releases>
                    <snapshots><enabled>true</enabled></snapshots>
                </pluginRepository>
            </pluginRepositories>
        </profile>
    </profiles>
    <activeProfiles>
        <activeProfile>nexus</activeProfile>
    </activeProfiles>
</settings>

result

If you execute the Maven build again after specifying the above, it will take more than 60 seconds to download the first time, but after the second time, the cache will work and the build will be completed in about 15 seconds!

You can also see that the pod build log also references Nexus 3. (Downloading from nexus: ～)

[INFO]
[INFO] ------------------------< SimpleApp:SimpleApp >-------------------------
[INFO] Building SimpleApp 1.0
[INFO] --------------------------------[ war ]---------------------------------
Downloading from nexus: http://nexus3-nexus3.162.133.79.188.nip.io/repository/maven-central/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom 
Progress (1): 4.1/8.1 kBProgress (1): 7.7/8.1 kBProgress (1): 8.1 kB                        Downloaded from nexus: http://nexus3-nexus3.162.133.79.188.nip.io/repository/maven-central/org/apache/maven/plugins/maven-resources-plugin/2.6/maven-resources-plugin-2.6.pom  (8.1 kB at 43 kB/s)
Downloading from nexus: http://nexus3-nexus3.162.133.79.188.nip.io/repository/maven-central/org/apache/maven/plugins/maven-plugins/23/maven-plugins-23.pom 
Progress (1): 4.1/9.2 kBProgress (1): 7.8/9.2 kBProgress (1): 9.2 kB                        Downloaded from nexus: http://nexus3-nexus3.162.133.79.188.nip.io/repository/maven-central/org/apache/maven/plugins/maven-plugins/23/maven-plugins-23.pom  (9.2 kB at 766 kB/s)
...

in conclusion

Nexus 3 Maven Proxy in OpenShift is very easy to deploy and configure, and can help reduce Maven build time when building DevOps pipelines for Java applications. Please try. You can also use Nexus 3 to build a private Docker image registry. OpenShift comes standard with its own image registry, so it may not be necessary, but it may be a good idea to keep in mind as an alternative. that's all.

Reduce Java / Maven build time with Nexus 3 in OpenShift (okd 3.11) DevOps environment