I made a new Java deployment tool

Java deployment problem

Suddenly, what are you doing with Java deployment?

Of course, there are many ways to deploy in the end. There is, but in the script language, a light project that can be git cloned with gem, pip, npm, etc. and can solve some dependencies, but when it comes to Java, the fat of the fat jar will surprise you and drop the whole jar file. It will be a little subtle style, such as generating and replacing it manually. (That's not the case! Please let me know if you have a good solution)

In the first place, it seems that you should not write in Java until such a time, but it is said that Java (or JVM language) is lightly required, such as reusing code assets with peripheral tools or wanting to perform parallel computing with a little multithreading. Still exists.

Deployment method in Java

First, let's consider an existing Java deployment method.

Source code transfer

It's a way to do something with the same idea as the scripting language. Copy the source code with git or some tool, resolve the dependency with gradle (hereinafter including maven / sbt / etc.) on the execution server and build locally. In some cases, it will be executed as it is with gradle.

advantage

Disadvantage

It's not a bad method, but it's more like a development server than a deployment destination because it contains the entire set. Also, it's hard to say that it hasn't been modified in the server. I'm worried that if I change and build, and then someone tries to do something and switch to another branch, I don't know.

class file transfer

This is a method to transfer a .class file that is a compilation of .java files to a server and execute it.

advantage

Disadvantage

Is it a method? I used to do it when I first started using Java. It's easy because you just copy the file built by the IDE. Rather than deploying, it's just a way to run it on a server. If it's a scripting language, it's like copying with .zip or .tar. It is the worst method when considering actual operation, such as not having all .classes, of course not knowing the version well, or mixing.

jar file generation

Collect .class files with IDE or jar command, or transfer .jar files generated by gradle to the server. Transfer the .jar files of the dependent libraries as well.

advantage

Disadvantage

It's a legitimate method for its primitive nature. In terms of implementation, it is the same as sending a .class file as a zip file, but it is overwhelmingly easy to handle because various Java tools support it.

You can include source code, memos, documents, anything, so I think it's still useful when you can't manage it. However, it's hard to collect all the .jar files that have dependencies. And different versions of .jar are mixed and it becomes Jar hell instead of DLL hell ...

Put together in fat jar

The royal method. Rebuild all dependent libraries as one executable jar. Now that we define main, we can do the following:

> java -jar xxxx.jar arg1 arg2

Recently (?) Seems to be called uber jar or shadow jar.

advantage

Disadvantage

This is a common technique. The good thing is that you can just transfer a single file. It's all in, so you don't have to worry about it being partially updated and terrible, and you don't have to worry about forgetting the main class.

Personally, it's too fat (fucking big), and when I look only at the production environment, I don't immediately know the name of the library used or the main class I'm running, so I can't say anything unless I unzip it. It doesn't really suit my taste when running on a server. Maybe it's better to operate under proper naming conventions.

It's a bit sad because I sometimes choose fat jar because it's troublesome to collect dependency .jar files such as gradle. It doesn't seem to be that common, but the dependency library is [this way](https://stackoverflow.com/questions/23109276/gradle-task-to-put-jars-from-maven-repository-into-project- It is collected by lib-folder).

Put together in a war file

You can do the same with fat jars with war and ear files, which you rarely see outside the web. war may be more suitable because it allows finer control, such as including the application configuration file but excluding it from the classpath.

advantage

Disadvantage

I think the war file is useful if you have many Java EE environments. If it is a war file, it may be acceptable even if the file size is a little large (prejudice)

Docker image

Create an image by creating an execution environment in the Docker container. How heavy is a Java Docker image? No, Alphine Linux / musc libc / Jigsaw makes it reasonably compact.

Nowadays, you can also create a Docker image with Maven / Gradle at once with Google / Jib.

advantage

Disadvantage

It will be the favorite from now on. Java's support for Docker is also advancing, and it can be said that the easy-to-understand problems have almost disappeared. It also does version control and you don't have to worry about being tied to Java local behavior.

However, on the other hand, it's Docker ... There are quite a few situations that aren't suitable for Docker, and it's a shame to put a JVM that can control memory relatively straightforwardly in a container.

Think about a new deployment method

Looking at various things, we want to solve the following problems for ideal deployment.

What tool meets this requirement? By the way, if you look at other languages, you can usually deploy with gem, pip, npm, etc. The main code can be git cloned or deployed from a private repository. Versions are managed in the repository, and you can see what happened in one shot by looking at the production environment.

Isn't that the case with Java? Given that, at least the repository has a Maven repository. I can't do git clone, which requires compilation, but I feel that support for private repositories is more powerful than other languages. After that, it would be nice if there was a tool. .. ..

That's why I made it.

marun https://github.com/nishemon/marun

A tool to download target artifacts from Maven repositories while resolving dependencies.

> marun install com.example:example:1.0.1

advantage

Disadvantage

Gradle and maven can resolve dependencies, but they are gorgeous because they are just build tools, and they have the ability to resolve dependencies in dependency libraries (artifacts) and take care of them, but the target There is no feature to download only artifacts. There is a tool called Apach Ivy that is specialized only for resolving dependencies in the Maven repository, but this is also for building in combination with Apache Ant. It is mainly considered to be used for Ivy, and Ivy.xml must be written to use it by itself.

So, I used Apache Ivy to create a tool that downloads only the target artifacts and their dependencies.

I need a private repository, but now, for example, maven-publish plugin and [aws-maven plugin](https://qiita.com/ suzutt / items / bf1e8a8e425a9077b96c) makes it easy to build a Maven repository on Amazon S3.

Installation

It's not good to be difficult to use at first, so it is installed with pip. The command line part is Python2 so that it works with system Python as much as possible.

> sudo pip install marun

init Initialize for the time being. Java is required for execution, so please install it first.

> sudo marun init
configuration file is not found!
Your Maven Repository URL []: s3://your_repository
S3 Access Key []:(Enter the access key)
S3 Secret Key []:(Enter the secret key)

Besides s3, you can also use http / https. When you enter it, a configuration file * /etc/marun.conf * will be created.

By the way, Ivy will do his best to download the dependencies of marun itself like this. Due to Amazon S3, there are a lot of jar file dependencies. Dip into * / var / lib / marun *.

Ivy works hard


Download: 130982 / 130982
Download: 1282424 / 1282424
Download: 241622 / 241622
:: loading settings :: url = jar:file:/var/lib/marun/lib/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
:: resolving dependencies :: caller#all-caller;working
	confs: [runtime]
	found jp.cccis.marun#marun;0.1.1 in maven.cccis.jp.s3.amazonaws.com
	found com.amazonaws#aws-java-sdk-s3;1.11.475 in bintray/jcenter
	[1.11.475] com.amazonaws#aws-java-sdk-s3;1.11.+
	found com.amazonaws#aws-java-sdk-kms;1.11.475 in bintray/jcenter
	found com.amazonaws#aws-java-sdk-core;1.11.475 in bintray/jcenter
	found commons-logging#commons-logging;1.1.3 in bintray/jcenter
	found org.apache.httpcomponents#httpclient;4.5.5 in bintray/jcenter
	found org.apache.httpcomponents#httpcore;4.4.9 in bintray/jcenter
	found commons-codec#commons-codec;1.10 in bintray/jcenter
	found software.amazon.ion#ion-java;1.0.2 in bintray/jcenter
	found com.fasterxml.jackson.core#jackson-databind;2.6.7.2 in bintray/jcenter
...
	commons-logging#commons-logging;1.1.3 from bintray/jcenter in [runtime]
	joda-time#joda-time;2.8.1 from bintray/jcenter in [runtime]
	jp.cccis.marun#marun;0.1.1 from maven.cccis.jp.s3.amazonaws.com in [runtime]
	org.apache.httpcomponents#httpclient;4.5.5 from bintray/jcenter in [runtime]
	org.apache.httpcomponents#httpcore;4.4.9 from bintray/jcenter in [runtime]
	software.amazon.ion#ion-java;1.0.2 from bintray/jcenter in [runtime]
	:: evicted modules:
	commons-logging#commons-logging;1.2 by [commons-logging#commons-logging;1.1.3] in [runtime]
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      runtime     |   16  |   2   |   0   |   1   ||   17  |   1   |
	---------------------------------------------------------------------

Try using

It's a good idea to download something from your private repository and run it, but here I'll download Google Closure Compiler.

> sudo marun install com.google.javascript:closure-compiler:+
:: loading settings :: url = jar:file:/var/lib/marun/lib/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml
:: resolving dependencies :: caller#all-caller;working
	confs: [runtime]
	found com.google.javascript#closure-compiler;v20181210 in repo1.maven.org
	[v20181210] com.google.javascript#closure-compiler;+
	found com.google.javascript#closure-compiler-externs;v20181028 in bintray/jcenter
	found args4j#args4j;2.0.26 in bintray/jcenter
	found com.google.errorprone#error_prone_annotations;2.3.1 in bintray/jcenter
	found com.google.guava#guava;25.1-jre in bintray/jcenter
	found org.checkerframework#checker-qual;2.0.0 in bintray/jcenter
	found com.google.j2objc#j2objc-annotations;1.1 in bintray/jcenter
	found org.codehaus.mojo#animal-sniffer-annotations;1.14 in bintray/jcenter
	found com.google.protobuf#protobuf-java;3.0.2 in bintray/jcenter
	found com.google.code.gson#gson;2.7 in bintray/jcenter
	found com.google.code.findbugs#jsr305;3.0.1 in bintray/jcenter
	found com.google.jsinterop#jsinterop-annotations;1.0.0 in bintray/jcenter
	found com.google.auto.value#auto-value;1.4.1 in bintray/jcenter
	found org.apache.ant#ant;1.9.7 in bintray/jcenter
downloading https://repo1.maven.org/maven2/com/google/javascript/closure-compiler/v20181210/closure
...
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      runtime     |   16  |   14  |   14  |   2   ||   16  |   16  |
	---------------------------------------------------------------------

I feel like this.

> ls -l
total 8
drwxr-xr-x 1 root root 512 December 23 21:37 lib
-rw-r--r--1 root root 4438 December 23 21:37 marun.json

> ls -l lib
total 18980
-rw-r--r--2 root root 3482 February 26 2015 animal-sniffer-annotations-1.14.jar
-rw-r--r--2 root root 2036195 April 12 2016 ant-1.9.7.jar
-rw-r--r--2 root root 74703 November 3 2013 args4j-2.0.26.jar
-rw-r--r--2 root root 1504726 April 7 2017 auto-value-1.4.1.jar
-rw-r--r--2 root root 343222 May 6 2016 checker-qual-2.0.0.jar
-rw-r--r--2 root root 189289 October 31 04:55 closure-compiler-externs-v20181028.jar
-rw-r--r--2 root root 10248761 December 13 03:35 closure-compiler-v20181210.jar
-rw-r--r--2 root root 13162 April 21 2018 error_prone_annotations-2.3.1.jar
-rw-r--r--2 root root 231952 June 15 2016 gson-2.7.jar
-rw-r--r--2 root root 2734339 May 24 2018 guava-25.1-jre.jar
-rw-r--r--2 root root 8764 July 27 2016 j2objc-annotations-1.1.jar
-rw-r--r--2 root root 4075 July 29 2016 jsinterop-annotations-1.0.0.jar
-rw-r--r--2 root root 19943 October 9 2015 jsr305-3.0.1.jar
-rw-r--r--2 root root 1304415 September 7 2016 protobuf-java-3.0.2.jar

You can also run it with marun run. The Closure Compiler runs, for example, com.google.javascript.jscomp.CommandLineRunner, but it's a bit clever, so you can do this to find main with the abbreviated name.

> marun run CommandLineRunner
The compiler is waiting for input via stdin.

contents

As you can see in the displayed message, it is almost Apache Ivy. It didn't seem that hard to write on your own if you just wanted to resolve the dependencies, but the deciding factor was that there were pitfalls and that gradle used it in the past. (It seems that they are solving their own dependencies now)

I downloaded the minimum jar file required to start Ivy with Python, and after that I included Ivy with Ivy and tried to resolve the dependency again.

In addition, the contents of marun.json are like this, only a little analysis data is included.

marun.json


{
    "1545568648": {
        "mains": [
            "com.google.javascript.jscomp.CommandLineRunner", 
            "com.google.javascript.jscomp.LinterMain", 
            "org.kohsuke.args4j.Starter", 
            "org.apache.tools.ant.Diagnostics", 
            "org.apache.tools.ant.taskdefs.KeySubst", 
            "org.apache.tools.ant.taskdefs.optional.ejb.IPlanetEjbc", 
            "org.apache.tools.ant.taskdefs.optional.jlink.jlink", 
            "org.apache.tools.ant.util.ProcessUtil"
        ], 
        "dependencies": {
            "com.google.code.gson:gson": {
                "cache": "/var/lib/marun/ivy/com.google.code.gson/gson/jars/gson-2.7.jar", 
                "name": "gson-2.7.jar", 
                "revision": "2.7"
            }, 
...
        },
        "install": [
            "com.google.javascript:closure-compiler:+"
        ]
    },
    "context": [
        1545568648
    ]
}

As you can see in the json file, it also helps save and run the library version, but it essentially just downloads all the jars into lib, so you can run it with java -cp lib / *.

end

So, I was worried about how to deploy Java, so I tried to make a deployment tool that fills the gap. I think there are some addictive situations, so please use it if you like. Don't expect perfection for now. (I want to do my best)

Also, if you don't have to make such a tool, or if you would like to do this, please let us know!

Recommended Posts

I made a new Java deployment tool
I made a Diff tool for Java files
I made a shopify app @java
I made a package.xml generation tool.
I made a primality test program in Java
I made a rock-paper-scissors game in Java (CLI)
I made a simple calculation problem game in Java
I made a check tool for the release module
I made roulette in Java.
I made a chat app.
I made a Wrapper that calls KNP from Java
[Beginner] I made a program to sell cakes in Java
I made a Dockerfile to start Glassfish 5 using Oracle Java
I made a GUI with Swing
I made an annotation in Java.
I made a matching app (Android app)
[Android] I made a pedometer app.
I made a method to ask for Premium Friday (Java 8 version)
I made a tool to output the difference of CSV file
[Ruby] I made a simple Ping client
I tried the new era in Java
I made a risky die with Ruby
I made a plugin for IntelliJ IDEA
I made a rock-paper-scissors app with kotlin
I made a rock-paper-scissors app with android
I made a bulletin board using Docker 1
I made a class that can use JUMAN and KNP from Java
[LINE BOT] I made a ramen BOT with Java (Maven) + Heroku + Spring Boot (1)
New Java engineers can read a refreshing series!
I made StringUtils.isBlank
04. I made a front end with SpringBoot + Thymeleaf
I made a mosaic art with Pokemon images
java I tried to break a simple block
I tried to develop a man-hour management tool
I did Java to make (a == 1 && a == 2 && a == 3) always true
I made a gender selection column with enum
I tried hitting a Java method from ABCL
I wanted to make (a == 1 && a == 2 && a == 3) true in Java
I made a viewer app that displays a PDF
I made a Docker container to run Maven
I made a Ruby extension library in C
[Rails] I made a draft function using enum
java1.8 new features
Learn Java with Progate → I will explain because I made a basic game myself
I touched on the new features of Java 15
I made a LINE bot with Rails + heroku
I wrote a prime factorization program in Java
I tried to break a block with java (1)
I tried running Java on a Mac terminal
I made a portfolio with Ruby On Rails
I made a Docker image of SDAPS for Japanese
I made a method to ask for Premium Friday
[Ruby] I made a crawler with anemone and nokogiri.
I made a drawing chat "8bit paint chat" on WebAssembly
I tried to create a Clova skill in Java
Try Easy Ramdom, a PropertyBase Testing tool for java
I tried using Log4j2 on a Java EE server
I made a Restful server and client in Spring.
I made a library that works like a Safari tab !!
I tried OCR processing a PDF file with Java
I made a library for displaying tutorials on Android.