Waiting for a JMX connection on 18080 Connecting to multiple Java Application servers over a bastion from a local machine at the same time

What are you talking about?

This is a setting example when it is even more troublesome to monitor the JVM by connecting to JMX, which is troublesome if you go over the platform.

Use Nginx's TCP Proxy feature to connect at the same time.

Configuration assumptions

I don't want to request the setting of the port of the stepping stone one by one when the number of AP servers increases.

What you want to solve

To monitor the JVM with JMX, you usually start the server function that listens for JXM connections by giving the following arguments. In addition, there are some arguments that can be used only with Java 7 or later.

This is for the AP server app-srv-01.

-Djava.rmi.server.hostname=`hostname`              # hostname=app-srv-01
-Dcom.sun.management.jmxremote.port=18080
-Dcom.sun.management.jmxremote.rmi.port=18080
-Dcom.sun.management.jmxremote.ssl=false           #Any
-Dcom.sun.management.jmxremote.authenticate=false　#Any

The thing to note here is java.rmi.server.hostname </ code>.

This setting is sent to the connecting JMX client and uses this host name as the JMX connection destination.

In other words, from the JMX client's point of view, if you can't see the JMX connection with app-srv-01: 18080 </ code>, you can't connect.

If you are going through a springboard, you will not be able to connect because you cannot see app-srv-XX.

Easy for one

With one, this issue can be resolved quickly.

All you have to do is set the following port forwarding in ssh's Config and add app-srv-XX to hosts.

~/.ssh/config


Host humidai-srv.net
     user  hoge-user
     IdentityFile    ~/.ssh/id_rsa.hoge
     LocalForward   18080 ap-srv-01:18080

hosts


### app-srv-XX jmx ###
127.0.0.1  app-srv-01

Now you can see ap-srv-ap01: 18080 </ code> as your JMX connection on your local machine.

If app-srv-01 has already been resolved as a different IP due to the environment of the local machine, jmx. </ Strong> is added to the beginning of the host name set in the hosts and jvm arguments. Add a prefix such as code>.

It is troublesome when there are multiple units

With the setting for one unit, port 18080 of localhost will be used for port forwarding of ap-srv-ap01, so port 18080 cannot be used as the JMX connection destination port for the second and subsequent units. I will end up.

I want to manage this issue and monitor multiple remote JVMs.

Solution

This time, we will solve this problem by using Nginx's TCP Proxy function.

Fixed the settings of the first unit & added port forwarding and Hosts settings up to the nth unit

Before setting Nginx, modify the port forwarding as follows so that it does not block port 18080 of the local machine (the fifth and subsequent units are omitted).

~/.ssh/config


Host humidai-srv.net
     user  hoge-user
     IdentityFile    ~/.ssh/id_rsa.hoge
     LocalForward   18081 ap-srv-01:18080
     LocalForward   18082 ap-srv-02:18080
     LocalForward   18083 ap-srv-03:18080
     LocalForward   18084 ap-srv-04:18080
                   :
                   :

Add it to the Hosts file as well. Please note that the IPs that can be drawn are all different. The reason will be described later.

hosts


### app-srv-XX jmx ###
127.0.0.1  app-srv-01
127.0.0.2  app-srv-02
127.0.0.3  app-srv-03
127.0.0.4  app-srv-04
          :
          :

Nginx settings

Version and required Module

There is no problem if you install the latest one normally, but it works with the following.

nginx version: nginx/1.11.9
--with-stream=dynamic #Of course it is OK even if you install it statically

Nginx Stream settings

Add the following to the Nginx settings.

Since the place to write the Stream directive must be at the top level, write it directly in nginx.conf.

nginx.conf


stream {
    upstream up.app-srv-01 { server localhost:18081; }
    upstream up.app-srv-02 { server localhost:18082; }
    upstream up.app-srv-03 { server localhost:18083; }
    upstream up.app-srv-04 { server localhost:18084; }
                :
                :
    
    map $server_addr $upstr{
       127.0.0.1 "up.app-srv-01";
       127.0.0.2 "up.app-srv-02";
       127.0.0.3 "up.app-srv-03";
       127.0.0.4 "up.app-srv-04";
                  :
                  :
       default   "up.app-srv-01";
    }

    server {
       listen 18080;
       proxy_pass  $upstr;
    }
}

Although it is a TCP proxy setting, unlike the http protocol, server_name is not included in the header, so it is not possible to specify the forwarding destination using server_name.

So, by assigning different local machine IPs in Hosts, you can decide the upstream to be assigned.

Connection confirmation

Check the following JMX connections with a tool like java VisualVM.

app-srv-01:18080
app-srv-02:18080
app-srv-03:18080
app-srv-04:18080
   :
   :

So far this time.

Relation

Nginx's Steam module is really handy and can be used in many ways.

Always receive SMTP on localhost: 25-http://qiita.com/aya_eiya/items/dbd2ffff1f07a7bfedd6