[JAVA] VisualVM (JMX connection) with Kubernetes

Introduction

This time I'm going to use VisualVM to connect to a Java app running on k8s. I think it will be used when monitoring and analyzing Java applications. Since it is a container, I think that collecting metrics with Prometheus is common, but this time I will use the tool provided by Java because it is a tool familiar to Java developers.

• This time, we did not prepare Java application and implemented it for Jenkins container.
• JMX and Jenkins can also collect metrics with Prometheus.

Environmental information

Kubernetes：1.18 helm：v3.2 ※https://helm.sh/docs/intro/quickstart/ Java：1.8 Jenkins：2.235

Jenkins container preparation

Deploy the Jenkins container with helm.

First, create a parameter file for deployment.

• Use NodePort to connect to the pod.
• The port to connect with jmx is specified by 32000 this time.
• The IP address specified by hostname is the IP of the worker node of k8s. You can check it with the following command.

IP address of worker node

$ kubectl get nodes --namespace jenkins -o jsonpath="{.items[0].status.addresses[0].address}"

values.yaml

master:
  serviceType: NodePort
  javaOpts: >
    -Djava.rmi.server.hostname=0.0.0.0
    -Dcom.sun.management.jmxremote=true
    -Dcom.sun.management.jmxremote.port=32000
    -Dcom.Sun.management.jmxremote.local.only=false
    -Dcom.sun.management.jmxremote.rmi.port=32000
    -Dcom.sun.management.jmxremote.ssl=false
    -Dcom.sun.management.jmxremote.authenticate=false
    -Djava.rmi.server.hostname=192.168.10.51
  jmxPort: 32000

Also, create a yaml file that defines the Service for JMX access.

svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: jenkins-jmx
  namespace: jenkins
spec:
  ports:
  - name: jmx
    nodePort: 32000
    port: 32000
    protocol: TCP
    targetPort: 32000
  selector:
    app.kubernetes.io/component: jenkins-master
    app.kubernetes.io/instance: jenkins
  type: NodePort

Deploy immediately.

• You can download a set of files used by helm at `` `helm pull stable / jenkins```.

deploy_jenkins

#Create jenkins namespace
$ kc create ns jenkins

#k8s connection settings(context)Verification
$ kc config current-context
sandbox

#k8s connection settings(context)Namespace setting
$ kc config set-context sandbox --namespace jenkins

#k8s connection destination namespace confirmation
$ kc config get-contexts
CURRENT   NAME                CLUSTER             AUTHINFO   NAMESPACE
*         sandbox             sandbox             sandbox    jenkins

#Helm repository added / repository information updated
$ helm repo add stable https://kubernetes-charts.storage.googleapis.com
"stable" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "stable" chart repository
Update Complete. ? Happy Helming!?

#Jenkins container deployment
# helm install <Release name> <Chart name> -f <parameter file> --namespace <Deploy destination namespace>
#Release name: Helm's administrative name for the deployment
#Chart name: Template name to use
$ helm install jenkins stable/jenkins -f values.yaml --namespace jenkins
NAME: jenkins
LAST DEPLOYED: Sat Jul 25 hh:mm:ss 2020
NAMESPACE: jenkins
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:
  printf $(kubectl get secret --namespace jenkins jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode);echo
2. Get the Jenkins URL to visit by running these commands in the same shell:
  export NODE_PORT=$(kubectl get --namespace jenkins -o jsonpath="{.spec.ports[0].nodePort}" services jenkins)
  export NODE_IP=$(kubectl get nodes --namespace jenkins -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT/login

3. Login with the password from step 1 and the username: admin
・ ・ ・

#Confirm deployment
$ helm list
NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART         APP VERSION
jenkins jenkins         1               2020-MM-DD hh:mm:ss.950509962 +0900 JST deployed        jenkins-2.4.1 lts

Since the container created by helm does not have a definition of Service to connect to JMX, deploy Service separately.

deploy_service

$ kubectl get svc
NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
jenkins         NodePort    10.43.173.89   <none>        8080:31036/TCP   12m
jenkins-agent   ClusterIP   10.43.130.60   <none>        50000/TCP        12m

$ kubectl apply -f svc.yaml
service/jenkins-jmx created

# kubectl get svc
NAME            TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)           AGE
jenkins         NodePort    10.43.173.89   <none>        8080:31036/TCP    12m
jenkins-agent   ClusterIP   10.43.130.60   <none>        50000/TCP         12m
jenkins-jmx     NodePort    10.43.97.82    <none>        32000:32000/TCP   19s

Connect with Java Visual VM

Now, JMX connect to the started Jenkins container. Let's start Java VisualVM and connect to the Jenkins container. It is also included in the JDK etc., but this time I will connect with the 1.8 Java VisualVM that comes with pleiades (all in one eclipse). https://mergedoc.osdn.jp/

• After decompressing pleiades, you can start it with java \ 8 \ bin \ jvisualvm.exe.

1. Right-click "Remote" on the left menu ⇒ Click "Add Remote Host"
2. Enter the host name (IP address) specified in javaOpts in "Host Name" and click "OK".
3. Confirm that the remote machine is added to the left menu
4. Right-click the registered machine on the left menu ⇒ click "Add JMX Connection"
5. Since the host name is entered in "Connect", enter the port number (32000 in this case) after the: (colon).
6. Check "Do not use SSL connection" and click "OK"
7. Confirm that the connection is added under the registered remote machine in the left menu.
8. Double-click the added connection and confirm that the information is displayed on the main screen

** Screen after connection **

** Monitoring tab **

** Thread tab **

As a bonus, when I try to connect with jconsole, it looks like this.

• Jconsole also has a binary file in the same folder as VisualVM.
• The connection destination is the same as the connection destination set in VisualVM.

Summary

This time I tried to connect to a container running on k8s with VisualVM or jconsole. I couldn't connect by opening a common port, so I added the following and set it.

--Add hostname to java boot options (javaOpts) and set the IP address of the Worker node --Set the port to connect with jmx and the port to open with Nodeport to the same value.

Use this as a reference when connecting with a familiar tool instead of Prometheus.