[GO] How to build a NEM (current version) node (NIS1: NEM Infrastructure Server)

The NEM blockchain is currently undergoing various preparations on the testnet for the launch of the next version of Symbol. In the midst of such timing, I feel that there is a great deal of documentation and reference information on how to build a node for the test net of the next version of Symbol. However, surprisingly, I had a hard time finding documentation and reference information on how to build the nodes of the current version of the NEM blockchain. (There may also be my own googleability issues ...) For my own reminder, I'll summarize in this article how to build a node for the current version of NEM (called NIS1 ... NEM Infrastructure Server) on Google Cloud Ploatform. If you have any mistakes or better ways, please let me know. Thank you.

Overview

  1. Create a firewall rule
  2. Create a VM on GCE and log in with SSH
  3. Java installation
  4. Download and unzip the main body of NIS1
  5. Modify the node startup script
  6. Start the node
  7. Confirmation after startup

procedure

1. Create a firewall rule

NIS1 seems to use port 7890 for http and port 7778 for WebSocket. Also, it seems customary to use port 7891 for https and port 7779 for WebSocket (SSL). Create a firewall rule as shown in the example below. The name and target tag are free, but the target tag will be used later in the VM settings.

  1. Click the navigation menu (three-line icon) on the upper left
  2. Click "VPC Network" (scroll down to see it)
  3. Click "Firewall"
  4. Click "Create Firewall Rule" to "Create" the following two firewall rules

1st firewall rule: for http

--Name: nem-http-allow --Target tag: nem-http-allow --Source IP range: 0.0.0.0/0 --Protocols and ports: Check the radio buttons for the specified protocol and port, check the checkbox to the left of "tcp", and enter 7890, 7778 to the right of it. --Click "Create"

Second firewall rule: for https

--Name: nem-https-allow --Target tag: nem-https-allow --Source IP range: 0.0.0.0/0 --Protocols and ports: Check the radio buttons for the specified protocol and port, check the checkbox to the left of "tcp", and enter 7891, 7779 to the right of it. --Click "Create"

2. Create a VM on GCE

As a result of investigating in advance and listening to rumors, it seemed that heavy processing would run as it was, so at first I tried to set it to a higher spec. When the synchronization at the time of initial setting is completed and the state is stable, I think it is better to consider narrowing down the specifications considering the cost. The VM setting example is as follows.

VM setting example

--Name: nem-mainnet-1 --Region: asia-northeast1 (Tokyo) --Zone: asia-northeast1-b --Machine configuration --Series: E2 --Machine type: Custom --Number of cores: 2 vCPU --Memory: 16 GB (I feel like I've secured too much, but ...) --Boot disk --Click "Change" --Operating system: Ubuntu --Version: Ubuntu 20.04 LTS --Boot disk type: Standard persistent disk --Size: 400 GB (feeling too much secured ... omitted below) --Click "Select" to save the boot disk settings --Firewall --"Allow HTTP traffic" check ON --"Allow HTTPS traffic" check ON --Click on Administration, Security, Disk, Networking, Single Hand Nancy and select the Networking tab --Network tags: set nem-http-allow and nem-https-allow --Network interface: Click the edit icon to set --Select "Create IP Address" from "Ephemeral" for "External IP", enter an appropriate name (for example, nem-mainnet-1) in the dialog, and click "Reserve". --Click "Finish" to save the network interface settings --Click "Create" to create the VM

SSH login to the VM

After creating the VM, after a while, the startup of the VM is completed, and you can click the select list "SSH" in the column "Connect" in the list of VMs. .. Subsequent operations are mainly performed on that screen.

3. Java installation

Java 8 is required to run NIS1, so install it. Let's do other updates as well. Execute the following command.

update

sudo apt update

Install Java 8 (OpenJDK 8 to be exact)

sudo apt install -y openjdk-8-jdk

Confirm Java 8 installation

Execute the following command

java -version

If you get the following results, the installation is successful.

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1ubuntu1-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

4. Download and unzip the main body of NIS1

The tool for running NIS1 on Linux is compressed into a tgz file and distributed from the following URL, so download it, unzip it, and use it. Latest version as of May 30, 2020: https://bob.nem.ninja/nis-0.6.97.tgz Page with links to other versions and related data: https://bob.nem.ninja/

download

You can download it as the file name nis1.tgz with the following command. The file name of nis1.tgz can be anything as long as it has a .tgz extension, but please read the file name appropriately in the following decompression command.

curl -o nis1.tgz https://bob.nem.ninja/nis-0.6.97.tgz

Defrost

(If you downloaded it with the file name nis1.tgz at the time of download) You can unzip it with the following command. When you unzip it, you will have a folder called package.

tar -xzvf nis1.tgz

5. Fixed node startup script

Move to the package folder with the following command and

cd package

Open the nix.runNis.sh file (shell script for starting the node) with an editor and modify it with the following command. Here is an example of using nano as the editor, but please use the editor of your choice.

nano nix.runNis.sh

At a minimum, the changes you need to make are:

Relaxed maximum memory usage

With the default settings, it seems that there is not enough memory to start normally. This time, I created a VM with 16GB of memory, so I will take the plunge and try it as a setting that can use up to 14GB of memory for the NIS1 process. (Maybe even a slightly more conservative maximum memory allocation will work.)

Add nohup to the beginning of the startup command and&to the end of the startup command

When I run the node-initiated shell script by default, NIS1 runs in the foreground. This will force NIS1 to quit when you log off. By adding nohup to the beginning of the startup command and&to the end of the startup command, it seems that you can keep it running in the background even if you are not logged in.

変更前のnix.runNis.sh

#!/bin/bash

cd nis
java -Xms512M -Xmx1G -cp ".:./*:../libs/*" org.nem.deploy.CommonStarter
cd -

変更後のnix.runNis.sh

#!/bin/bash

cd nis
nohup java -Xms512M -Xmx14G -cp ".:./*:../libs/*" org.nem.deploy.CommonStarter &
cd -

If you use nano as the editor, to overwrite and save, first press "Ctrl" + "X", then "Y", and when the file name is displayed, press "Enter".

6. Start the node

Now that you are ready to start the node, start the node with the following command.

./nix.runNis.sh

7. Confirmation after startup

Since it is set to operate in the background, no special message etc. is displayed on the screen even after startup, so hit the API to get the block height and node information from the outside and an appropriate value will be returned. Let's check. Please read the IP address according to your environment.

Block height

http://35.200.86.73:7890/chain/height

--Response example

{"height":44967}

Since it is just after launch, you can see that the synchronization has not been completed yet, and the synchronization has been completed only to a place much lower than the latest block height. This number will increase little by little, and when it catches up with the latest block height, node construction will be a break.

Originally, in order to shorten the time required for this synchronization, it is possible to separately download and expand the DB information of the blockchain up to the latest to some extent, so I would like to add that method somewhere. I am.

Node information

http://35.200.86.73:7890/node/info

--Response example

{
	"metaData": {
		"features": 1,
		"application": null,
		"networkId": 104,
		"version": "0.6.97-BETA",
		"platform": "Private Build (1.8.0_252) on Linux"
	},
	"endpoint": {
		"protocol": "http",
		"port": 7890,
		"host": "35.200.86.73"
	},
	"identity": {
		"name": "NBAKSE34ZOWCIZC4AUU2XXMNAANH7BSXMGV4AYEC",
		"public-key": "79140632da5b00327bba8d5cb3b258d04843c10520cb757fde1a5a859f0ae567"
	}
}

In addition, if you hit the API of node information etc., you will deepen your confidence in your own settings. You can see that networkId 104 indicates MAIN_NET, which indicates the NIS1 and Java versions, as well as the API protocol, port and hostname. Also, what I found a little interesting is that a NEM account is automatically generated as an identity when the node is started and used as an identifier. (I wonder if these accounts are related to the harvest settings, etc.)

Impressions

After working through it, I thought, "Is it just running a shell script?" (So, there are many parts that need not be explained ... and you may not be able to find much information such as a solid procedure ... Of course, it is done internally after starting the shell script. I get the impression that being there is very complicated ...)

Start it for the time being and leave it alone! If the level is good (≒ this article level), I got a simple impression.

However, where is the maximum memory value corrected, the default is foreground execution, and where is the NIS1 main program? I feel that there are some "addiction points" in that respect, so I felt happy for users who will build NIS1 in the future if there are no such addiction elements by default.

Contents to be added (want to)

I would like to add the following contents in due course.

--Appropriate management such as service conversion --Domain settings --SSL --Harvest acceptance settings --Settings related to incentives after the launch of the next version

Reference information

I referred to the following information. Thank you to all the pioneers.

--NIS1 node construction of NEM official blog: https://blog.nem.io/ubuntu-installation-guide-standalone/ --NIS1 node SSL conversion of NEM official blog: https://blog.nem.io/https-nis-node/ --NIS1 node construction of testnet: https://blog.44uk.net/2018/02/12/up-nem-testnet-node/ --Mainnet NIS1 node construction: https://mizunashi-rin.hatenablog.jp/entry/2017/02/15/002342 --NIS 1 node SSL: https://blog.44uk.net/2017/10/31/nis-with-https-by-dehydrated/

Recommended Posts

How to build a NEM (current version) node (NIS1: NEM Infrastructure Server)
How to build my own Linux server
How to build a sphinx translation environment
How to set up a local development server
[PyTorch] Sample ⑧ ~ How to build a complex model ~
How to build a Django (python) environment on docker
CTF beginner tried to build a problem server (web) [Problem]
How to build a development environment for TensorFlow (1.0.0) (Mac)
How to create a simple TCP server / client script
How to run Django on IIS on a Windows server
How to build a Python environment on amazon linux 2
How to use GitHub on a multi-person server without a password
How to specify a public directory Python simple HTTP server
Build a Python environment and transfer data to the server
How to build a python2.7 series development environment with Vagrant
Tabpy 1.0 (2020-01 version) How to install
How to call a function
How to hack a terminal
How to change Python version
How to build a LAMP environment using Vagrant and VirtulBox Note
Overview of how to create a server socket and how to establish a client socket
A note on how to check the connection to the license server port
How to build a Python environment using Virtualenv on Ubuntu 18.04 LTS
How to continue processing after returning a response on aiohttp Server
How to make a Japanese-English translation
How to put a symbolic link
How to make a slack bot
How to create a Conda package
How to make a crawler --Advanced
How to get the Python version
How to make a recursive function
How to create a virtual bridge
How to create a Dockerfile (basic)
[Blender] How to make a Blender plugin
How to delete a Docker container
How to make a crawler --Basic
How to build MongoDB C driver
How to check Linux OS version
How to create a config file
Summary of how to build a LAMP + Wordpress environment with Sakura VPS
The programmer tried hard to build a server to overcome it, saying that infrastructure seems to be difficult. WebAP server edition