[Java] Set the time from the browser with jsoup

Requirements

• In a certain application, the time goes wrong immediately.
• Cannot connect to NTP. In-house NTP is also not possible.
• The application can be accessed from the browser by IP address.
• You can set the time manually from the browser.
• Nearly 30 servers running the application are scattered all over the country.
• Servers may increase or decrease in the future.
• Enter the login password when executing the batch (argument or prompt)
• Java is introduced as an internal standard. Procedures are required to introduce other languages.

Execution environment

• Windows8.1, 10 (mainly 8, but likely to be 10 in the future)
• Java 1.8 and above

Development environment

• Windows7
• Eclipse
• Java1.8
• jsoup 1.10.2
• gradle
• Make jar executable from bat file with application plugin

Source code

Time adjustment start batch

• Placed under src \ dist \

TimeAdjusterMain.bat

@echo off

rem ------------------------------------------------
rem ---Time adjustment batch
rem ------------------------------------------------

rem --Based on the current directory
cd /d %~dp0

echo ****Start time adjustment: %date% %time%

rem --Check arguments(Add xx to prevent it from becoming empty)
if xx%1==xx goto confirmPw
goto setPw



:confirmPw

rem --Enter the administrator password if no arguments are specified
set /P PASSWD=Please enter the administrator password: 

goto exec



:setPw

rem --Set as administrator password if argument is specified
set PASSWD=%1

goto exec



rem --Execution of processing batch
:exec

rem --Start execution of processing batch
call bin\TimeAdjuster.bat %PASSWD%

echo ****Time adjustment end: %date% %time%

rem --Stay without completing
pause

IP address list to be processed tsv

• Placed under src \ dist \

list.tsv

10.20.30.40 abc123 ABC123 A branch B floor AB # 1
10.20.50.60 def456 DEF456 C branch D floor CD # 2

Java source code

Main

TimeAdjusterMain.java

package jp.co.sample.timeadjuster;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import jp.co.sample.timeadjuster.bean.SettingBean;
import jp.co.sample.timeadjuster.launcher.TimeAdjusterLauncher;

public class TimeAdjusterMain {

	private static Logger logger = LoggerFactory.getLogger(TimeAdjusterMain.class);

	public static void main(String[] args) {
		logger.info("TimeAdjusterMain started");

		//Setting information
	    SettingBean setting;
	    try {
			// args[0]:Administrator password
			// args[1]: src/dist folder absolute path
			setting = new SettingBean(args[0], args[1], "list.tsv");
		} catch (Exception e) {
			logger.error("Setting information error", e);
			return;
		}

	    //Processing execution
	    TimeAdjusterLauncher launcher = new TimeAdjusterLauncher(setting);
	    try {
	    	launcher.execute();
		} catch (Exception e) {
			logger.error("Time adjustment processing error", e);
			return;
		}
	}
}

Setting information

SettingBean.java

package jp.co.sample.timeadjuster.bean;

import java.io.BufferedReader;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.ArrayList;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVRecord;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class SettingBean {

	public static Logger logger = LoggerFactory.getLogger(SettingBean.class);

	/**
	 *Administrator password
	 */
	String passwd;

	/**
	 *Execution path reference directory
	 */
	String basePath;

	/**
	 *tsv filename
	 */
	String tsvName;

	/**
	 *Server information list
	 */
	ArrayList<ServerBean> serverList = new ArrayList<>();


	public SettingBean(String passwd, String basePath, String tsvName) throws IOException {
		this.passwd = passwd;

		this.basePath = basePath;

		logger.debug("this.basePath: "+ this.basePath);

		this.tsvName = tsvName;

		logger.debug("this.tsvName: "+ this.tsvName);

		File targetFile = new File(this.basePath, this.tsvName);
		if (!targetFile.isFile()) {
			logger.error("Please specify an existing file path.: "+ path);
			return;
		}

		//Read the file
		try (BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(targetFile),"UTF-8"));) {
			//TAB separated file with header. Exclude whitespace
			Iterable<CSVRecord> records = CSVFormat.TDF.withIgnoreEmptyLines().withIgnoreSurroundingSpaces().parse(reader);
				for (CSVRecord record : records) {

					logger.debug(record.get(0));

					//Check the IP address type for the IP address of the server.
					//When you create a tsv file with Notepad, it starts at the beginning?Because it will enter
					Pattern p = Pattern.compile("\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}");
					Matcher m = p.matcher(record.get(0));

					if (!m.find()) {
						throw new RuntimeException("The IP address is specified incorrectly. IP address:"+ record.get(0));
					}

					//Get server information and add it to the list
					ServerBean server = new ServerBean(m.group(), record.get(1), record.get(2), record.get(3), record.get(4), record.get(5));
					serverList.add(server);

					logger.debug("server: "+ server.getIpaddress() + " / "+ server.getSerial_no() + " / "+ server.getDevice_name());
				}
		} catch (FileNotFoundException e) {
			logger.error("TSV file read error", e);
			throw e;
		} catch (IOException e) {
			logger.error("TSV file read error", e);
			throw e;
		}

	}


	public String getPasswd() {
		return passwd;
	}


	public void setPasswd(String passwd) {
		this.passwd = passwd;
	}


	public String getBasePath() {
		return basePath;
	}


	public void setBasePath(String basePath) {
		this.basePath = basePath;
	}


	public String getTsvName() {
		return tsvName;
	}


	public void setTsvName(String tsvName) {
		this.tsvName = tsvName;
	}


	public ArrayList<ServerBean> getServerList() {
		return serverList;
	}


	public void setServerList(ArrayList<ServerBean> serverList) {
		this.serverList = serverList;
	}

}

Server information (information read from tsv)

ServerBean.java

package jp.co.sample.timeadjuster.bean;

public class ServerBean {
	/**
	 *IP address
	 */
	String ipaddress;

	/**
	 *hostname
	 */
	String host_name;

	/**
	 *Serial number
	 */
	String serial_no;

	/**
	 *Base name
	 */
	String base_name;


	/**
	 *Installation location
	 */
	String installation_location;

	/**
	 *Equipment name
	 */
	String device_name;

	public ServerBean(String ipaddress, String host_name, String serial_no, String base_name,
			String installation_location, String device_name) {
		super();
		this.ipaddress = ipaddress;
		this.host_name = host_name;
		this.serial_no = serial_no;
		this.base_name = base_name;
		this.installation_location = installation_location;
		this.device_name = device_name;
	}

	public String getIpaddress() {
		return ipaddress;
	}

	public void setIpaddress(String ipaddress) {
		this.ipaddress = ipaddress;
	}

	public String getHost_name() {
		return host_name;
	}

	public void setHost_name(String host_name) {
		this.host_name = host_name;
	}

	public String getSerial_no() {
		return serial_no;
	}

	public void setSerial_no(String serial_no) {
		this.serial_no = serial_no;
	}

	public String getBase_name() {
		return base_name;
	}

	public void setBase_name(String base_name) {
		this.base_name = base_name;
	}

	public String getInstallation_location() {
		return installation_location;
	}

	public void setInstallation_location(String installation_location) {
		this.installation_location = installation_location;
	}

	public String getDevice_name() {
		return device_name;
	}

	public void setDevice_name(String device_name) {
		this.device_name = device_name;
	}

}

Time adjustment process

TimeAdjusterLauncher.java

package jp.co.sample.timeadjuster.launcher;

import java.io.IOException;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;

import org.apache.commons.lang3.StringUtils;
import org.jsoup.Connection.Method;
import org.jsoup.Connection.Response;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import jp.co.sample.timeadjuster.bean.ServerBean;
import jp.co.sample.timeadjuster.bean.SettingBean;

public class TimeAdjusterLauncher {

	public static Logger logger = LoggerFactory.getLogger(TimeAdjusterLauncher.class);

	/**
	 *Setting information
	 */
	SettingBean setting;

	/**
	 *Login screen display result
	 *Hold cookies etc.
	 */
	Response loginRes;

	/**
	 *constructor
	 *
	 * @param setting setting information
	 */
	public TimeAdjusterLauncher(SettingBean setting) {
		super();

		//Hold settings
		this.setting = setting;
	}

	/**
	 *Executes the time adjustment process
	 * @throws RuntimeException
	 * @throws IOException
	 */
	public void execute() throws RuntimeException, IOException {

		//Loop processing for each server
		for (ServerBean server : setting.getServerList()) {
			logger.info("Start time adjustment****************** ");
			logger.info("Target server: "+ server.getIpaddress() + " / "+ server.getDevice_name());

			//Login
			login(server);

			//Check the currently set time
			String zone = getTimer(server);

			//New time setting
			adjustTimer(server, zone);
		}

	}

	/**
	 *Login process
	 * @throws RuntimeException
	 * @throws IOException
	 */
	private void login(ServerBean server) throws RuntimeException, IOException {
		//Login process execution
		String loginUrl = "http://"+ server.getIpaddress() + "/logon.cgi";
		try {
			loginRes = Jsoup.connect(loginUrl)
				    .data("Password", setting.getPasswd())
				    .data("Mode", "Admin")
				    .data("Lang", "Japanese")
				    .method(Method.POST)
				    .execute();
		} catch (IOException e) {
			logger.error("Initial screen login error", e);
			throw e;
		}

//		logger.debug(loginRes.parse().outerHtml());

		//HTTP status matching
		checkStatusCode(loginRes, loginUrl);


		if (StringUtils.contains(loginRes.parse().outerHtml(), "/pages/_top2.htm") ) {
			//If the login error screen is redirected
			//It means that the administrator login failed, so the error ended.
			throw new RuntimeException("Administrator login failed. It suspends processing.");
		}


		//Redirect after successful login
		//Processing that is not really necessary. I just went for confirmation.
		String redirectUrl = "http://"+ server.getIpaddress() + "/pages/_devadm.htm";
		Response redirectRes;
		try {
			redirectRes = Jsoup.connect(redirectUrl)
				//Set the cookie obtained at login
			    .cookies(loginRes.cookies())
			    .method(Method.GET)
			    .execute();
		} catch (IOException e) {
			logger.error("Initial screen login redirect error", e);
			throw e;
		}

//		logger.debug(redirectRes.parse().outerHtml());

		//HTTP status matching
		checkStatusCode(redirectRes, redirectUrl);

	}

	/**
	 *Gets the time currently set on the server
	 *
	 * @param server
	 * @throws RuntimeException
	 * @return zone
	 * @throws IOException
	 */
	private String getTimer(ServerBean server) throws RuntimeException, IOException {

		//Get the server time setting screen
		String timerUrl = "http://"+ server.getIpaddress() + "/pages/ed_time.htm";
		Response timerRes;
		try {
			timerRes = Jsoup.connect(timerUrl)
					//Set the cookie obtained at login
				    .cookies(loginRes.cookies())
				    .method(Method.GET)
				    .execute();
		} catch (IOException e) {
			logger.error("Time setting screen acquisition error", e);
			throw e;
		}

		//HTTP status matching
		checkStatusCode(timerRes, timerUrl);

//		logger.debug(timerRes.parse().outerHtml());

		Document doc = timerRes.parse();

		//Gets the time currently set on the server.
		String year = doc.getElementsByAttributeValue("name", "DateYYYY").get(0).val();
		String month = doc.getElementsByAttributeValue("name", "DateMM").get(0).val();
		String day = doc.getElementsByAttributeValue("name", "DateDD").get(0).val();
		String hour = doc.getElementsByAttributeValue("name", "TimeHH").get(0).val();
		String minutes = doc.getElementsByAttributeValue("name", "TimeMM").get(0).val();
		String zone = doc.select("select option[selected]").get(0).val();

		logger.info("Current server time: "+ year + "/"+ month + "/"+ day + " "+ hour +":"+ minutes + " ("+ zone + ")");

		//Only zone uses the existing value, so it is returned.
		return zone;
	}

	/**
	 *Set the time.
	 *
	 * @param server
	 * @param zone
	 * @throws RuntimeException
	 * @throws IOException
	 */
	private void adjustTimer(ServerBean server, String zone) throws RuntimeException, IOException {
		//Gets the current time of the execution environment.
		LocalDateTime now = LocalDateTime.now();

		logger.info("Current time: "+ now.format(DateTimeFormatter.ISO_LOCAL_DATE_TIME));

		//Acquires the timing of the next time adjustment in order to adjust to 0 seconds after 1 minute.
		LocalDateTime nextLdt =
		        LocalDateTime.now()
		        .plusMinutes(1)
		        .truncatedTo(ChronoUnit.MINUTES);

		logger.debug("Time to match: "+ nextLdt.format(DateTimeFormatter.ISO_LOCAL_DATE_TIME));

		//Wait milliseconds
		long localDiffMsec = ChronoUnit.MILLIS.between(now, nextLdt);

		logger.debug("Wait for milliseconds until the time to match: "+ localDiffMsec);

		try {
			//Wait milliseconds, wait
			Thread.sleep(localDiffMsec);
		} catch (InterruptedException e) {
			logger.error("Server time adjustment millisecond wait failure");
		}

		logger.info("Time adjustment execution");

		//Server time adjustment processing execution(POST)
		//For some reason it worked without cookies, so I just throw POST
		String timerUrl = "http://"+ server.getIpaddress() + "/settime.cgi";
		Response timerRes;
		try {
			timerRes = Jsoup.connect(timerUrl)
				    .data("DateYYYY", ""+ nextLdt.getYear())
				    .data("DateMM", ""+ nextLdt.getMonthValue())
				    .data("DateDD", ""+ nextLdt.getDayOfMonth())
				    .data("TimeHH", ""+ nextLdt.getHour())
				    .data("TimeMM", ""+ nextLdt.getMinute())
				    .data("TimeZone", ""+ zone)
				    .method(Method.POST)
				    .execute();
		} catch (IOException e) {
			logger.error("Server time adjustment posting error", e);
			throw e;
		}

//		logger.debug(timerRes.parse().outerHtml());

		//HTTP status matching
		checkStatusCode(timerRes, timerUrl);

		logger.info("Time adjustment completed: "+ server.getIpaddress() + " / "+ server.getDevice_name() + " / " + nextLdt.format(DateTimeFormatter.ISO_LOCAL_DATE_TIME));

	}

	/**
	 *HTTP status check
	 *
	 * @param res
	 * @param url
	 * @throws RuntimeException
	 */
	private void checkStatusCode(Response res, String url) throws RuntimeException {

		if (res.statusCode() != 200) {
			//HTTP status is 200(normal)If not, an error
			throw new RuntimeException("URL access failure: url="+ url + " status="+ res.statusCode());
		}
	}

}

Impressions

• Scraping often comes up when you say jsoup, but there is also such a usage.
• The jsoup code of login-> cookie retention-> post-login processing was not so much in Japanese, so I took this opportunity to summarize it.
• The hardest part of jsoup is finding out what the URL is where the process is actually executed. Specified in form, redirected ...
• When I touch tsv with Notepad, a mysterious phenomenon that a? Is inserted at the beginning has occurred, so I use a regular expression to get the IP address. I can't specify an editor ...
• It's an old custom to make beans and pack them, but these days it's not like this ... I have to study new methods ^^;

I would like to do my best this year as well. Thank you very much.