[JAVA] Let's try Zoomdata (Introduction)
** Zoomdata ** was founded in Virginia, USA in 2012, and is a company that provides advanced solutions for visualization and analysis of big data, and introduced its product ** Zoomdata **. By doing so, it will be possible to realize visual analytics compatible with hybrid clouds that can transparently utilize cloud environments and on-premises environments.
Currently (as of August 2017), a limited-time version on the cloud and a limited-time software download version that can be built on the on-premises environment are provided as trial environments, and evaluations that take advantage of their respective characteristics. It is possible to use it properly according to the strategy. If you want to build a big data environment on a certain scale from the beginning and then perform PoC etc. on it, register for a trial of the cloud version and follow the terms and instructions of the provider, and concrete environment construction, operation verification and performance It would be good if you could proceed with the evaluation.
Who is ** Zoomdata ** for the time being in this project? And what can you do? I want to get a feel for it! I would like to make it a direction to summarize as a "first step" content, and aim for the next step, "Then, if you use ** Zoomdata ** ..." I will.
(1) First
First of all, it quickly creates an environment to install ** Zoomdata **. There are various principles, assertions, styles, and schools for this part, so please read it as a reference for the time being, and finally aim to be able to link the environmental elements that can operate ** Zoomdata **. I would like. I think there are various routes for climbing mountains depending on each skill and individuality, but it is OK if you can finally reach the summit (goal) safely after completing the necessary work.
You can enter the URL link for trial registration from the following. (As of August 2017)
Originally, it would be nice if you could reach the trial registration page while "picking up" various information from the top page of ** Zoomdata **, but at the beginning you declared "crisp" I've done it, so let's hurry up and hurry up ... (sweat)
Please fill in each item on the right side of the screen
(A) Name (B) Last name (C) Business email address The final installation instructions will be sent to this address. (D) Company name (E) Contact phone number
I don't think you will receive a direct call from the United States, but since the Japan branch was recently established, various information provision may begin in Japan in the future.
Don't forget to check it at the end and press the green button to finish.
After a while, the installation information will be sent to the registered e-mail address, so please do not delete it.
(2) About the trial environment
Information about the environment required for installation can be confirmed via the email because the URL is embedded in the previous email, but since it is a story to proceed with environment preparation in advance, hurry up and put the link below Put.
Information about the installation environment
By the way, the OS used for this work ...
I'm not particular about it, but I'll try CentOS 7 for the time being. Since the size of the memory is ** Zoom data ** in terms of settings, I would like to secure it as large as possible, but please keep at least about 8GB. Also, we will start with a disk capacity of 40GB.
Also, the important point is that this work emphasizes the point of "trying for the time being", so we will utilize the virtual environment of the desktop. This time, we will use VMware Workstation 12 Player, but since the work is basically the same for Oracle VM VirtualBox, you should choose the one you are familiar with.
By the way, the hardware used for this verification will be a hyper environment (due to the work in the table) that the memory 32GB is Windows with Core i7 of the notebook made by D company. The more cores you have, the better, but if you feel like checking the functions with one user, I think that hardware that can allocate about 2 cores can handle it. However, it is better to have a lot of memory, so it is better to use 8GB even in the situation of 1 user trial. Also, I want to evaluate the performance! In that situation, we recommend trying with the configuration recommended by ** Zoomdata **.
The method of downloading CentOS 7 and installing it in a virtual environment is no problem as long as you change the memory settings and storage capacity. Also, regarding the number of CPUs, it is difficult to make a definitive setting because it depends on the original CPU specifications, but if possible, set two or more. Other items can be entered normally, so explanations are omitted.
Also, this time it will be convenient later, so I will install it with a server configuration with GUI. This area can be freely selected, so I think it should be selected according to each case.
(3) Introduction of Java
In this installation setting, I think that OpenJDK is installed in advance as the initial setting, so use the following command to check the status. (The output for the command may vary depending on the version selected, etc.)
$ java -version openjdk version "1.8.0_141" OpenJDK Runtime Environment (build 1.8.0_141-b16) OpenJDK 64-Bit Server VM (build 25.141-b16, mixed mode)
Although it is within the support range in terms of version, you can easily change the java VM with the alternatives command, so this time I will change it to Oracle Java 8 and build it.
Go to Oracle's Java Downloads page in your browser and get jdk-8u144-linux-x64.rpm (at the time of writing). You can check Accept and then click the target (I think you can get it from the command line, but since it is installed with GUI, you can proceed quickly via the browser)
I think that the download will be in the home download folder by default, so if necessary, copy it to the working folder and issue the following command.
$ sudo yum localinstall jdk-8u144-linux-x64.rpm
It will come in quickly, so next use the alternatives command.
$ sudo alternatives --config java
Two Javas are displayed, and I think that OpenJDK is selected, so enter the number of the person who installed this time to switch.
Use the command to check if the switch has been made.
$ java -version java version "1.8.0_144" Java(TM) SE Runtime Environment (build 1.8.0_144-b01) Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
For this part, there is also a choice to make with general environment variables, so you can freely choose how to proceed.
(4) Introduction of Spark
Next, we will introduce Spark. This time, since it is a server configuration with GUI, we will go quickly by making full use of the browser as in Java.
Visit the Apache Spark home page (https://spark.apache.org/).
Go to the specified page via the Download Spark button.
There are several versions, but the above configuration information has a recommended description of 1.5.1, so this time I will try using 1.6.3, which seems to be the final version of that flow.
When you select an item, you will see a link called spark-1.6.3-bin-hadoop2.6.tgz. So click on it to download it locally.
$tar -xzvf spark-1.6.3-bin-hadoop2.6.tgz
Expand with
$ sudo mv spark-1.6.3-bin-hadoop2.6 spark $ sudo cp -rf spark/ /usr/local/
$ vi .bash_profile
Now add * / usr / local / spark / bin * to your PATH.
$ source ~/.bash_profile
It was installed successfully, or try launching Spark.
$ spark-shell
After this, it's OK if you stand up with the spark logo.
Type * exit * in * scala> * to return to the normal command prompt.
(5) Installation and initial setting of ** Zoomdata **
At this point, it's a little more patience to be able to meet the ** Zoomdata ** login screen. The above-mentioned email from ** Zoomdata ** contains information about system installation, so please use that information for final processing. When all the processing is completed successfully, it will be the work to call the first login screen. Call ** Zoomdata ** in the URL field of a supported web browser
https://xxx.xxx.xxx.xxx:8443
Please enter. (Xxx.xxx.xxx.xxx is the IP address of the machine on which ** Zoomdata ** is installed.) Do you really connect due to signature problems? However, if the address is correct, proceed with the process as it is. Once connected, the first login screen will appear.
This time, it will be a trial for a limited time, so select ** Skip Activation ** on the lower right side to proceed with the process.
In ** Zoomdata **, each system and service has its own administrator. The former is ** Supervisor **, which defines only one account unique to the server, and the latter is ** Admin **, which can define multiple accounts for each user group. Set a password for each on this screen, and when all the information is consistent (you will be asked to enter the same information twice), the ** Change Password ** button will be highlighted. Press that button. The installation is finished.
If all goes well, you'll see the normal login screen.
For the time being, thank you for your hard work! !!
By the way, in the initial setting state, I think that the port communication problem will occur due to being blocked by the firewall, but set it in advance using the GUI tool, or add / change the setting each time using the command line. Please give me. If you go quickly
$ sudo systemctl stop firewalld
So, there is a method of forcibly stopping.
(6) Finally
From the next time onward, I would like to actually try ** Zoomdata ** using some patterns. In order to take advantage of the characteristics of ** Zoomdata **, the process of linking the time transition and the data until the final data is finalized is not a post-guess, but a situation that actually occurs in real life at high speed. It is necessary to pay attention to the part that by visual analysis, new strategies, problem solving, development of competitive advantage, etc. can be implemented in the reality that is actually generated and transitioned. Indeed, business and daily problems are occurring in the field, and the situation is changing from moment to moment on the time axis ... By utilizing ** Zoomdata **, now It will be possible to reform and improve (real), and it may be possible to change the future actively and strategically. Of course, there is no present without the past, but if you change the real present, the real future will surely change ... It may be said that ** Zoom data ** is a solution with the potential.
Recommended Posts