(First half) EC2 provisioning using Vagrant (Second half) Start Jupyter (IPython Notebook) using Docker
I will write to Qiita for the first time. During the New Year holidays, I tried Deep learning from scratch and, hopefully, study TensorFlow, and study environment construction as well. , It is an introduction that also serves as a memorandum. Since the first and second half of this time are independent, I think it is possible to refer to only one of them.
:)
:(
As a premise
Is required.
I think this will be helpful.
Refer to HashiCorp's GitHub page: Vagrant AWS Provider to configure the settings.
First, install the plug-in to use AWS.
$ vagrant plugin install vagrant-aws
It is a BOX equivalent to a machine image, but since a dummy box is open to the public for EC2, I will use this. (Officially, the box name is dummy, but it's a bit vague, so I changed it to ec2.)
$ vagrant box add ec2 https://github.com/mitchellh/vagrant-aws/raw/master/dummy.box
Next, move to the working directory and create a Vagrantfile.
$ mkdir jupyter_ec2
$ cd jupyter_ec2
$ vim Vagrantfile
Edit the Vagrantfile and set various parameters. This time, we have specified booting with a VPC created in advance.
There seem to be quite a few other parameters, and if you want to specify them, you can use the README or AWS API documentation. I think it is better to refer to).
Vagrantfile
Vagrant.configure("2") do |config|
config.vm.box = "ec2" # dummy box name
config.vm.provider :aws do |aws, override|
#AWS Credentials
aws.aws_dir = ENV['HOME'] + "/.aws/"
aws.aws_profile = "YOUR_PROFILE"
aws.keypair_name = "YOUR_AWS_ACCESS_KEY"
#SSH credentials
override.ssh.username = "ec2-user"
override.ssh.private_key_path = "SSH_KEY_PATH"
#VPC settings
aws.region = 'ap-northeast-1'
aws.availability_zone = 'ap-northeast-1a'
aws.subnet_id = 'subnet-XXXXXXXX'
aws.security_groups = 'sg-XXXXXXXX'
#EC2 settings
aws.ami = "ami-9f0c67f8" # Amazon Linux 2016.09.01
aws.instance_type = 'm4.large'
aws.elastic_ip = true
aws.associate_public_ip = true
aws.tags = { 'Name' => 'NAME_TAGS_VALUE' }
#EBS settings
aws.block_device_mapping = [
{
'DeviceName' => "/dev/xvda",
'Ebs.VolumeSize' => 50,
'Ebs.DeleteOnTermination' => true,
'Ebs.VolumeType' => 'standard',
}
]
end
end
After editing, start EC2.
$ vagrant up --provider=aws
When it doesn't work,
$ export VAGRANT_LOG=DEBUG
Then re-execute vagrant up
and it will display a detailed error message.
If you want to undo the error message ʻexport VAGRANT_LOG = WARN `
Make sure that the security group SSH is also open properly. If the upper limit is not relaxed, the upper limit of the number of EIPs tends to be caught.
When it finishes starting
$ vagrant ssh
Check if you can SSH login with.
$ vagrant halt
You can stop the instance with, so when you are not working, stop it as appropriate to save money.
First, install Docker on EC2 by referring to Installing Docker.
$ sudo yum update -y
$ sudo yum install -y docker
Start the docker daemon so that it will start automatically even after the instance is restarted.
$ sudo service docker start
$ sudo chkconfig docker on
Add ec2-user to the docker group so that you can add docker commands without sudo privileges.
$ sudo usermod -a -G docker ec2-user
Once you exit and log in again, docker will start without sudo.
Next, describe the Dockerfile.
$ vim Dockerfile
This time, for the sake of simplicity, we will install TensorFlow based on the image of anaconda.
In CMD, specify that jupyter should be started at docker run
.
FROM continuumio/anaconda:latest
RUN mkdir /opt/notebooks
RUN pip install tensorflow
# Another installations comes here.
CMD ["/opt/conda/bin/jupyter", "notebook", "--notebook-dir=/opt/notebooks", "--ip='*'", "--port=8888", "--no-browser"]
After editing, create a container image from the Dockerfile.
$ docker build -t myjupyter .
Finally start the container. Specify volume and mount it so that the notebook can be made persistent and can be seen from the host side. Also, the --restart option causes the docker daemon to automatically start the container even after the instance is stopped / started.
$ mkdir notebooks
$ docker run -p 8888:8888 -v /home/ec2-user/notebooks:/opt/notebooks --restart=always myjupyter
After allowing TCP port 8888 communication in the security group, access [Elastic IP]: 8888
with a browser to confirm that Jupyter has started.
Only when studying with this, learn on the browser with vagrant up
, stop charging with vagrant halt
when quitting, build from the same Dockerfile when you want to try locally, easy porting if you get ipythonnb via Vagrant It became an environment ...! It should be, so I will correct it as appropriate while studying books.
Recommended Posts