Use AWS interpreter with Pycharm

things to do

Develop with local pycharm using the AWS instance interpreter!

background

When it is extremely difficult to install a specific module, it is very easy to use the AWS AMI or docker container that is already installed. However, I want to code with an IDE such as pycharm, so entering the server and developing with vim is a bit difficult. .. With pycharm, I found a function that allows me to log in to a server with remote and code using the interpreter of that server, so I will try it and write a note.

environment

Method

1 Press Add Remote in the pycharm interpreter settings.

Click Pycharm Preferences-> Project: hoge-> Project Interpreter and the following screen will appear. If you click on this gear in the upper right, スクリーンショット 2017-05-04 16.22.06.png The following window will appear, so press "Add Remote". スクリーンショット 2017-05-04 16.37.16.png

2 SSH settings

When you press Add Remote, the interpreter Configure will appear, so when you press SSH Credentials, the following setting screen will appear. スクリーンショット 2017-05-04 16.22.29.png

Specify the following for each here.

--Specify a public IP such as Host: 54.xx.xx.xx --port: Specify the port number for ssh connection --user name: ec2-Specify a ssh connection user such as user --Specify the connection method of Auth type: password or pem --password: Specify the password or the save destination of the pem file --Python interpreter path: Specify the interpreter path.

This is OK. Occasionally, depending on the AMI, the working environment may be built as a root user instead of a general user such as ec2-user, so in this case I will play with sshd_config and the public key.

# sshd_Modify config
vi /etc/ssh/sshd_config
# PermitRootLogin forced-commands-only below.
PermitRootLogin without-password

#Copy of public key
mv /root/.ssh/authorized_keys /root/.ssh/authorized_keys_bak
cp -f /home/ec2-user/.ssh/authorized_keys /root/.ssh/authorized_keys
service sshd reload

I've written so far, but I've noticed that this feature is quite annoying. .. First, it cannot be executed unless the local path exists on the remote side. Perhaps it is correct to use this function by uploading what was written locally to the remote as it is? Then, there are some modules that cannot be imported for some reason. I'm using caffe, but import caffe says it doesn't. There is no pycaffe under anaconda, so I may have to move it under anaconda.

Hmmm, can I use it to create a source with a local IDE that doesn't give an import error and raise it? ?? Wonder. .. It's pretty sad that I can't debug, though. .. .. I will investigate the above problem if there is spare capacity.

Postscript

I noticed the correspondence between local and remote paths and the auto upload function, so I added it.

How to map local and remote

1 Click "Edit Configration .." from "RUN" on the upper tab. 2 Click the right side of Path Mapping. 3 Set the local path and Remote path. that's all.

Auto upload method

Even if you associate it, the work is local, so you have to upload the changed source to AWS one by one. How to automatically upload the source modified by local pycharm to AWS. 1 In Preferences-> Deployment, press the + button. 2 Select SFTP and specify the global IP Root path = "/" keypair. You can leave Browse files on server as it is. 3 In Preferences-> Deployment-> Options, change Upload changed files automatically ... to Always and specify Override default permissions on files: as any permissions. that's all.

Now you can code locally in pycharm and develop it with immediate reflection on AWS. You can also debug. Great.

When using cuda

An error may occur because there is no path for cuda's lib system. In that case, set the following in Environment Variable in the Configurations of Run.

LD_LIBRARY_PATH /usr/local/cuda/lib64:$LD_LIBRARY_PATH

When using the remote interpreter, ": cannot connect to X server" is displayed in debugging and an error occurs.

I don't know exactly, but it seems to cause an error if Xcode cannot be used on the server. Therefore, work is required to complete with local Xcode.

  1. Connect from local with the following.
ssh -Y username@ipaddress

With the connection of 2.1, check the contents of DISPLAY with the following command.

echo $DISPLAY
> localhost:10.0
  1. Set the contents of 2 in Environment Variable of pycharm.
DISPLAY localhost:10.0

Debug with pycharm with 4.1 connection. OK.

If you want to check the tensorboard etc. on the cloud such as AWS but the port is not released

If you connect from the local with the following, specify the port to use on the server as 9999, and go to the local browser with localhost: 8888, the access will be port forwarded to 9999 and you can display it. ..

ssh -L 8888:localhost:9999 user@pablic_ip

Recommended Posts

Use AWS interpreter with Pycharm
How to use Cmder with PyCharm (Windows)
PyCharm remote interpreter with SSH (WSL2, venv, pyenv)
Use python in Docker container as Pycharm interpreter
Use mecab-ipadic-neologd with igo-python
Use RTX 3090 with PyTorch
Use ansible with cygwin
Use pipdeptree with virtualenv
[Python] Use JSON with Python
Use Mock with pytest
Use indicator with pd.merge
Use Gentelella with django
Use mecab with Python3
Use tensorboard with Chainer
Use DynamoDB with Python
Use Python 3.8 with Anaconda
Use python with docker
Use TypeScript with django-compressor
Use LESS with Django
AWS CDK with Python
Use MySQL with Django
Use Enums with SQLAlchemy
Use tensorboard with NNabla
Use GPS with Edison
Use nim with Jupyter
[Mac OS] Use Kivy with PyCharm! [Python application development]
Use Trello API with python
Use "$ in" operator with mongo-go-driver
Use custom tags with PyYAML
Run python with PyCharm (Windows)
Use TensorFlow with Intellij IDEA
Use Twitter API with Python
Use pip with Jupyter Notebook
Try using matplotlib with PyCharm
Use DATE_FORMAT with SQLAlchemy filter
Use TUN / TAP with Python
Use sqlite3 with NAO (Pepper)
Use sqlite load_extensions with Pyramid
Use Windows 10 fonts with WSL
Use chainer with Jetson TK1
Use SSL with Celery + Redis
Use Cython with Jupyter Notebook
Debug with PEPPER python interpreter
Linux fastest learning with AWS
AWS Lambda with PyTorch [Lambda import]
Use Maxout + CNN with Pylearn2
Use django model from interpreter
Use OpenBLAS with numpy, scipy
Getting Started with Django with PyCharm
Use subsonic API with python3
Use Sonicwall NetExtener with Systemd
Use Python on Windows (PyCharm)
Use prefetch_related conveniently with Django
Use Bokeh with IPython Notebook
Use Python-like range with Rust
Use single quotes without escaping with the AWS Redshift UNLOAD command
Prepare an environment to use OpenCV and Pillow with AWS Lambda
Use pyright with CentOS7, emacs lsp-mode
Python: How to use async with
Use Azure SQL Database with SQLAlchemy
Use PointGrey camera with Python (PyCapture2)