I tried running BERT with Sakura VPS (without GPU)

Introduction

This article Explanation about BERT! Let's make a prediction using the Japanese model! If you follow the street, you can easily experience BERT on Google cola boratory.

I wondered if it could be used publicly on a cheap server instead of on Google colaboratory, so I tried running it on Sakura VPS1G (800 yen per month).

The point is that the GPU that can be used with Google colaboratory cannot be used with Sakura VPS, so it is necessary to install the CPU version of the library.

Installation environment

Sakura VPS Memory 1 GB, storage SSD 50 GB, CPU 2 cores (800 yen per month) Standard installation CentOS8 x86_64

Introduction

First of all, referring to the following article, I created an "anaconda" user and created an environment "ml_env" for machine learning. In the article, it was built with Centos 7.7, but I had no problems with CentOS 8. [Part 1] Build a Python machine learning development environment on CentOS (Anaconda installation) --Qiita

Continuing from the above article, we will activate the machine learning environment "ml_env" as an "anaconda" user and introduce BERT in this environment.

Installing the CPU version of Transformers

pip install transformers[tf-cpu]

• TensorFlow will be installed at the same time with this installation. Reference

Installing the CPU version of Torch

pip install torch==1.7.0+cpu torchvision==0.8.1+cpu torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

• When I try to install with Anaconda conda install pytorch, I get an error likeIllegal instruction (core dumped). It seems to be a GPU related error, so I installed the CPU version as above. This command is for "START LOCALLY" in https://pytorch.org/get-started/locally/ It is generated by selecting like. CUDA is None and there is no GUP.

You have now installed the required libraries.

BERT Japanese Pretrained model download

Next, [Kurobashi / Kawahara / Murawaki Laboratory](http://nlp.ist.i.kyoto-u.ac.jp/index.php?BERT%E6%97%A5%E6%] of Kyoto University 9C% AC% E8% AA% 9EPretrained% E3% 83% A2% E3% 83% 87% E3% 83% AB) is available for download, BERT Japanese Pretrained model.

Assuming you are currently in / home / anaconda / as an anaconda user, create and move the directory / home / anaconda / bert / japan_test to store your Japanese model.

mkdir bert
cd bert
mkdir japan_test
cd japan_test

Then download, unzip, and delete the download file.

curl -OL http://nlp.ist.i.kyoto-u.ac.jp/nl-resource/JapaneseBertPretrainedModel/Japanese_L-12_H-768_A-12_E-30_BPE.zip
unzip Japanese_L-12_H-768_A-12_E-30_BPE.zip
rm -rf Japanese_L-12_H-768_A-12_E-30_BPE.zip

The model is now installed at / home / anaconda / bert / japan_test / apanese_L-12_H-768_A-12_E-30_BPE.

Exit from the directory you created.

cd ../..

Place the Python program file bert_test.py here for the BERT test in/ home / anaconda /.

bert_test.py

from transformers import BertTokenizer, BertForMaskedLM, BertConfig
import torch
import numpy as np

config = BertConfig.from_json_file('/home/anaconda/bert/japan_test/Japanese_L-12_H-768_A-12_E-30_BPE/bert_config.json')
model = BertForMaskedLM.from_pretrained('/home/anaconda/bert/japan_test/Japanese_L-12_H-768_A-12_E-30_BPE/pytorch_model.bin', config=config)
bert_tokenizer = BertTokenizer('/home/anaconda/bert/japan_test/Japanese_L-12_H-768_A-12_E-30_BPE/vocab.txt',
 do_lower_case=False, do_basic_tokenize=False)

tokenized_text = ['[CLS]', 'Once upon a time', 'Urashima', 'That', 'Fisherman', 'But', 'Live', 'Was there', '。'
, 'is there', 'Day', '、', '[MASK]', 'But', 'beach', 'To', 'Walking', 'When you are', '[SEP]']
masked_index = 12

tokens=bert_tokenizer.convert_tokens_to_ids(tokenized_text)
tokens_tensor=torch.tensor([tokens])

model.eval()
with torch.no_grad():
  outputs = model(tokens_tensor)
  predictions = outputs[0]
_,predicted_indexes=torch.topk(predictions[0,masked_index],k=10)
predicted_tokens = bert_tokenizer.convert_ids_to_tokens(predicted_indexes.tolist())
print(predicted_tokens)

This program

['[CLS]', 'Once upon a time', 'Urashima', 'That', 'Fisherman', 'But', 'Live', 'Was there', '。', 'is there', 'Day', '、', '[MASK]', 'But', 'beach', 'To', 'Walking', 'When you are', '[SEP]']

Predict the words that go into the [MASK] of.

python bert_test.py

If you run it with

['Fisherman', '[UNK]', 'Man', 'people', 'Man', 'Youth', 'woman', '役Man', 'boy', 'fish']

That's the result. You have succeeded in running BERT.

Other

I wanted to use this BERT analysis result in a web application, so I set up a local web server in Flask and made the BERT model daemonized (resident) in Flask. Then I set up a web server with Apache and published it as an API. In other words, when I access the Apache API from the outside, the API program accesses the BERT on Flask and obtains the analysis result of the BERT. that is,

BERT Word Guess Demo

It will be. Please try it if you like.