[DOCKER] Use Tabpy with Cloud Run (on GKE)

I've never touched Tabpy, so A memo when I tried running Tabpy on Cloud Run (on GKE) while also studying Cloud Run

What is Tabpy

A mechanism that enables Tableau to visualize the results of statistical processing and machine learning in Python by linking Tableau and Python. Install the Python module with pip on any PC or server, start the Tabpy service, and access Tabpy using Tableau's external service connection.

What is Cloud Run?

https://cloud.google.com/run/?hl=ja Fully managed service for running containers Charged according to request processing time It seems to be an auto scale with a nice feeling, so it requires less infrastructure than GKE, but there are various restrictions (time limit, port designation, etc.) Cloud Functions-like image that allows individuals to move anything

Difference from on GKE https://tomokazu-kozuma.com/the-difference-between-cloud-run-and-cloud-run-on-gke/

Why Cloud Run on GKE

I couldn't connect to Tabpy with Cloud Run. .. I'm not sure if the inconsistency between the SSL certificate automatically issued by Cloud Run and the SSL certificate used for SSL conversion of Tabpy is bad. .. .. (Certificates automatically issued by Cloud Run cannot be used)

Development environment

Premise to use Cloud Shell

Deploy Tabpy to Cloud Run

Dockerfile

Create a GKE cluster (sample)

Set up referring to the following https://cloud.google.com/run/docs/gke/setup

Create sample cluster


gcloud beta container clusters create <Cluster name> \
   --machine-type=g1-small \
   --preemptible \
   --num-nodes=3 \
   --disk-size=10 \
   --zone=us-central1-a \
   --addons=HorizontalPodAutoscaling,HttpLoadBalancing,Istio,CloudRun \
   --cluster-version=latest \
   --enable-stackdriver-kubernetes \
   --enable-ip-alias \
   --scopes cloud-platform

To enable Cloud Run for an already established GKE cluster, enable the blue circle items in the capture below in order from the top.

image.png

Creating a Dockerfile

Add libraries to install as needed Just install tabpy with pip install and start tabpy, it's basically OK

Dockerfile


from continuumio/miniconda3:latest

ARG config_dir=/tmp/
ADD file.conf $config_dir

WORKDIR $config_dir

RUN pip install --upgrade pip && \
        pip install numpy pandas scikit-learn scipy textblob nltk vaderSentiment && \
        pip install reverse_geocoder geopy && \
        pip install tabpy==1.0.0

RUN tabpy-user add -u <username> -p <password> -f pwd.txt

ENV PORT 8080
EXPOSE 8080

ENTRYPOINT tabpy --config=file.conf

file.conf


#Create this file in the same directory as the Dockerfile
[TabPy]
TABPY_PORT = 8080
TABPY_PWD_FILE = /tmp/pwd.txt

Creating a Docker image

Execute the following command in the directory where the Dockerfile is stored. An image called tabpy-sample is created.

python


docker image build -t asia.gcr.io/<Project ID>/tabpy-sample:latest .

PUSH of Docker image

Push to Container Registry

python


docker push asia.gcr.io/<Project ID>/tabpy-sample:latest

Deploy to Cloud Run

CloudRunOnGKE


gcloud beta run deploy tabpy-sample \
   --image asia.gcr.io/<Project ID>/tabpy-sample \
   --platform gke \
   --cluster <GKE cluster name> \
   --cluster-location <Location>

When the platform option is managed, it will be deployed to CloudRun

(reference)CloudRun


gcloud beta run deploy tabpy-sample \
   --image asia.gcr.io/<Project ID>/tabpy-sample \
   --platform managed \
   --region <region>

Custom domain mapping

https://cloud.google.com/run/docs/mapping-custom-domains?hl=ja

When deployed to Cloudrun on GKE, By default you can't access the service without specifying the host in the header via curl Since Tableau's external connection service cannot specify headers, it is necessary to map and access a custom domain.

Confirmation of domain ownership

python


gcloud domains verify [DOMAIN]

Map domain to service

python


gcloud beta run domain-mappings create --service [SERVICE] --domain [DOMAIN]

Get DNS record data

python


gcloud beta run domain-mappings describe --domain [DOMAIN]

Fixed IP Reservation (if using Cloudrun on GKE)

python


gcloud compute addresses create [IP-NAME] --addresses [EXTERNAL-IP] --region [REGION]

EXTERNAL-IP specifies the IP address of the A record acquired in the procedure for acquiring DNS record data.

Add a DNS record with your domain registrar

https://cloud.google.com/run/docs/mapping-custom-domains?hl=ja#dns_update

Addictive point

Dockerfile I made a Dockerfile with reference to the following (plagiarized), but it seems that it was a little old. .. https://github.com/erichannell/TabPy-docker Currently, pip install seems to complete the tabpy installation.

Modify Dockerfile by referring to the official Tabpy documentation https://github.com/tableau/TabPy/blob/master/docs/server-install.md#starting-tabpy

Port when deploying Cloudrun

Cloudrun must specify listen PORT to 8080 https://stackoverflow.com/questions/55662222/container-failed-to-start-failed-to-start-and-then-listen-on-the-port-defined-b https://cloud.google.com/run/docs/reference/container-contract?hl=ja#port

Tabpy has a default port of 9004, so you need to specify the port 8080 using the config file. https://github.com/tableau/TabPy/blob/master/docs/server-config.md

Access Tabpy on Cloud Run with Tableau

Connect from Tableau Desktop

Click Manage External Service Connections from the top help menu image.png

Server: Custom domain set Port: 80 Username: User set in Dockerfile Password: User set in Dockerfile

Connect from Tableau Server

Allow External Service Connections in Tableau Server

https://help.tableau.com/current/server-linux/ja-jp/config_r_tabpy.htm https://help.tableau.com/current/server-linux/ja-jp/cli_security_tsm.htm#tsm_security_vizql-extsvc-ssl-enable

Run the following on the Tableau Server command line

Enable external services on tableau server


tsm security vizql-extsvc-ssl enable --connection-type tabpy --extsvc-host <hostname> --extsvc-port <PORT> --extsvc-username <Tabpy username> --extsvc-password <Tabpy password>

tsm pending-changes apply

At the moment (January 2020), it seems that it cannot be used from Tableau Online. .. ..

orz https://help.tableau.com/current/pro/desktop/ja-jp/r_connection_manage.htm

Workbooks containing external service scripts cannot be published to Tableau Online.

Pages that I referred to in various ways

[Let's run Tabpy in a container using Docker! ](Http://lovedata.main.jp/2018/11/20/docker%E3%82%92%E4%BD%BF%E3%81%A3%E3%81%A6mac%E3%81%A7% E3% 82% 82tabpy% E3% 82% 92% E5% 8B% 95% E3% 81% 8B% E3% 81% 9D% E3% 81% 86% EF% BC% 81 /)

Think about Google Cloud Run ports

[Knative in Cloud Run on GKE](https://qiita.com/toshi0607/items/eeeabe81b1beac343b6b#cloud-run-on-gke%E3%81%AE%E3%83%87%E3%83%97% E3% 83% AD% E3% 82% A4)

Recommended Posts

Use Tabpy with Cloud Run (on GKE)
Run SwitchBot on Windows 10 with Bleak
Run XGBoost with Cloud Dataflow (Python)
Use Tensorflow 2.1.0 with Anaconda on Windows 10!
Run the IDCF cloud CLI with Docker
Run servo with Python on ESP32 (Windows)
Run Linux on ARM architecture with QEMU
Python> Run with run-time arguments> Use import argparse
Easily realize microservices with Cloud Run x Flask
Run Flask on CentOS with python3.4, Gunicorn + Nginx.
Until you use PhantomJS with Python on Heroku
Use Python / Django with Windows Azure Cloud Service!
Run the flask app on Cloud9 and Apache Httpd
Run LEDmatrix interactively with Raspberry Pi 3B + on Slackbot
Run Kali Linux on Windows with GUI (without VirtualBox)
Run a machine learning pipeline with Cloud Dataflow (Python)
Run Paints Chainer on CPU with official python on win10
Ubuntu 20.04 on raspberry pi 4 with OpenCV and use with python
Autoencoder with Chainer (Notes on how to use + trainer)
Use of Google Cloud Storage (GCS) with "GAE / Py"
Run Google Cloud Functions locally with Cloud Native Build packs
Beginners use Python for web scraping (4) --2 Scraping on Cloud Shell
Use mecab-ipadic-neologd with igo-python
Use RTX 3090 with PyTorch
Use ansible with cygwin
Use pipdeptree with virtualenv
Run Python with VBA
[Python] Use JSON with Python
Use Mock with pytest
Use indicator with pd.merge
Use Gentelella with django
Run Django on PythonAnywhere
Use mecab with Python3
Use tensorboard with Chainer
Use DynamoDB with Python
Run mysqlclient on Lambda
Use pip with MSYS2
Use pyvenv on Windows
Run Blender with python
Use Python 3.8 with Anaconda
Use pyright with Spacemacs
Use python with docker
Use TypeScript with django-compressor
Point Cloud with Pepper
Use LESS with Django
Run OpenMVG on Mac
Use Ansible on Windows
Use MySQL with Django
Use QuTiP on Windows
Cloud Run tutorial (python)
Use pip on Windows
Use Enums with SQLAlchemy
Use tensorboard with NNabla
Use GPS with Edison
Use nim with Jupyter
Run iperf with python
Easy to use Nifty Cloud API with botocore and python
Use Python 3 introduced with command line tools on macOS Catalina