Bluemix Python Microservices Development
Python is a program language with a rich library in fields such as IoT, Analytics, Deep Learning, and natural language processing. Specialized processing in such fields can be developed as a microservice in Python by a specialized team, and can be called and used as a REST-API from a front-end application that is highly productive on the PHP web screen. High productivity and maintainability are expected.
In addition, Watson's API returns a response in JSON format, so by connecting Cloudant, which is a document type DB in JSON format, to the Node.js runtime, which is good at processing JSON, and building a corpus, Watson You can develop the application system of the above with high productivity. And, in the same way, you can add functions by using it as a REST service from the PHP front end.
As shown in the following figure, the structure that calls a microservice written in Node.js and Python from the front-end PHP application allows you to add functions without significantly affecting the web screen application. I will. In other words, by adopting such a microservices architecture, web screens are PHP, numerical calculations are Python, Watson is Node.js, etc., which are the programming languages that we are most good at, and mutual good productivity. You can expand the functions of the system while maintaining the above.
As a function to support such a microservice architecture, Bluemix has a function that allows you to bind and use user-provided REST services from other apps. By using this, it is possible to build a large-scale system that is loosely coupled in units of functions and organizations.
- This article is a concrete implementation example of Bluemix App Development and Management # 21.5 User-Provided Service Instances. *
Implementation details
In this article, I would like to create a Python RESTful web service that connects and uses from a PHP app. Of course, in this development process, HTTPS and authentication are considered assuming an environment where REST is called from the local development environment and the operation is confirmed. Also, by binding the Python service to the PHP app, you can get the access URL and authentication information, and avoid hard coding in the PHP app.
Creating a Python program environment
Build the stack of the development environment locally with Vagrant + VirtualBox. A Vagrantfile is prepared at the address of GitHub https://github.com/takara9/bluemix-dev, so you can use it to prepare a Python development environment on the virtual server of your PC. I can do it. You can also use this Vagrantfile to create a development environment for PHP, Ruby, and Node.js at the same time.
- Clone Vagrantfile from GitHub to your local environment
$ git clone https://github.com/takara9/bluemix-dev bluemix-rest
- Move the directory and start the virtual machine
$ cd bluemix-rest/
$ vagrant up
- Login with ssh
$ vagrant ssh
- Install the latest version of Python 2.7 series. You can also select 3.4 series with the pyenv command, but the REST server code in this article has not been tested with 3.4 series Python.
$ pyenv install 2.7.13
- Set to use installed Python
$ pyenv versions
* system (set by /home/vagrant/.pyenv/version)
2.7.13
$ pyenv global 2.7.13
$ python --version
Python 2.7.13
Now the Python development environment is ready.
Python REST server program development
We will write a minimal REST server for Python. The function of this code is as follows:
- Responds to GET / without authentication. Used to respond to health checks when deployed as a CF app on Bluemix.
- Receive POST with / calc. This / calc must pass BASIC authentication.
- The HTTP server port number is determined by referring to the environment variable PORT. The default is 5000 if the environment variable does not exist.
- POSTed data in / calc is for the form encoding application / x-www-form-urlencoded format.
- Multiply the numbers "a" and "b" in the POST data and respond in text JSON format {ans: result}
The following example code is available on GitHub https://github.com/takara9/REST_Sample_Python.
REST server Python sample code
1 #!/usr/bin/env python
2 # -*- coding:utf-8 -*-
3 import os
4 from flask import Flask
5 from flask_restful import Resource, Api, reqparse
6 from flask_httpauth import HTTPBasicAuth
7
8 # for Health Check
9 class HelloWorld(Resource):
10 def get(self):
11 return {'message': 'Hello World'}
12
13 # for POST
14 class Calc(Resource):
15 auth = HTTPBasicAuth()
16 @auth.login_required
17 def post(self):
18 args = parser.parse_args()
19 ans = float(args['a']) * float(args['b'])
20 return {'ans': ans }
21
22 @auth.verify_password
23 def verify_password(username, password):
24 return username == 'user' and password == 'pass'
25
26
27 if __name__ == '__main__':
28
29 app = Flask(__name__)
30 api = Api(app)
31 api.add_resource(Calc, '/calc')
32 api.add_resource(HelloWorld, '/')
33
34 parser = reqparse.RequestParser()
35 parser.add_argument('a')
36 parser.add_argument('b')
37
38 bx_port = os.getenv("PORT")
39 listen_port = int(bx_port if bx_port else 5000)
40 app.run(host='0.0.0.0', port=listen_port, debug=True)
Program description
Lines 1 and 2
The first line is a declaration that uses python as an interpreter. / usr / bin / env is the specification for executing Python set in pyenv. You can find the manual in man env. The second line is a declaration to use UTF-8.
Lines 3-6
I'm using Python's FLASK framework to create a REST server. You can find out more about the three modules you are importing at the following URL.
Lines 8-11
Code to respond to health checks when deployed to Bluemix. Since the content to be responded is not specified,'Hello World' is returned. The callback specification for this class is on line 32.
Lines 13-24
This is the core part of the REST service. Lines 15 and 16 specify that BASIC authentication is required. In lines 17 to 20, the data received by POST is converted (parsed) into a variable, multiplied, and returned. I think the Python framework that returns a response with a variable that returns from a callback is applauding.
Lines 22-24 match the user ID and password. If it is a real application, you will access the ID management system and DB from here.
After the 27th line
This is the main routine part. Instantiate the framework, set the callback class, specify the parameters to parse, get the port number from the environment variable, and start the server.
Installation and testing in a local environment
In this section, we'll write about how to test the code cloned from GitHub locally. You can deploy it to Bluemix and run it all at once, but it's convenient to have a local development environment, so check the operation on your Mac and then deploy it.
First, clone the code as follows:
git clone https://https://github.com/takara9/REST_Sample_Python
Navigate to the REST server directory where you'll find the list of required Python modules requirements.txt.
cd REST_Sample_Python/restServerPython
Run the following command to bulk install the Python prerequisite packages.
pip install -r requirements.txt
Run REST server (local)
Start the REST server in the following way: If you set it as chmod + x app.py in advance, you can also execute it directly as ./app.py.
vagrant@vagrant-ubuntu-trusty-64:~/REST_Sample_Python/restServerPython$ python app.py
* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
* Restarting with stat
* Debugger is active!
* Debugger PIN: 267-639-962
*
Run REST client (local)
Open another terminal, log in to the virtual server in your development environment with vagrant ssh, and then run the REST client. The REST client is also included in the code cloned from GitHub, so move the directory.
cd REST_Sample_Python/restClientPython
As shown in the figure above, the client is written in PHP because we are proceeding with calling a service written in Python from a PHP application. The client code is designed to get the URI and authentication information of the connection destination from vcap-local.json, so rewrite this file and connect to the REST service started locally.
The change is uri on the 7th line, rewrite it to "http: // localhost: 5000 / calc".
1 {
2 "VCAP_SERVICES": {
3 "user-provided": [
4 {
5 "credentials": {
6 "password": "pass",
7 "uri": "https://pycalcxx.mybluemix.net/calc", <-fix this line
8 "username": "user"
9 },
10 "label": "user-provided",
11 "name": "pycalcxxu",
12 "syslog_drain_url": "",
13 "tags": [],
14 "volume_mounts": []
15 }
16 ]
17 }
18 }
Next, install PHP with phpenv and check the version.
phpenv install 5.6.31
phpenv global 5.6.31
vagrant@vagrant-ubuntu-trusty-64:~/REST_Sample_Python/restClientPython$ php -v
PHP 5.6.31 (cli) (built: Aug 14 2017 05:28:21)
Now that you have an environment where you can run PHP code, run it.
vagrant@vagrant-ubuntu-trusty-64:~/REST_Sample_Python/restClientPython$ php client.php
Result = 2130.951794
It is a good idea to change the values of array variables a and b appropriately in the client code.
PHP REST client code description
I would like to present the entire code first to supplement the key points.
1 <?php
2 include "cfenv.php";
3
4 $ch = curl_init();
5 $vcap = new Cfenv();
6 $vcap->byInstName('pycalcxxu');
7
8 // POST
9 $form = array(
10 'a' => 391.345,
11 'b' => 5.4452
12 );
13
14 $options = array(
15 CURLOPT_POST => 1,
16 CURLOPT_HEADER => 0,
17 CURLOPT_URL => $vcap->uri,
18 CURLOPT_FRESH_CONNECT => 1,
19 CURLOPT_RETURNTRANSFER => 1,
20 CURLOPT_FORBID_REUSE => 1,
21 CURLOPT_TIMEOUT => 4,
22 CURLOPT_USERPWD => $vcap->user.":".$vcap->pass,
23 CURLOPT_POSTFIELDS => http_build_query($form)
24 );
25
26 curl_setopt_array($ch, $options);
27 $resp = curl_exec($ch);
28 $result = json_decode($resp);
29 print "Result = ".$result->{'ans'}."\n";
30
31 curl_close($ch);
32 ?>
2nd line cfenv.php
This module is a module that can get the connection destination information by environment editing when deployed to Bluemix, and can read the connection destination information from vcap-local.json when testing in the local environment.
Lines 8-12
Data for POST, the sample Python REST server returns the result of multiplying a and b.
Lines 14-24
Parameters required for POST, such as URI, basic authentication user and password, form data, etc. are set.
26th line ~
Execute HTTP POST to get the result and display it on the screen.
Deploying a REST server on Blemix
The files for deploying the Python execution module to Bluemix are the following four files.
vagrant@vagrant-ubuntu-trusty-64:~/REST_Sample_Python/restServerPython$ ls
app.py manifest.yml requirements.txt runtime.txt
- app.py: REST service Python program
- manifest.yml: Manifest for deploying to Bluemix
- requirements.txt: Packages required to run Python apps
- runtime.txt: Python version specification
1 ---
2 applications:
3 - buildpack: https://github.com/cloudfoundry/python-buildpack
4 name: python-rest-svc
5 memory: 64MB
6 disk_quota: 256MB
7 random-route: false
8 domain: mybluemix.net
9 name: pyCalcxx
10 command: python app.py
The point is the build pack on the third line, which specifies the latest build pack for Cloud Foundry. Then specify python app.py as the start command to start python on line 10.
Finally, run the following command to allow Bluemix to deploy the required modules at deployment time, such as when additional modules occur.
pip freeze > requirements.txt
A file that specifies the Python version. Please refer to the Cloud Foundry manual for details, as there is a range that can be used with the build pack.
python-2.7.13
Log in with the Blumiex CLI and deploy your app.
bx login
bx cf push
After successfully deploying, try changing the URI of the test client and testing. To find out the URL of the deployed REST service:
vagrant@vagrant-ubuntu-trusty-64:~/REST_Sample_Python/restServerPython$ bx cf a
Invoking 'cf a'...
name requested state instances memory disk urls
pyCalcxx started 1/1 64M 256M pycalcxx.mybluemix.net
In this case, I found pycalcxx.mybluemix.net, so I'll modify and test line 7 of the client's vcap-local.json.
7 "uri": "https://pycalcxx.mybluemix.net/calc",
If you get the same result as when you run it locally, the REST server deployment is complete.
vagrant@vagrant-ubuntu-trusty-64:~/REST_Sample_Python/restClientPython$ php client.php
Result = 2130.951794
Defined as a user-provided service
Finally, the user-provided service can be executed. We will set it as a service provided by the user.
Check the name and address of this Python REST service.
vagrant@vagrant-ubuntu-trusty-64:~/REST_Sample_Python/restClientPython$ bx cf a
Invoking 'cf a'...
name requested state instances memory disk urls
pyCalcxx started 1/1 64M 256M pycalcxx.mybluemix.net
In addition to the app name mentioned above, set the service name and specify the address for the -p credentials.
vagrant@vagrant-ubuntu-trusty-64:~/REST_Sample_Python/restClientPython$ bx cf cups pycalcxxu -p '{"username":"user", "password":"pass", "uri": "https://pycalcxx.mybluemix.net/calc"}'
This will provide the credentials that will be given to your app when you bind the service. And when viewed from the Bluemix console screen, the Bluemix service and the user-provided service will appear in the same line as shown below.
For example, you can connect from another app and use it as follows.
Of course, you can bind and unbind not only from the GUI screen of the web but also from the Bluemix CLI command.
bx cf bs phpSamplexx pycalcxxu
If you refer to the service credentials from the application side, you can see that it is provided to the application as follows.
bx cf env phpSamplexx
<Omitted>
System-Provided:
{
"VCAP_SERVICES": {
"user-provided": [
{
"credentials": {
"password": "pass",
"uri": "https://pycalcxx.mybluemix.net/calc",
"username": "user"
},
"label": "user-provided",
"name": "pycalcxxu",
"syslog_drain_url": "",
"tags": [],
"volume_mounts": []
}
]
}
}
Sample code to use user-provided REST service
I put the PHP code that works with this Python REST service on GitHub, so I think that you can deepen your understanding if you refer to it as well. https://github.com/takara9/php_sample_apl/tree/db2
This is a simple sample, but I think it can be applied to applications such as utilizing a machine learning model using Python.
Screen copy 1 Input screen
Screen copy 2 REST processing result display screen
Summary
This time, the story is limited to one stage, but in cases such as DevOps where development and production operation are parallel, the environment side can provide the connection destination information of the microservice to the application, so during the busy development , You can prevent accidents such as accidentally making a mistake in the connection destination and destroying the production data, so you can use it as a safe and secure DevOps environment.
And in production, if your microservices lack performance, you can easily increase the number of instances. Since the gorouter that handles load distribution is included in the routing of microservices, you can increase the number of instances and scale out by simply clicking + as shown in the figure.
You'll want a little more useful features. Next is Bluemix API Connect.
Reference material
(1) Flask Micro Framework http://flask.pocoo.org/ (2) Flask RESTful https://flask-restful.readthedocs.io/en/0.3.5/ (3) Flask-BasicAuth https://flask-basicauth.readthedocs.io/en/latest/ (4) Python Buildpack http://docs.cloudfoundry.org/buildpacks/python/index.html (5) PHP manual curl exec http://php.net/manual/ja/function.curl-exec.php (6) The key to deploying Flask apps using Python Buildpack on Cloud Foundry (Diego) http://qiita.com/yuta_h3/items/4798ec83a26391c5627f (7) Cloud Foundry Documentation Python Buildpack https://docs.cloudfoundry.org/buildpacks/python/index.html (8) Bluemix App Development and Management # 21.5 User-provided service instance http://qiita.com/MahoTakara/items/9d03414689fe1b2b9cd2
Recommended Posts