Tips for Python beginners to use Scikit-image examples for themselves 5 Incorporate into network apps

Tips for Python beginners to use the Scikit-image example for themselves Tips for Python beginners to use the Scikit-image example for themselves 2 Process multiple files Tips 3 for Python beginners to use Scikit-image examples for themselves Tips for Python beginners to use the Scikit-image example for themselves 4 Use the GUI Following, the example of Scikit-image is slightly modified and the content for Python beginners is added to play with it. I will write it.

This time, I will use Python's convenient network library. Python has enough libraries to write [HTTP Server](Base HTTP Server — HTTP server with basic functions) and [POP Client](poplib — POP3 protocol client) with only standard libraries. It has the feature that they can be used in common on Windows, Linux, and Mac. You can write network programs without having to worry about CPU endianness problems or character code problems.

Blog article Transfer webcam images using python socket

Has a server-side script and a client-side script. The camera image is acquired on the server side, and the image is received and displayed from the server on the client side.

Python standard library SocketServer — Framework for building network servers And OpenCV functions (imported by import cv2) can realize a network camera.

It's achieved without any OS dependency or CPU endianness issues.

This time, let's consider processing the image and distributing the image on the web server of this web example.

Let's process the image before encoding the image acquired by OpenCV on the server side.

[Normalized Cut] (http://scikit-image.org/docs/dev/auto_examples/segmentation/plot_ncut.html#example-segmentation-plot-ncut-py) Normalized Cut Let's change the input image with the example of.

The image with Normalized Cut loses detailed features from the original image, so Will it be a network camera with some privacy in mind?

Here, the following part is extracted as a function from the example of Normalized Cut of scikit-image.

python


from skimage import segmentation, color

def plotNcut(img):    
    labels1 = segmentation.slic(img, compactness=30, n_segments=200)
    out1 = color.label2rgb(labels1, img, kind='avg')
    return out1

Blog post Transfer webcam images using python socket In the server script in import cv2.cv as cv
I'm using, but here to use the numpy data format import cv2 As a result, we will use the cv2 interface as follows.


cap = cv2.VideoCapture(cameraid)
ret, frame = cap.read()
jpegstring = cv2.imencode('.jpeg', frame)[1].tostring()

As you can see, Python has a rich network library, so it is very easy to incorporate the results and incorporate image processing and image recognition using scikit-image and OpenCV. It can be easily run on programs written on Windows or on ARM-based Linux. It is also an advantage that you do not have to worry about the difference even if the CPU has different endianness.

.py:server.py


"""
 webcamera server
  for opencv 2.3
"""

import SocketServer
import cv2

from skimage import segmentation, color

def plotNcut(img):    
    labels1 = segmentation.slic(img, compactness=30, n_segments=200)
    out1 = color.label2rgb(labels1, img, kind='avg')
    return out1

class TCPHandler(SocketServer.BaseRequestHandler):
    capture = ''

    def handle(self):
        # self.request is the TCP socket connected to the client
        self.data = self.request.recv(1024).strip()
        print "%s connected:" % self.client_address[0]
        ret, frame = cap.read()
        frame = plotNcut(frame)

        jpegstring = cv2.imencode('.jpeg', frame)[1].tostring()
        print len(jpegstring)
        self.request.send(jpegstring)

if __name__ == "__main__":
    HOST, PORT = '127.0.0.1', 12345

    #init camera
    cameraid = 0
    cap = cv2.VideoCapture(cameraid)
    cap.set(3, 640)
    cap.set(4, 480)
    if not cap:
        print "Could not open camera"
        exit()

    server = SocketServer.TCPServer((HOST, PORT), TCPHandler)

    # Activate the server; this will keep running until you
    # interrupt the program with Ctrl-C
    server.capture = cap
    server.serve_forever()

.py:client.py


'''
  Image receiver
  for OpenCV 2.4 python interface
'''

import socket
import numpy
import cv2

def getImageFromServer(HOST, PORT):
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((HOST, PORT))
    sock.send('HELLO\n')
    recvlen = 100
    buffer = ''
    while recvlen > 0:
        receivedstr = sock.recv(1024*8)
        recvlen = len(receivedstr)
        buffer += receivedstr

    print '%d bytes received' %len(buffer)
    narray = numpy.fromstring(buffer, dtype='uint8')
    decimg = cv2.imdecode(narray, 1)
    sock.close()
    return decimg

if __name__ == "__main__":
    HOST, PORT = "127.0.0.1", 12345
    
    # Receive data from the server and shut down

    while 1:
        img = getImageFromServer(HOST, PORT)
        cv2.imshow('Capture', img)
        key = cv2.waitKey(100)
        if(int(key) > 27):
            break
        img = ''
    

The above script Transfer webcam image using python socket It is a slight modification with reference to. The client.py is almost the same. One of the attractions of Python is that it makes this kind of cooperation easy.


** Addition: Image processing server **

I created an example of processing only the image on the server. In this example, send the image taken by the client This is an example of returning the image processed by the server to the client.

When sending binary data, it is difficult to determine the end of the binary data, so when the character string "DONE" was sent and it was sent, the binary data had already reached the end. Reference: Socket Programming HOWTO

.py:server2.py


#pylint:disable=C0103
"""
image processing server
"""

import SocketServer
import cv2
import numpy as np

from skimage import segmentation, color

def Ncut(img):
    """
    Normalized Cut in scikit-image
    """
    labels1 = segmentation.slic(img, compactness=30, n_segments=200)
    out1 = color.label2rgb(labels1, img, kind='avg')
    return out1

class TCPHandler(SocketServer.BaseRequestHandler):
    capture = ''

    def handle(self):
        """
        Image processing server
        """
        # self.request is the TCP socket connected to the client

        recvlen = 100
        buff = ''
        while recvlen > 0:
            receivedstr = self.request.recv(1024*8)
            if receivedstr.find("DONE") > -1:
                break
            recvlen = len(receivedstr)
            buff += receivedstr

        print '%d bytes received' % len(buff)

        narray = np.fromstring(buff, dtype='uint8')
        frame = cv2.imdecode(narray, 1)
        print "decoded image", frame.shape
        frame = Ncut(frame)
        print "passed Ncut"
        jpegstring = cv2.imencode('.jpeg', frame)[1].tostring()
        print len(jpegstring)
        self.request.send(jpegstring)

if __name__ == "__main__":
    HOST, PORT = '127.0.0.1', 12345

    server = SocketServer.TCPServer((HOST, PORT), TCPHandler)

    # Activate the server; this will keep running until you
    # interrupt the program with Ctrl-C
    server.serve_forever()

.py:client2.py


#pylint:disable=C0103
'''
  Image client
'''

import socket
import numpy as np
import cv2

def getProcessedImageFromServer(HOST, PORT, frame):
    """
    send image and recieve proccessed image from the server
    """
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect((HOST, PORT))

    jpegstring = cv2.imencode('.jpeg', frame)[1].tostring()
    print len(jpegstring)
    sock.send(jpegstring)
    sock.send("DONE\n")

    recvlen = 100
    buff = ''
    while recvlen > 0:
        receivedstr = sock.recv(1024*8)
        recvlen = len(receivedstr)
        buff += receivedstr

    print '%d bytes received' %len(buff)
    narray = np.fromstring(buff, dtype='uint8')
    decimg = cv2.imdecode(narray, 1)
    sock.close()
    return decimg

if __name__ == "__main__":
    HOST, PORT = "127.0.0.1", 12345

    # Receive data from the server and shut down
    cameraid = 0
    cap = cv2.VideoCapture(cameraid)
    cap.set(3, 320)
    cap.set(4, 240)

    while 1:
        ret, frame = cap.read()
        img = getProcessedImageFromServer(HOST, PORT, frame)
        cv2.imshow('processed', img)
        key = cv2.waitKey(100)
        if int(key) > 27:
            break
        img = ''

** Note: When writing in Boost and C ++ **

If I were to write a network program in C ++ now, I would use Boost. [ip::tcp::socket] (http://www.boost.org/doc/libs/1_55_0/doc/html/boost_asio/reference/ip__tcp/socket.html) boostjp: Boost Japanese information site Qiita article Notes on opening Raw Sockets in Boost.Asio

** Old-fashioned Unix-derived C socket library: ** It's a library of the era that opened up the TCP / IP network, and I don't think it should be used to start writing anew. Since the network was created, unicode has appeared and standardized multithreading has appeared. The standard Unix / Liunx libraries cannot throw away their functions due to backwards compatibility.

** Windows-specific socket library: ** It's quite different from the Unix socket library, and porting the functions written there doesn't seem easy to port to Linux.

** Note: Significance of using socket communication **

If you use socket communication between multiple processes, Program implemented in C ++ <-> Program implemented in Python The whole function can be realized by the method. That way, you can link your Python program to C ++. You can concatenate functions without the need to link C ++ programs from Python so that they can be used. os.system (cmd) os — Miscellaneous operating system interfaces subprocess.Popen () subprocess — Subprocess Management

When using, it is a very high overhead procedure to start and terminate the process every time, but if you leave the process running and realize it using socket communication, the overhead will be There is none.

** Note: Web server using Python **

An example of a web server using Python is CherryPy. In "Practical Computer Vision", an example of a web server for image search using CherryPy is written.

** Note: In the case of C ++ **

In the case of C / C ++, it is very difficult to have the skills to write programs related to OpenCV and machine learning, while having the skills to write socket programs for server programs and client programs at the same time. The fact that the socket program on Windows and the socket on Linux are different by default also makes such a thing difficult.

Hint 6 Improve Python code

Recommended Posts

Tips for Python beginners to use Scikit-image examples for themselves 5 Incorporate into network apps
Tips for Python beginners to use Scikit-image examples for themselves 4 Use GUI
Tips for Python beginners to use Scikit-image examples for themselves 3 Write to a file
Tips for Python beginners to use the Scikit-image example for themselves
Tips for Python beginners to use the Scikit-image example for themselves 9 Use from C
Tips for Python beginners to use the Scikit-image example for themselves 6 Improve Python code
Tips for Python beginners to use the Scikit-image example for themselves 2 Handle multiple files
Tips for Python beginners to use the Scikit-image example for themselves 7 How to make a module
Tips for Python beginners to use the Scikit-image example for themselves 8 Processing time measurement and profiler
~ Tips for beginners to Python ③ ~
[For beginners] How to use say command in python!
Beginners use Python for web scraping (1)
Beginners use Python for web scraping (4) ―― 1
[Python] Organizing how to use for statements
How to use Pylint for PyQt5 apps
How to use "deque" for Python data
Memo # 4 for Python beginners to read "Detailed Python Grammar"
The fastest way for beginners to master Python
Python for super beginners Python for super beginners # Easy to get angry
How to use Template Engine for Network Engineer
Memo # 3 for Python beginners to read "Detailed Python Grammar"
Memo # 1 for Python beginners to read "Detailed Python Grammar"
~ Tips for Python beginners from Pythonista with love ① ~
How to use data analysis tools for beginners
Try to calculate RPN in Python (for beginners)
Memo # 2 for Python beginners to read "Detailed Python Grammar"
Memo # 7 for Python beginners to read "Detailed Python Grammar"
Introduction to Programming (Python) TA Tendency for beginners
Memo # 6 for Python beginners to read "Detailed Python Grammar"
How to make Python faster for beginners [numpy]
~ Tips for Python beginners from Pythonista with love ② ~
Memo # 5 for Python beginners to read "Detailed Python Grammar"
Beginners can use Python for web scraping (1) Improved version
How to convert Python # type for Python super beginners: str
[For beginners] How to study Python3 data analysis exam
Python # How to check type and type for super beginners
Beginners use Python for web scraping (4) --2 Scraping on Cloud Shell
python textbook for beginners
OpenCV for Python beginners
[python] How to use the library Matplotlib for drawing graphs
Tips for developing apps with Azure Cosmos DB in Python
[For beginners] How to use for statements on Linux (variables, etc.)
How to learn TensorFlow for liberal arts and Python beginners
Tips for coding short and easy to read in Python
How to use machine learning for work? 03_Python coding procedure
Use mitmproxy to force your app's API into your development environment
How to convert Python # type for Python super beginners: int, float
[For beginners] Script within 10 lines (4. Connection from python to sqlite3)
I didn't know how to use the [python] for statement
[Python] Introduction to graph creation using coronavirus data [For beginners]
Use a scripting language for a comfortable C ++ life-OpenCV-Port Python to C ++-