How to process camera images with Teams or Zoom

`This article is also introduced here. `` https://cloud.flect.co.jp/entry/2020/03/31/162537

Hello everyone.

At present, at the request of the Governor of Tokyo, our company works from home in principle to prevent the spread of new coronavirus infection. There are many people who are taking similar measures, and I think there are various difficulties, but I would like to work together to overcome this difficult song.

By the way, in principle, if you work from home for a long time, you may get stressed, such as not being able to have casual conversations that were held on a daily basis. In such a situation, I would like to create a situation where you can laugh and take a break, so I would like to introduce one small story.

The content is a method of hooking a webcam, processing it, and delivering it in a video conference such as Microsoft Teams or Zoom. Since I am a Linux user, this time I will introduce it on Linux. I think other platforms will be introduced somewhere.

In addition, I choose the time and the case to create a "situation where I can laugh and take a break", so please do so at your own risk (^ _ ^) /.

test.gif

Premise

I think it works fine on most Linux systems, but the environment I've worked in is Debian Buster.

$ cat /etc/debian_version
10.3

Also, if python3 is not included, please install it.

$ python3 --version
Python 3.7.3

Installation of related software

Virtual webcam device

This time we will use a virtual webcam device called v4l2loopback. https://github.com/umlaeute/v4l2loopback

We need to identify the virtual webcam device from the real webcam, so first check the actual webcam device file. In the example below, it seems that video0 and video1 are assigned to the actual webcam.

$ ls /dev/video*
/dev/video0  /dev/video1

Now let's introduce v4l2 loopback. First, git clone, make and install.

$ git clone https://github.com/umlaeute/v4l2loopback.git
$ cd v4l2loopback
$ make
$ sudo make install

Then load the module. At this time, it seems that it is necessary to add exclusive_caps = 1 in order to recognize it especially with chrome. [https://github.com/umlaeute/v4l2loopback/issues/78]

sudo modprobe v4l2loopback exclusive_caps=1

Now that the module is loaded, let's check the device file. In the example below, video2 has been added.

$ ls /dev/video*
/dev/video0  /dev/video1  /dev/video2

ffmpeg The easiest way to send data to a virtual webcam device is to use ffmpeg. Please install it quickly with apt-get etc.

Webcam hook and video distribution

This time, if a smile is detected, I will try to process the image. When a smile is detected, a smile mark is displayed on the video.

First, clone the files in the following repository to install the module.

$ git clone https://github.com/dannadori/WebCamHooker.git
$ cd WebCamHooker/
$ pip3 install -r requirements.txt

Get the cascade file from here. Please check the opencv official for details on the cascade file. https://github.com/opencv/opencv/tree/master/data/haarcascades

$ wget https://raw.githubusercontent.com/opencv/opencv/master/data/haarcascades/haarcascade_frontalface_default.xml -P models/
$ wget https://raw.githubusercontent.com/opencv/opencv/master/data/haarcascades/haarcascade_smile.xml -P models/

Let's borrow the smile mark from Mr. Toya.

$ wget https://4.bp.blogspot.com/-QeM2lPMumuo/UNQrby-TEPI/AAAAAAAAI7E/cZIpq3TTyas/s160/mark_face_laugh.png -P images/

It would be nice to have a folder structure like this.

$ ls -1
haarcascade_frontalface_default.xml
haarcascade_smile.xml
mark_face_laugh.png
webcamhooker.py

The execution is as follows. Enter the actual webcam device number in --input_video_num. For / dev / video0, enter the trailing 0. For --output_video_dev, specify the device file of the virtual webcam device. In addition, please use ctrl + c to end it.

$ python3 webcamhooker.py --input_video_num 1 --output_video_dev /dev/video2

When you execute the above command, ffmpeg will run and the video will start to be delivered to the virtual camera device.

Let's have a video chat!

When you have a video chat, you will see dummy ~ ~ in the list of video devices, so select it. This is an example of Teams. Think of the left and right as the screens of each participant. The user on the left is using this virtual camera device. The right side is the receiving side. When you smile, you will see a smile mark. Great success (^ _ ^) /.

test.gif

Finally

Now that face-to-face communication is difficult, I hope you can enjoy more using video chat. This time, I showed an example of detecting a smile and processing the image, but I think that various processing can be done by using opencv and other tools depending on the device. Please try various things!

reference

I referred to here for smile detection with opencv. https://qiita.com/fujino-fpu/items/99ce52950f4554fbc17d

Please refer to here for pasting images with opencv. https://qiita.com/a2kiti/items/9672fae8e90c2da6f352

Recommended Posts

How to process camera images with Teams or Zoom
How to process camera images with Teams and Zoom Sentiment analysis with Tensorflow
How to process camera images with Teams and Zoom Volume of processing in animation style
How to debug with Jupyter or iPython Notebook
How to display images continuously with matplotlib Note
How to update with SQLAlchemy?
How to cast with Theano
How to Alter with SQLAlchemy?
How to separate strings with','
How to RDP with Fedora31
How to Delete with SQLAlchemy?
How to make a Cisco Webex Teams BOT with Flask
How to kill a process instantly with Python's Process Pool Executor
How to not load images when using PhantomJS with Selenium
How to cancel RT with tweepy
How to enable Keras Regressor to be saved with pickle or joblib
Python: How to use async with
How to collect images in Python
How to use virtualenv with PowerShell
How to deal with imbalanced data
How to install python-pip with ubuntu20.04LTS
How to deal with imbalanced data
How to make a surveillance camera (Security Camera) with Opencv and Python
How to get started with Scrapy
How to get started with Python
How to deal with DistributionNotFound errors
How to get started with Django
How to Data Augmentation with PyTorch
How to use FTP with Python
Node.js: How to kill offspring of a process started with child_process.fork ()
How to calculate date with python
How to install mysql-connector with pip3
How to INNER JOIN with SQLAlchemy
How to install Anaconda with pyenv
The fastest way to get camera images regularly with python opencv
How to authenticate with Django Part 2
How to authenticate with Django Part 3
How to search using python's astroquery and get fits images with skyview
How to do arithmetic with Django template
[Blender] How to set shape_key with script
How to title multiple figures with matplotlib
How to get parent id with sqlalchemy
Learn to colorize monochrome images with Chainer
How to add a package with PyCharm
How to install DLIB with 2020 / CUDA enabled
How to use ManyToManyField with Django's Admin
How to use OpenVPN with Ubuntu 18.04.3 LTS
How to use Cmder with PyCharm (Windows)
How to prevent package updates with apt
How to work with BigQuery in Python
How to use Ass / Alembic with HtoA
Single pixel camera to experience with Python
How to deal with enum compatibility errors
How to use Japanese with NLTK plot
How to do portmanteau test with python
How to search Google Drive with Google Colaboratory
How to display python Japanese with lolipop
How to download youtube videos with youtube-dl
Add images to iOS photos with Pythonista
How to use jupyter notebook with ABCI
How to power off Linux with Ultra96-V2