It is difficult to install a green screen, so I cut out only the face and superimposed it on the background image

I detected my face from the webcam and tried to overlay only the face on the background image. If you pass the video as a webcam to the web conference application, you can easily appear in the web conference. treeman_example.png You can also be a tree sage who wanted to be (?) When you were little.

Note) There is no additional function for blindfolding. The blindfold was added to the screenshot later by hand for this article.

Task

Since the beginning of March, I have been at home for web conferencing. The room was so dirty that I managed to do it with no video, just audio and screen sharing. However, I feel that the web conferencing alone is breaking down the trust savings that I have had so far. I'm very happy to see someone who sometimes shows my face. I also want to show up. But the room is too small and dirty!

Usually you'll give up and clean up the room, or set up a green screen and make a video with chroma key composition. But I can't get rid of it. .. ..

The web conferencing system widely used in the company is WebEx. There is no background conversion function like Zoom, and I don't feel like developing it immediately like Teams. .. .. I use Zoom occasionally, but the machine power is not enough and a green screen is essential. But there is no place to install a green screen. .. ..

Delusion

By the way, Zoom cuts out not only the face but also the hands and upper body from the background. So is heavy processing necessary? ?? If you divide it only by face detection, you should be able to do it without so much load even on a weak PC! ??

Concept

Like this.

Webcam-> Face area detection with OpenCV-> Extract only face area and overlay it on background image-> Output as webcam image-> Used with WebEx and Zoom

Looking at this blog post, it seems that Face detection can be done immediately with OpenCV. .. I've done cutting and overlaying at the Python Imaging Library (Pillow). If it can be processed about 5 times per second, it should be enough video to be used in a web conference. Now, what should I do to output the created image as a webcam image? OpenCV does not seem to have such a function. I couldn't find a library that can be used on Mac or Windows from Python.

It's not very smart, but on MacOS, it seems that CamTwist can capture a part of the screen and let other software recognize it as a webcam and pass it on.

I tried it

I have uploaded the code to GitHub. The part I wrote is Python, but I use OpenCV and Qt. Since it is in GW, it has not been put into actual battle yet. I wonder if it will be the next web conference soon.

Recommended Posts

It is difficult to install a green screen, so I cut out only the face and superimposed it on the background image
I want to cut out only the face from a person image with Python and save it ~ Face detection and trimming with face_recognition ~
I tried to cut out a still image from the video
I made a function to crop the image of python openCV, so please use it.
I want to create a histogram and overlay the normal distribution curve on it. matplotlib edition
I tried to find out the difference between A + = B and A = A + B in Python, so make a note
How easy is it to synthesize a drug on the market?
I made a POST script to create an issue on Github and register it in the Project
I ran GhostScript with python, split the PDF into pages, and converted it to a JPEG image.
Start the webcam to take a still image and save it locally
Cut out A4 print in the image
I have a private Python package but I want to install pipenv on GitHub Actions and build a Docker Image
Use Pillow to make the image transparent and overlay only part of it
I failed to install django with pip, so a reminder of the solution
I made an image classification model and tried to move it on mobile
Define a division value in Django and easily reflect it on the screen
I set up TensowFlow and was addicted to it, so make a note
The background of the characters in the text image is overexposed to make it easier to read.
It was a life I wanted to OCR on AWS Lambda to locate the characters.
[Rails] v1.0 came out on google-cloud-vision of gem, so I tried to support it
Try to Normalize Cut the image with scikit-image (although it gets angry on the way)
How to find out which process is using the localhost port and stop it
I don't like to be frustrated with the release of Pokemon Go, so I made a script to detect the release and tweet it
It took me half a day to install Polyglot in the anaconda environment on my mac, so I'll leave a note
I tried to install scrapy on Anaconda and couldn't
I made a tool that makes it a little easier to create and install a public key.
When I tried to install PIL and matplotlib in a virtualenv environment, I was addicted to it.
POST the image selected on the website with multipart / form-data and save it to Amazon S3! !!
I tried to make a script that traces the tweets of a specific user on Twitter and saves the posted image at once