I tried using Microsoft's Cognitive Services facial expression recognition API

I tried using Microsoft's Cognitive Services facial expression recognition API

___ Note: The usage of the API has changed a lot, so I'm rewriting the article now. ___

Blog promotion: http://jotarout.hatenablog.com/

Is your face a window of your heart?

By knowing a person's facial expression, it may be possible to estimate the person's physiological and internal conditions. Let's recognize the facial expressions of everyone with the help of computers.

Use Microsoft's Facial Expression Recognition API.

After registering from the site above and getting the API key, save it in api_key.txt in the same directory as the __ source code. __

This is the code. Have your API key and your favorite images in the same directory. For the time being, write a code like pulling a UI that displays 5 emotions and levels next to the facial expression.

ms_emotion.py


# Calling Emotion APIs is
# restricted to 20 transactions per minute
# and 30,000 transactions per month.
# 20 transactions per 60 seconds
# 1 transaction per 3 seconds

import httplib
import urllib
import base64
import os
import sys
import cv2
import numpy as np
import json
import math

def get_emotion(file_path, headers):
    try:
        conn = httplib.HTTPSConnection('api.projectoxford.ai')
        conn.request("POST", "/emotion/v1.0/recognize?",
                     open(file_path, 'rb'), headers)
        response = conn.getresponse()
        data = response.read()
        conn.close()
        return data
    except Exception as e:
        print("[Errno {0}] {1}".format(e.errno, e.strerror))
        print(e.message)

def display_expression(data,img):
    font = cv2.FONT_HERSHEY_PLAIN
    text = "test cyber"
    font_size = 1
    data = json.loads(data)
    for face in data:
        f_rec  =  face['faceRectangle']
        width  =  f_rec['width']
        height =  f_rec['height']
        left   =  f_rec['left']
        top    =  f_rec['top']
        f_rec  =  face['scores']
        f_rec = sorted(f_rec.items(), key=lambda x:x[1],reverse = True)
        cv2.rectangle(img,(left,top),(left+width,top+height),(130,130,130),2)
        cv2.rectangle(img,(left+width,top),(left+width+150,top+50),(130,130,130),-1)

        for i in range(0,5):
            val = round(f_rec[i][1],3)
            emo = f_rec[i][0]
            cv2.rectangle(img,(left+width,top+10*i),(left+width+int(val*150),top+10*(i+1)),(180,180,180),-1)
            cv2.putText(img, emo+" "+str(val),(left+width,top+10*(i+1)),font, font_size,(255,255,255),1)

if __name__ == '__main__':
    if len(sys.argv) != 2:
        print 'Usage: # python %s /path/to/image' % sys.argv[0]
        quit()

    with open('api_key.txt', 'r') as f:
        key = f.read().rstrip('\n')
    f.close()
    headers = {
        'Content-Type': 'application/octet-stream',
        'Ocp-Apim-Subscription-Key': key,
    }

    data = get_emotion(sys.argv[1], headers)
    img = cv2.imread(sys.argv[1],-1)

    display_expression(data,img)

    cv2.imshow('image',img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

It can be executed with python ms_emotion.py your_image.png.

20160517002308.png スクリーンショット 2016-05-25 21.06.06.png スクリーンショット 2016-05-25 21.08.03.png

Screenshot from 2016-08-21 02-58-00.png

I think the accuracy is quite good (I can't get a small face very much). There are about 10 different facial expressions, so it will be fun to play around with them.

It seems that it can also be used for video analysis! As long as the API allows ...

Recommended Posts

I tried using Microsoft's Cognitive Services facial expression recognition API
Facial expression recognition using Pepper's API
I tried face recognition using Face ++
I tried using the checkio API
I tried using docomo speech recognition API and Google Speech API in Java
I tried using Twitter api and Line api
I tried using YOUTUBE Data API V3
I tried using UnityCloudBuild API from Python
I tried using the BigQuery Storage API
I tried to verify the speaker identification by the Speaker Recognition API of Azure Cognitive Services with Python. # 1
I tried to verify the speaker identification by the Speaker Recognition API of Azure Cognitive Services with Python. # 2
I tried using Remote API on GAE / J
I tried using the Google Cloud Vision API
When I tried using Microsoft's Computer Vision API, I recognized the Galapagos sign "Stop"
I tried using parameterized
I tried using argparse
I tried using mimesis
I tried using anytree
I tried using aiomysql
I tried using Summpy
I tried using coturn
I tried using Pipenv
I tried using matplotlib
I tried using "Anvil".
I tried using Hubot
I tried using ESPCN
I tried using openpyxl
I tried using Ipython
I tried using PyCaret
I tried using cron
I tried using ngrok
I tried using face_recognition
I tried using Jupyter
I tried using PyCaret
I tried using Heapq
I tried using doctest
I tried using folium
I tried using jinja2
I tried using folium
I tried using time-window
I tried APN (remote notification) using Parse.com REST API
I tried using the API of the salmon data project
I tried to search videos using Youtube Data API (beginner)
Facial expression detection using Yolov5
[I tried using Pythonista 3] Introduction
Age recognition using Pepper's API
I tried using easydict (memo).
I tried using Random Forest
I tried face recognition of the laughter problem using Keras.
I tried using BigQuery ML
I tried using Amazon Glacier
I tried using git inspector
[Python] I tried using OpenPose
[Python] I tried collecting data using the API of wikipedia
I tried using magenta / TensorFlow
I tried handwriting recognition of runes with CNN using Keras
I tried using AWS Chalice
[For beginners] I tried using the Tensorflow Object Detection API
I tried using Slack emojinator
I tried to make a regular expression of "amount" using Python
I tried to make a regular expression of "time" using Python