Negative / Positive Analysis 1 Application of Text Analysis

Aidemy 2020/10/30

Introduction

Hello, it is Yope! I am a liberal arts student, but I was interested in the possibilities of AI, so I went to the AI-specialized school "Aidemy" to study. I would like to share the knowledge gained here with you, and I am summarizing it on Qiita. I am very happy that many people have read the previous summary article. Thank you! This is the first post in the negative-positive analysis. Nice to meet you.

What to learn this time ・ What is negative / positive analysis? ・ Execution of negative / positive analysis using polar dictionary

What is Negative / Positive Analysis?

Negative-Positive Analysis is to judge whether a person's remarks or ideas are positive or negative. Actually, a method is taken in which expressions related to evaluations and emotions contained in sentences are extracted and emotions in the sentences are analyzed.

About polarity dictionary

-"Negative / positive" is called __ "polarity" __, and __polarity dictionary __ is a collection of polarities for each word __. The word is given a value from __ "-1 to + 1" __, and the larger the value, the more positive, and the smaller the value, the negative.

Morphological analysis

・ (Review) __ Morphological analysis __ is to divide a sentence into the smallest unit called a word __. By making it a word, it becomes possible to refer to the __polar dictionary __. Morphological analysis can be performed using MeCab etc. In addition, the following is also a lot of review content, but all of them will be done this time.

-Morphological analysis by MeCab can be done by __mecab.parse (file) __ after creating an instance with __MeCab.Tagger ('') __. __ One word per line __, so in order to separate each word, store it in "lines" for each line break with __split ('\ n') __. Also, the last two lines are 'EOS (EndOfSplit?)' And __ blank line __, so __ delete __. In addition, the created lines are divided into "__tab __ and __, __" for each word, and stored in "diclist" so that they become __ {'BaseForm': word} __.

・ Code![Screenshot 2020-10-26 22.01.37.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/698700/3ad8f2b5-00a1-f482- 5257-2dc44407ead5.png)

Refer to the word in the polarity dictionary and get the PN value

-A function that has the role of searching for __words in the wordlist returned from the morphological analysis function "get_diclist ()" created in the previous section in the polarity dictionary, and extracting the polarity values if they exist. _ Create "add_pnvalue ()" __. -When creating a function, first convert the __polar dictionary (pn_df) to a dictionary in which "word is key and PN value is value" __. List ['Word'] and ['PN'] of pn_df, respectively, and combine them into one dictionary as "pn_dict". -For add_pnvalue (), for each word whose key is'BaseForm'in each element of the passed word list, __ if there is that word in _pn_dict, add the key'PN' to the word list _ The PN value of the word is used as the value __, and it is stored in the empty list "diclist_new" and returned. By the way, if you want to know the PN value of the word in pn_dict, you can use "pn_dict [word]".

·code スクリーンショット 2020-10-26 23.38.38.png

Summary

Negative-Positive Analysis is to judge whether a person's remarks or ideas are positive or negative. Negative / positive analysis is performed using the __polar dictionary __. -In order to perform negative / positive analysis with the polarity dictionary, it is necessary to perform __morphological analysis of the sentence so that the polarity dictionary can be referred to.

This time is over. Thank you for reading until the end.

Recommended Posts

Negative / Positive Analysis 1 Application of Text Analysis
Python: Negative / Positive Analysis: Text Analysis Application
Negative / Positive Analysis 2 Twitter Negative / Positive Analysis (1)
Negative / Positive Analysis 3 Twitter Negative / Positive Analysis (2)
Scraping & Negative Positive Analysis of Bunshun Online Articles
Python: Negative / Positive Analysis: Twitter Negative / Positive Analysis Using RNN-Part 1
Creation of negative / positive classifier using BERT
Challenge principal component analysis of text data with Python
Basics of regression analysis
Application of Python 3 vars
Text analysis of pub rice on Yokohama City IR
Negative / positive judgment of sentences and visualization of grounds by Transformer
Negative / positive judgment of sentences by BERT and visualization of grounds
Static analysis of Python programs
Implementation of independent component analysis
Application of CNN2 image recognition
Is Vtuber Positive? Is it negative?
Text sentiment analysis with ML-Ask