Negative / Positive Analysis 1 Application of Text Analysis

Aidemy　2020/10/30

Introduction

Hello, it is Yope! I am a liberal arts student, but I was interested in the possibilities of AI, so I went to the AI-specialized school "Aidemy" to study. I would like to share the knowledge gained here with you, and I am summarizing it on Qiita. I am very happy that many people have read the previous summary article. Thank you! This is the first post in the negative-positive analysis. Nice to meet you.

• This article is a summary of what you learned in "Aidemy" "in your own words". It may contain mistakes and misunderstandings. Please note.

What to learn this time ・ What is negative / positive analysis? ・ Execution of negative / positive analysis using polar dictionary

What is Negative / Positive Analysis?

・ Negative-Positive Analysis is to judge whether a person's remarks or ideas are positive or negative. Actually, a method is taken in which expressions related to evaluations and emotions contained in sentences are extracted and emotions in the sentences are analyzed.

About polarity dictionary

-"Negative / positive" is called __ "polarity" __, and __polarity dictionary __ is a collection of polarities for each word __. The word is given a value from __ "-1 to + 1" __, and the larger the value, the more positive, and the smaller the value, the negative.

Morphological analysis

・ (Review) __ Morphological analysis __ is to divide a sentence into the smallest unit called a word __. By making it a word, it becomes possible to refer to the __polar dictionary __. Morphological analysis can be performed using MeCab etc. In addition, the following is also a lot of review content, but all of them will be done this time.

-Morphological analysis by MeCab can be done by __mecab.parse (file) __ after creating an instance with __MeCab.Tagger ('') __. __ One word per line __, so in order to separate each word, store it in "lines" for each line break with __split ('\ n') __. Also, the last two lines are 'EOS (EndOfSplit?)' And __ blank line __, so __ delete __. In addition, the created lines are divided into "__tab __ and __, __" for each word, and stored in "diclist" so that they become __ {'BaseForm': word} __.

・ Code

Refer to the word in the polarity dictionary and get the PN value

-A function that has the role of searching for __words in the wordlist returned from the morphological analysis function "get_diclist ()" created in the previous section in the polarity dictionary, and extracting the polarity values if they exist. _ Create "add_pnvalue ()" __. -When creating a function, first convert the __polar dictionary (pn_df) to a dictionary in which "word is key and PN value is value" __. List ['Word'] and ['PN'] of pn_df, respectively, and combine them into one dictionary as "pn_dict". -For add_pnvalue (), for each word whose key is'BaseForm'in each element of the passed word list, __ if there is that word in _pn_dict, add the key'PN' to the word list _ The PN value of the word is used as the value __, and it is stored in the empty list "diclist_new" and returned. By the way, if you want to know the PN value of the word in pn_dict, you can use "pn_dict [word]".

·code

Summary

・ Negative-Positive Analysis is to judge whether a person's remarks or ideas are positive or negative. Negative / positive analysis is performed using the __polar dictionary __. -In order to perform negative / positive analysis with the polarity dictionary, it is necessary to perform __morphological analysis of the sentence so that the polarity dictionary can be referred to.

This time is over. Thank you for reading until the end.