Sentiment analysis of tweets with deep learning

Introduction

This time, we performed Sentiment Analysis (or Sentiment Classification) of labeled tweet data. ** 2015/10/19 An additional experiment was conducted. ** ** ** 2015/12/19 The source code of SCNN has been released. hogefugabar / CharSCNN-theano ** ** 2015/12/27 Not only SCNN but also CharSCNN implementation has been released. hogefugabar / CharSCNN-theano **

algorithm

This time, I tried to use the algorithm called CharSCNN described in the paper Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts. For convenience, I used an algorithm called SCNN. This algorithm gives a sentence (Sentence) as input as a series of one-hot expressions of words. CharSCNN gives one-hot expressions to letters in addition to words. If my understanding is correct, SCNN seems to have an architecture close to the following.

From [UNITN: Training Deep Convolutional Neural Network for Twitter Sentiment Classification](http://alt.qcri.org/semeval2015/cdrom/pdf/SemEval079.pdf)

Implementation

There was satwantrana / CharSCNN on GitHub, so I tried to use it as it is, but I fixed it myself because various codes were strange. ** 2015/12/19 The source code has been released. hogefugabar / CharSCNN-theano Please refer to here. ** **

I implemented it like To Word Embeddings → Convolution → Max Pooling → Fully-Connected → Fully-Connected → Softmax. I also use Dropout, RMSprop, etc.

data set

I used 20,000 tweets of tweets_clean.txt in satwantrana / CharSCNN. Training data 18000 tweets and test data 2000 tweets. Each tweet is labeled 0/1 (negative / positive), so it is classified into 2 classes.

Experimental result

Graph 10 Seed average is taken, but moving average is taken. It felt like overfitting started around 2 epoch (180000 * 2 iteration) turns. The maximum classification accuracy is about 0.8. The original paper said that it went from 0.82 to 0.85, so I think that the difference in the data set and the difference in the parameters have an effect.

At the end

Converting the very first input to Word Embeddings It seems that the result will be better if you use the weight pre-learned with Word2Vec, so I would like to try that as well.

Additional experiment (updated on October 19, 2015)

Since Word2Vec was included in the Chainer sample, I used the result of pre-learning with Skip-gram with Negative-Sampling. Pre-learning with Chainer and turning the Theano program lol. Thank you cPickle.

Well, it was better to pre-learn the rise, but it is better not to pre-learn the final result. .. .. What if I turn it a little longer?

Recommended Posts

Sentiment analysis of tweets with deep learning
Try deep learning of genomics with Kipoi
Deep learning 1 Practice of deep learning
Sentiment analysis of corporate word-of-mouth data of career change meetings using deep learning
Try deep learning with TensorFlow
Deep learning image analysis starting with Kaggle and Keras
Deep running 2 Tuning of deep learning
Sentiment analysis with Python (word2vec)
Deep Kernel Learning with Pyro
Deep reinforcement learning 2 Implementation of reinforcement learning
Generate Pokemon with Deep Learning
Text sentiment analysis with ML-Ask
3. Natural language processing with Python 5-1. Concept of sentiment analysis [AFINN-111]
Try Deep Learning with FPGA-Select Cucumbers
Cat breed identification with deep learning
[Examples of improving Python] Learning Python with Codecademy
Try deep learning of genomics with Kipoi
Sentiment analysis of tweets with deep learning
Make ASCII art with deep learning
Solve three-dimensional PDEs with deep learning.
Check squat forms with deep learning
Categorize news articles with deep learning
Forecasting Snack Sales with Deep Learning
Make people smile with Deep Learning
Deep Learning
[Examples of improving Python] Learning Python with Codecademy
Classify anime faces with deep learning with Chainer
Othello-From the tic-tac-toe of "Implementation Deep Learning" (3)
Machine learning with python (2) Simple regression analysis
Meaning of deep learning models and parameters
Try Bitcoin Price Forecasting with Deep Learning
Try with Chainer Deep Q Learning --Launch
Visualize the effects of deep learning / regularization
Data analysis starting with python (data preprocessing-machine learning)
Learning record of reading "Deep Learning from scratch"
Get lots of your tweets with Tweepy
Othello-From the tic-tac-toe of "Implementation Deep Learning" (2)
Python practice data analysis Summary of learning that I hit about 10 with 100 knocks
See the behavior of drunkenness with reinforcement learning
Deep learning / error back propagation of sigmoid function
A memorandum of studying and implementing deep learning
Deep Learning Memorandum
[Learning memo] Deep Learning from scratch ~ Implementation of Dropout ~
Basic understanding of stereo depth estimation (Deep Learning)
Deep Learning from scratch The theory and implementation of deep learning learned with Python Chapter 3
99.78% accuracy with deep learning by recognizing handwritten hiragana
Motion analysis of figure skating with VideoPose 3D
Python Deep Learning
Parallel learning of deep learning by Keras and Kubernetes
Analysis of shared space usage by machine learning
Deep learning × Python
Sentiment analysis of large-scale tweet data by NLTK
Implementation of Deep Learning model for image recognition
Static analysis of Python code with GitLab CI
Deep learning learned by implementation (segmentation) ~ Implementation of SegNet ~
Key points of "Machine learning with Azure ML Studio"
A story about predicting exchange rates with Deep Learning
I tried Amazon Comprehend sentiment analysis with AWS CLI.
Count the number of parameters in the deep learning model
[OpenCV / Python] I tried image analysis of cells with OpenCV
Calculate the regression coefficient of simple regression analysis with python
Application of Deep Learning 2 made from scratch Spam filter
Challenge principal component analysis of text data with Python