■ [Google Colaboratory] Use morphological analysis (janome)

Introducing how to use the Japanese morphological analysis tool ** janome **. Janome is Python's morphological analysis engine. Japanese text can be divided into morphemes to determine part of speech and to be divided (divided into words).

  1. Install of "janome"

!pip install janome
  1. How to use

from janome.tokenizer import Tokenizer
s = Tokenizer() # <-Create an instance of tokenizer
t = 'There is a chicken in a chicken'
tt = 'There are two chickens in the yard'
for _ in s.tokenize(t):
  print(_)
for __ in s.tokenize(tt):
  print(__)

[Output] =================================================== Ni ----- Particles, case particles, general, *, *, *, ni, ni, ni Crocodile ----- Noun, General, *, *, *, *, Crocodile, Crocodile, Crocodile Crocodile ----- Noun, General, *, *, *, *, Crocodile, Crocodile, Crocodile Wa ----- Particles, final particles, *, *, *, *, wa, wa, wa Chicken ----- Noun, General, *, *, *, *, Chicken, Chicken, Chicken ----- Particles, case particles, general, *, *, *, ga, ga, ga Is ----- verb, independence, *, *, one step, uninflected word, is, il, il

Garden ----- Noun, General, *, *, *, *, Garden, Niwa, Niwa Ni ----- Particles, case particles, general, *, *, *, ni, ni, ni Is ----- particle, particle, *, *, *, *, ha, ha, wa Two ----- nouns, numbers, *, *, *, *, two, two, two Feather ----- Noun, suffix, classifier, *, *, *, feather, wa, wa Chicken ----- Noun, General, *, *, *, *, Chicken, Chicken, Chicken ----- Particles, case particles, general, *, *, *, ga, ga, ga Is ----- verb, independence, *, *, one step, uninflected word, is, il, il [end] ======================================================

Reference

Comparison of morphological analysis tools (NLP2018)

Recommended Posts

■ [Google Colaboratory] Use morphological analysis (janome)
■ [Google Colaboratory] Use morphological analysis (MeCab)
■ [Google Colaboratory] Preprocessing of Natural Language Processing & Morphological Analysis (janome)
Japanese morphological analysis using Janome
Use music21 on Google Colaboratory
How to use Google Colaboratory
Google colaboratory
Use The Metabolic Disassembler on Google Colaboratory
Use cartopy without bugs in Google Colaboratory
Use TPU and Keras with Google Colaboratory
Google Colaboratory 90-minute session disconnection countermeasures --- Use Python! ---
Morphological analysis tool installation (MeCab, Human ++, Janome, GiNZA)
Google Colaboratory setup summary
Is it Google Colaboratory?
How to use Spacy Japanese model in Google Colaboratory
I can't use the darknet command in Google Colaboratory!
Building an environment to use CaboCha with google colaboratory
Let's use python janome easily
[Note] WordCloud from morphological analysis
[Python] Morphological analysis with MeCab
Python: Japanese text: Morphological analysis
Try StyleGAN on Google Colaboratory
Study Python with Google Colaboratory
Natural language processing 1 Morphological analysis
Google form aggregate analysis tool
Japanese morphological analysis with Python
Try OpenCV with Google Colaboratory
Tool organization: Google Colaboratory (updated 2020.2.24)
[PowerShell] Morphological analysis with SudachiPy
Pandas 100 knocks on Google Colaboratory
How to use Google Colaboratory and usage example (PyTorch x DCGAN)