The morphological analyzer included in the dictionary
decomposes Japanese!
https://mocobeta.github.io/janome/
$ python3 -V
Python 3.7.3
$ mkdir janome-sample
$ cd janome-sample
$ python3 -m venv venv
$ source ./venv/bin/activate
$ pip install janome
$ python3
Python 3.7.3 (default, Mar 6 2020, 22:34:30)
[Clang 11.0.3 (clang-1103.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from janome.tokenizer import Tokenizer
>>> t = Tokenizer()
>>> for token in t.tokenize(u'At the end of the song, Gauche rushed into the dressing room with a cello as quickly as the cat, without looking at everyone anymore.'):
... print(token)
...
Song noun,General,*,*,*,*,Song,Kyoku,Kyoku
Is a particle,Case particles,General,*,*,*,But,Moth,Moth
Ending verb,Independence,*,*,Five steps, La line,Uninflected word,end,Owar,Owar
And particles,Connection particle,*,*,*,*,When,To,To
Gauche noun,General,*,*,*,*,Gauche,*,*
Is a particle,Particle,*,*,*,*,Is,C,Wow
Another adverb,General,*,*,*,*,Already,Mou,Mo
Everyone noun,Pronoun,General,*,*,*,Everyone,Minna,Minna
Particles,Attributive,*,*,*,*,of,No,No
Noun,Non-independent,General,*,*,*,One,Hou,Ho
Etc. Particles,Particles,*,*,*,*,Such,Nad,Nad
Is a particle,Particle,*,*,*,*,Is,C,Wow
Verb,Independence,*,*,One step,Continuous form,to see,Mi,Mi
Also particles,Particle,*,*,*,*,Also,Mo,Mo
Verb,Independence,*,*,Sahen Suru,Nu connection,To do,Se,Se
Auxiliary verb,*,*,*,Special,Continuous connection,Nu,Zu,Zu
Just an adverb,General,*,*,*,*,Just,Choudo,Chode
That adnominal adjective,*,*,*,*,*,That,Sono,Sono
Cat noun,General,*,*,*,*,Cat,cat,cat
Particles,Attributive,*,*,*,*,of,No,No
Noun,Non-independent,Auxiliary verb stem,*,*,*,Yo,You,Yaw
Particles,Adverbization,*,*,*,*,To,D,D
Quick adjectives,Independence,*,*,Adjective, Auoudan,Continuous connection,Quick,Subayak,Subayak
Cyril noun,General,*,*,*,*,Cyril,Cyril,Cyril
Particles,Case particles,General,*,*,*,To,Wo,Wo
Motto verb,Independence,*,*,Five steps / Ta line,Continuous connection,Offal,Mot,Mot
Particles,Connection particle,*,*,*,*,hand,Te,Te
Dressing room noun,General,*,*,*,*,dressing room,Gakuya,Gakuya
To particle,Case particles,General,*,*,*,What,F,D
Noun,General,*,*,*,*,Liao,*,*
Ge noun,suffix,General,*,*,*,Ge,Ge,Ge
Including verb,Independence,*,*,Five steps, Ma line,Continuous form,Get in,Komi,Komi
Auxiliary verb,*,*,*,Special / mass,Continuous form,Masu,Mashi,Mashi
Auxiliary verb,*,*,*,Special,Uninflected word,Ta,Ta,Ta
>>>
It was good
Recommended Posts