Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 1 Memo "Preliminary Knowledge Before Beginning Exercises"

Contents

This is a memo for myself as I read Introduction to Natural Language Processing Applications in 15 Steps.

Preparation

--Personal MacPC: MacOS Mojave version 10.14.6 --docker version: Version 19.03.2 for both Client and Server

Chapter overview

As described in the book, we will cover the construction of the learning environment and the knowledge (programming language and library) that are the prerequisites for the exercises.

Note

1 Introduction / Natural language processing and machine learning

No notes

2 Writing / development environment for this book

The writing environment is distributed as requirements.txt, and Dockerfile is also distributed.

The requirements.txt describes the type and version of the library as shown below, and you can install the library in bulk using pip.

requirements.txt


numpy==1.15.0
scipy=1.1.0
...
mecab-python3==0.996.2
neologdn==0.3.2
...
#Installation example with pip
$ pip install --no-cache-dir -r requirements.txt

When using Dockerfile, build and start Docker container from Dockerfile. (I run it here)

###Container build###
#Run in the location where the Dockerfile is located
$ ls | grep Dockerfile
Dockerfile

#Build with repository name: 15step, tag name: latest
$ docker build -t 15step:latest .

###Container startup###
#When the container is started by default, python is started
$ docker run -it 15step:latest
Python 3.6.5 (default, Jun 27 2018, 08:22:23)
>>>

#Stop is quit()
>>>quit()
$

#How to start python after starting bash
$ docker run -it 15step:latest /bin/bash
/usr/src/app# 
/usr/src/app# python
Python 3.6.5 (default, Jun 27 2018, 08:22:23)
>>>

3 Python basics for machine learning

--Allows you to use lists, tuples, and dictionaries properly --Tuples are immutable ――How to write ternary operators in English compared to C language - price = 1000 if age >= 20 else 800 --If age> = 20, price = 1000 is executed, otherwise price = 800 is executed. --for statement --For each list, for val in <list>: --If you want a counter variable, set enumerate to for i, val in enumerate (<list>): --If you want only the counter variable, set range to for i in range (5): --Functions, classes, modules --Load module ʻimport module as md --Call a function withmd.method --from module import method to load module functions --Call a function with method --Regular expressions can be executed faster if the regular expression pattern is compiled in advance.

4 Numerical library NumPy

--The array is numpy.array (<list>) --The dimension is <array> .shape --Slice (obtained by specifying multiple elements) is <array> [m: n] --Get elements m ~ n-1 --Optional if m = 0 or n = --Broadcasting --Automatically aligns dimensions when calculating vectors + scalars and arrays + vectors --The four arithmetic operations can be calculated with the <array 1> operator <array 2> --The inner product is numpy.dot (<array 1>, <array 2>) --If you want to get the maximum value --Getting the maximum value of an element is numpy.max (<array>) --Getting the index of the maximum value of an element is numpy.argmax (<array>) --Total, average, random number, initialization (initialize with 0, initialize with 1) --Sequence join --Vertical direction is numpy.vstack (<array 1>, <array 2>) --Horizontal direction is numpy.hstack (<array 1>, <array 2>)

5 Other major libraries used in this document

No notes

References

-Introduction to natural language processing application in 15 steps

Recommended Posts

Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 1 Memo "Preliminary Knowledge Before Beginning Exercises"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 2 Step 06 Memo "Identifier"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 2 Step 02 Memo "Pre-processing"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 2 Step 07 Memo "Evaluation"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 4 Step 14 Memo "Hyperparameter Search"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 2 Step 04 Memo "Feature Extraction"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 4 Step 15 Memo "Data Collection"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 3 Step 08 Memo "Introduction to Neural Networks"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 2 Step 05 Memo "Features Conversion"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 3 Step 11 Memo "Word Embeddings"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 3 Step 12 Memo "Convolutional Neural Networks"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 3 Step 13 Memo "Recurrent Neural Networks"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 3 Step 09 Memo "Identifier by Neural Network"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 2 Step 01 Memo "Let's Make a Dialogue Agent"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 2 Step 03 Memo "Morphological Analysis and Word Separation"
Try the book "Introduction to Natural Language Processing Application Development in 15 Steps" --Chapter 3 Step 10 Memo "Details and Improvements of Neural Networks"
Summary from the beginning to Chapter 1 of the introduction to design patterns learned in the Java language
[WIP] Pre-processing memo in natural language processing
[Chapter 5] Introduction to Python with 100 knocks of language processing
[Chapter 6] Introduction to scikit-learn with 100 knocks of language processing
[Chapter 3] Introduction to Python with 100 knocks of language processing
[Chapter 2] Introduction to Python with 100 knocks of language processing
[Chapter 4] Introduction to Python with 100 knocks of language processing
[Job change meeting] Try to classify companies by processing word-of-mouth in natural language with word2vec
[Natural language processing] I tried to visualize the hot topics this week in the Slack community
[Natural language processing] I tried to visualize the remarks of each member in the Slack community
Introduction to Statistics The University of Tokyo Press Chapter 2 Exercises
[Python] Try to classify ramen shops by natural language processing