Beginning with Python machine learning

I came to the Open Developers Conference in Kamata.

Let's create a live article while listening to the story.

** 2017/12/21 postscript ** Click here for presentation slides → https://speakerdeck.com/terapyon/python-ji-jie-xue-xi-kotohazime-at-odc

Theme "Beginning with Python Machine Learning"

Predict epidemics from the number of reported influenza in Chiba City and temperature

procedure

  1. Select data and get data ~ ~ ~ Search for material
  2. Data processing, data connection ~ ~ ~ Easy to use
  3. Visualization ~ ~ ~ Somehow something can be seen. What kind of algorithm should I use?
  4. Algorithm selection
  5. Machine learning model selection
  6. Machine learning model evaluation

Select data and get data

Received from Chiba City and the Japan Meteorological Agency.

Data processing

Visualization

Algorithm selection

Machine learning model selection

Easy if you come here Immediately

Machine learning model evaluation

Come on python

Use Jupyter notebook

[Shift]-[Enter] will execute

Convenient

You can publish it to github as it is

View data with Pandas

I can read the bad CSV and process it somehow

Make full use of the data frame function

If you index by date, you can also concatenate weekly data and daily data with a single command.

Serialization

This is the most time consuming

Visualization with matplotlib

It doesn't work if Japanese is included

You can easily get a histogram from pandas using matplotlib

To define what a "fashion" is

Visualize multivariable calculus at once with pandas The graph that goes straight up from the lower left to the upper right is correlation 1

Consider algorithm candidates

And

Split training data and test data using scikit-learn

Explanatory variable x

Objective variable y Popular 1 Not popular 0

Save test data to avoid overfitting. 80% for learning, 20% for testing Percentage depends on model and amount of data

Algorithm initialization

use scikit-learn

Prediction clf.predict (X_val)

Use a confusion matrix because it's almost a hit

True positive False positive True negative False negative

Cross-validation

Was the ratio of learning and testing OK? Repeat the division and recalculation

Grid search (parameter adjustment)

(By the way, time runs out

Recommended Posts

Beginning with Python machine learning
Machine learning with Python! Preparation
Machine learning with python (1) Overall classification
"Scraping & machine learning with Python" Learning memo
Learning Python with ChemTHEATER 03
"Object-oriented" learning with python
Learning Python with ChemTHEATER 05-1
Amplify images for machine learning with python
Machine learning with python (2) Simple regression analysis
[Shakyo] Encounter with Python for machine learning
Learning Python with ChemTHEATER 01
Build AI / machine learning environment with Python
Machine learning starting with Python Personal memorandum Part2
Machine learning starting with Python Personal memorandum Part1
[Python] Collect images with Icrawler for machine learning [1000 images]
I started machine learning with Python Data preprocessing
Build a Python machine learning environment with a container
Machine learning learned with Pokemon
Reinforcement learning starting with Python
Machine learning Minesweeper with PyTorch
Python Machine Learning Programming> Keywords
Python Iteration Learning with Cheminformatics
Try machine learning with Kaggle
Run a machine learning pipeline with Cloud Dataflow (Python)
Machine learning
python learning
Build a machine learning application development environment with Python
Summary of the basic flow of machine learning with Python
I tried machine learning with liblinear
Machine learning summary by Python beginners
Input / output with Python (Python learning memo ⑤)
Perceptron learning experiment learned with Python
Try machine learning with scikit-learn SVM
<For beginners> python library <For machine learning>
Python: Preprocessing in Machine Learning: Overview
Quantum-inspired machine learning with tensor networks
Get started with machine learning with SageMaker
I started machine learning with Python Clustering & Dimension Compression & Visualization
Create a python machine learning model relearning mechanism with mlflow
Machine learning environment settings based on Python 3 on Mac (coexistence with Python 2)
Feature Engineering for Machine Learning Beginning with Part 3 Google Colaboratory-Scaling
Python & Machine Learning Study Memo: Environment Preparation
[Examples of improving Python] Learning Python with Codecademy
FizzBuzz with Python3
Scraping with Python
[Python] Learning Note 1
Statistics with python
Notes on PyQ machine learning python grammar
Scraping with Python
Python with Go
python learning output
[Machine learning] Try running Spark MLlib with Python and make recommendations
Machine learning imbalanced data sklearn with k-NN
Use machine learning APIs A3RT from Python
I installed Python 3.5.1 to study machine learning
[python] Frequently used techniques in machine learning
Why Python is chosen for machine learning
Twilio with Python
A story about machine learning with Kyasuket
"Python Machine Learning Programming" Summary Note (Jupyter)
Play with 2016-Python