How to use machine learning for work? 02_Overview of AI development project

Introduction

I am writing a series on the theme of "How to use machine learning for work". In this second article, I will organize how the AI development project will proceed.

The chapters are as follows.

――What is the difference between AI development and machine learning? --Overview of AI development ――Where do you use machine learning in AI development? --Procedures for building a model

What is machine learning? If you feel that your understanding of is suspicious, please also read "Part 1: Understanding the purpose of machine learning".

What is the difference between AI development and machine learning?

As I introduced in Part 1: Understanding the purpose of machine learning, there is a comprehensive relationship of "AI> machine learning". Specifically, it will be positioned as ** using machine learning technology while advancing AI development **.

image03.png

(Source: https://rikei-danshi.work/entry/ai-chigai)

Overview of AI development

AI development is organized in the figure below, but we will proceed with the process of "planning-PoC-development-operation".

--Planning: Clarify what you want to do with AI --PoC: Is the planned AI technically feasible? Is the project profitable? Check --Development: Build a "model" that is the heart of AI and implement it in the system --Operation: Perform maintenance and operation so that the developed system operates stably.

image03.png

Where to use machine learning?

Now that you have a look at the overall process of AI development, let's think about where to use machine learning. The answer is to use machine learning in "PoC" and "development".

The "model" shown in the figure is AI itself, and machine learning skills are required to build a good model. For those who have forgotten what a model is, see Part 1: Understanding the Purpose of Machine Learning.

Steps to build a model

To master machine learning, you need three main skills. Specific programming will be organized in Part 3. First, let's understand only the big picture this time.

--Data preprocessing --Model learning --Model validation

Data preprocessing

The purpose of data preprocessing is ** to prepare the data so that the model can be trained **.

Specifically, we will carry out three things, but in essence, the main analysis is the ** "preparation" ** of machine learning. Articles written by people with practical experience are often written in the field of ** data science ** rather than ** machine learning **.

--EDA (Exploratory Data Analysis): Grasp the statistical overall trend of data --Processing of missing and outliers: removing obstructive data that causes noise, etc. --Feature extraction: Analyze where ** the law of data ** appears

Model learning

From here, machine learning technology will appear in earnest. As introduced in "Part 1: Understanding the purpose of machine learning" ** Finding the law from a large amount of prepared data ** Is the main purpose.

image02.png

The machine learning method provided for that purpose is the ** algorithm **. If you compare it to cooking, it will be easier to imagine if you think of ** "data = ingredients" ** and ** "algorithm = recipe" **.

If you cook with the appropriate recipe according to the ingredients, you will get a delicious dish. Machine learning is the same, and if you use an appropriate algorithm for data, you can create a model with good prediction accuracy (= good AI).

Model validation

After training the model, you need a ** performance check ** to see how accurate the model predicts.

There are two things to do in the verification of this model.

--Measurement of prediction accuracy: When I read the reference book, various difficult explanations (holdout method, cross-validation, etc.) are introduced. ** Just try it out and check if you can predict it correctly **. --Consider the continuous maintainability of the model: I will omit it because the content will be deep, but if you are interested, please read Re-learning the model.

at the end

In the next article, I will organize the concrete implementation procedure of preprocessing-learning-verification based on the programming language Python. Please look forward to it as it covers and organizes a series of programming methods and specific study methods.

Also, various information is posted on SNS, so if you feel that you can read the article, I would be grateful if you could follow Twitter account "Saku731".

~~ Also, at the end of the sentence, we are doing "** Team Development Experience Project **" for a limited time. ~~ ~~ If you are interested, please check [Application Sheet] for details. ~~ (Addition) The deadline has been closed because it is full. The next time is scheduled for March 2019, so if you would like to be informed, please fill in [Reservation Form].

Until the end Thank you for reading.

Recommended Posts

How to use machine learning for work? 02_Overview of AI development project
How to use machine learning for work? 01_ Understand the purpose of machine learning
How to use machine learning for work? 03_Python coding procedure
Introduction to Machine Learning: How Models Work
scikit-learn How to use summary (machine learning)
Create a dataset of images to use for learning
Made icrawler easier to use for machine learning data collection
How to increase the number of machine learning dataset images
Summary of how to use pandas.DataFrame.loc
Summary of how to use pyenv-virtualenv
Summary of how to use csvkit
How to collect machine learning data
Summary of recommended APIs for artificial intelligence, machine learning, and AI
How to use an external editor for Python development with Grasshopper
[Python] Summary of how to use pandas
An introduction to OpenCV for machine learning
How to calculate Use% of df command
[Python] Organizing how to use for statements
[Python2.7] Summary of how to use unittest
How to use Pylint for PyQt5 apps
Jupyter Notebook Basics of how to use
How to enjoy Coursera / Machine Learning (Week 10)
How to use "deque" for Python data
Basics of PyTorch (1) -How to use Tensor-
An introduction to Python for machine learning
Summary of how to use Python list
[Python2.7] Summary of how to use subprocess
Creating a development environment for machine learning
How to use fingerprint authentication for KDE
[Question] How to use plot_surface of python
An introduction to machine learning for bot developers
How to use folium (visualization of location information)
A simple example of how to use ArgumentParser
How to use MkDocs for the first time
[Python] How to use two types of type ()
How to keep track of work in Powershell
How to use Template Engine for Network Engineer
Not much mention of how to use Pickle
Summary of how to use MNIST in Python
How to use data analysis tools for beginners
Overview of machine learning techniques learned from scikit-learn
About the development contents of machine learning (Example)
How to Introduce IPython (Python2) to Mac OS X-Preparation for Introduction to Machine Learning Theory-
[For beginners] Introduction to vectorization in machine learning
Arrangement of self-mentioned things related to machine learning
The first step of machine learning ~ For those who want to implement with python ~
[Introduction to Azure for kaggle users] Comparison of how to start and use Azure Notebooks and Azure Notebooks VM
How to set the development environment for each project with VSCode + Python extension + Miniconda
[BigQuery] How to use BigQuery API for Python -Table creation-
[For beginners] How to use say command in python!
A memo of how to use AIST supercomputer ABCI
I tried to summarize how to use matplotlib of python
[Recommended tagging for machine learning # 2] Extension of scraping script
[Recommended tagging for machine learning # 2.5] Modification of scraping script
About data preprocessing of systems that use machine learning
How to use App Engine's local development server dev_appserver.py
How to use Python Kivy ① ~ Basics of Kv Language ~
Preparing to start "Python machine learning programming" (for macOS)
How to build a development environment for TensorFlow (1.0.0) (Mac)
Everything for beginners to be able to do machine learning
How to use xml.etree.ElementTree