Useful for everyday life !? Semi-automation of COSPA's strongest design of experiments with Python

<!-Title How to get a lot of information with a small number of experiments with Python Automatically create efficient experimental designs in Python Automatically create a good experiment plan for cospa in Python I experimented 30,000 times, but in fact 16 times was enough Shorten 30,000 experiments to 16 Useful for everyday life!?->

Overview

--I will explain (roughly) the "design of experiments" that greatly improves the efficiency of the number of experiments. ――Since the procedure of design of experiments is a little complicated, we will automate the procedure part with Python (this time's purpose).

What is design of experiments (how useful)

Design of experiments (design of experiments) is an application of statistics that aims to design efficient experimental methods and analyze the results appropriately. is there. From Wikipedia

Suppose you want to know which effect is the largest among various parameters (causes) in scientific experiments. Design of experiments can be used to significantly reduce the number of ** experiments required to investigate it.

I wrote it difficult, but in fact, this design of experiments is also useful in everyday life. ** **

Take cooking as an example. ** I want to make delicious curry **, as a parameter that determines the deliciousness of curry

--Simmering time --Flour ratio --Type of curry powder

Try two types at a time ** Find out which parameters affect how good it is **.

In the example of the above table, if you try these with all combinations,

2 * 2 * 2 = ** You have to eat curry 8 times. ** **

However, using design of experiments

Just eat ** 4 times ** and you'll see which factors affect how delicious it is.

• The graph below is an example of the results.
• In this example, you can see that the stew time is the most effective and the influence of the flour ratio is small.

The power of design of experiments

It's pretty amazing just to halve the number of times (8⇒4), The more parameters there are, the more powerful the design of experiments will be.

--If there are 7 parameters, the total number of combinations is ** 128 times ** ⇒ With design of experiments, ** 8 times is enough! ** ** --If there are 15 parameters, the total number of combinations is ** 32768 times ** ⇒ With design of experiments, ** 16 times is enough! !! ** **

Wouldn't it be great to be so efficient?

What kind of person is effective

As mentioned above, it should have a wide range of applications such as ** daily life, research life, working life, etc. **. Life becomes convenient by using it.

As an example of myself, if I knew it when I was a university student, I could greatly streamline my research, but I didn't know it at that time. The motivation for writing this article is "** I want to teach myself as a college student **".

And this design of experiments can also be used in ** free study of elementary school students during summer vacation **. The rationale is that ** only "average" is used when calculating the effect of parameters in this way **. (It may be necessary for an adult who understands to provide some support.)

So what? Why python?

Design of experiments is a very powerful efficiency tool, so I hope many people will use it. However, it is a little difficult to learn how to use it.

This is a personal idea, but such a technique is like a mathematical formula, The engineering idea is that I usually want to use it as a tool easily.

** Automate complex procedures with Python so that you can use it as a tool on a regular basis! ** ** That is the reason for this Python conversion.

I want to know how it works

As with mathematical formulas, I think you need to understand the content once. However, once I understand it, I don't think it is necessary to remember the contents every time I actually use it.

Please study by referring to the links below.

Introduction to Design of Experiments Part2

I would like you to understand the ** mechanism that can reduce the number of experiments by using the orthogonal array ** of the design of experiments.

Design of Experiments with Python --Design_of_Experiment (DoE) with Python in Jupyter Notebook

This time, I implemented the following with Python + Jupyter notebook. The Jupyter notebook file is uploaded to GitHub.

--Creating an experimental design using a two-level orthogonal array (create_expt_plan_2_levels.ipynb) --Analysis of the above experimental results (from visualization to analysis of variance) (analysis_expt_result_2_levels.ipynb)

Feature --Feature

--Not only factors but also interactions can be assigned and analyzed. --Interaction is the effect of a combination of factors.

• In the curry example, the explanation is omitted assuming that there is no interaction.

Languages and libraries-Dependency

<!-Write the language and version used, the required libraries and their versions For Python, it is good to prepare requirements.txt->

• Python : 3.6.5
• pandas : 0.23.0
• openpyxl : 2.5.3
• matplotlib : 2.2.2
• scipy : 1.1.0

Preparation --Setup

<!-Write the setup method. Describe the command to set up the hardware and software to be prepared-> The above Python related and Jupyter Notebook environment is required. Anaconda is a convenient way to install them all at once. Explanation of Anaconda and installation method (reference link) Articles referenced in Jupyter notebook

Usage --Usage

<!-How to use. Write as specifically as possible. Also write a sample->

Creating an experimental design using a two-level orthogonal array

<!-Write what this software is and what you can do In addition, a simple demo (use example) etc. is displayed as a screenshot or GIF animation->

• create_expt_plan_2_levels.ipynb -** Assign factors and interactions to the 2-level orthogonal array **. --Output the experiment plan after allocation to EXCEL.

● Example Suppose you want to find out the parameters that affect delicious curry. --Simmering time: 20/60 min --Flour ratio: 5/10% --Curry powder type: Yokosuka Navy / Vermont

Enter this as shown below.

create_expt_plan_2_levels.ipynb

#Factor name
factor_symbols_all = ["time", "wheat", "Curry powder"]

#Level of each factor
factors = [
    [20,60],
    [5,10],
    ["Navy","BMT"]
]

#Factor number of the interaction you want to see(o～)Specified by.
interactions = [
#     [0, 1],    # AxB
#     [1, 2]     # BxC
]

When executed, the following experimental design table and EXCEL will be output. Output file name: output_Expt_Plan_L * .xlsx

Analysis of experimental results (from visualization to analysis of variance)

<!-Write what this software is and what you can do In addition, a simple demo (use example) etc. is displayed as a screenshot or GIF animation->

• analysis_expt_result_2_levels.ipynb --Using the experimental design created last time ** Using the result data after the experiment **, ** Visualize, analyze and estimate the effects of factors and interactions **.

● Example Enter the result in the EXCEL output earlier and save it. ――Here, let's assume that the deliciousness of the curry is evaluated on a scale of 5 points. --Save the file with the results as "result_Expt_Plan_L4.xlsx".

• Example of entry of experimental results

Specify the file in code.

analysis_expt_result_2_levels.ipynb

path = 'result_Expt_Plan_L4.xlsx'

When executed, the individual effect of each parameter (called a factor) is ** visualized in a graph **.

You can also see ** Statistical analysis results (ANOVA table) **.

Because the error is pooled if it has a small effect from the result. In this case, you need to specify wheat.

analysis_expt_result_2_levels.ipynb

# ['time', 'wheat', 'Curry powder']
pool_list = [0,1,0] 

Then, the analysis of variance will be performed as shown below.

License-License

Released under the MIT license

References --References

Information and links such as reference information sources (sites / papers)

http://jasst.jp/archives/jasst05w/pdf/S4-1.pdf https://www.slideshare.net/hajimemizuyama/par-49440366

Future tasks

Currently, only the "Interaction Table" method is used to assign interactions. In order to improve reliability, "method using interaction table" and "method using component symbol" are used as allocation methods. I would like to do two ways and add a check.

Also, currently only 2 level system is supported.

――Support for 3 level system ――Correspondence to 4 levels and pseudo levels

I would like to implement these as well in the future.

Afterword

I had the impression that the design of experiments I studied recently could be used quite a bit. I wrote this article with the primary purpose of making it known to many people.

In some parts, I omitted the explanation (due to my lack of explanation) in order to prioritize clarity. If you come up with a good example in the future, I may add it.

I'm also a beginner with Python, so I think there are some rough and hard-to-see parts, I hope it helps someone.

Previously posted when compiling the results into a report ** PowerPoint document automatic creation ** may also be helpful. → Automatic report creation using Python (Qiita article)

Finally, promotion

The iOS memo app "Saku Memo!" has been released as a personal development. It is a memo / ToDo management application that features intuitive operations such as a simple screen and sorting by dragging and dropping. → Sakumemo! --Always a quick note, Todo, Sort (Apple App Store)