Chapter 6 Supervised Learning: Classification pg212 ~ [Learn by moving with Python! New machine learning textbook]

Maximum likelihood estimation is w = 1/4.

** Maximum likelihood estimation elementary version finished! ** **

Logistic regression model (P216)

-By using a logistic regression model, the input x can be compressed in the range of 0 to 1, so it is easy to use as a classification probability.

By passing the sigmoid function through the linear model, the probability that x becomes t = 1 can be expressed as follows. ** Logistic regression ** $ y=w_0x+w_1\\ $ $ y=\sigma(w_0x+w_1)=\frac{1}{1+exp(-(w_0x+w_1))} $

(Average) Cross entropy error

・ Logistic regression is regarded as a probability (previous w value) ・ Therefore, from the input value x ・ I want to find a good weight w. Next, using the obtained w, enter the value of x in the created model to obtain the predicted value.

At this time, y represents the probability. Suddenly, a sigmoid function came out, but this is the one that puts the probability w that was handled so far in y. (By using the sigmoid function, y obtained from the input value x ($ y = w_0x + w_1 $)) can be expressed as a probability. I'm saying that.

⭐️ ** From the above, $ y = w = \ sigma (w_0x + w_1) = P (t = 1 | x) $. ** **

(important): On P218, maximum likelihood estimation is performed so that the parameters $ w_0 $ and $ w_1 $ of this model match the insect data. "Assuming that insect data was generated from this model, we will find the most probable (probabilistically high) parameters."

In the previous theory, we considered T = [0,0,0,1], but this time we will consider it so that it can be applied to all cases.

P218

If there is only one data and t = 1 for a certain body weight x, the probability that t = 1 will be generated from the model is the output value y of the logistic regression model itself.

There is a description, but what does this mean?

This is the image on the previous page. ** Looking at this, the logistic regression model can be treated as a probability because the linear model (this linear model is a linear expression that obtains the value y from the input value x) is crushed in the range of 1 to 0 by the sigmoid function. it can. ** **

So here y = (probability that t = 1) 1-y = (probability that t = 0) Treat as.

The model that generalizes this is the arrow $ P(t|x)=y^t(1-y)^{1-t} $

So now let's consider the case where the number of data is N Where I was thinking about T = [0,0,0,1] in the previous section X = [$ x_0, x_1, ..., x_n ] T = [ t_0, t_1, ..., t_n $] Consider the pattern (T = [0,0,0,1,1,1,0,1, ... N pieces]).

As we did in the previous section, when T = [0,0,0,1], we calculated something like [$ 0.8 * 0.8 * 0.8 * 0.2 = 0.104 $], but the generalized formula is also the maximum likelihood. Estimates should be calculated, so all patterns from $ x_0 to x_n $ should be multiplied.

This is what P219 says, "Since the probability of generating each piece of data can be multiplied by all the data, it becomes like equation (6-15). This is the likelihood."

This is the part of.

So equation (6-15) is as follows (because they are all multiplied) $ P(T|X)=\prod_{n=0}^{N-1}P(T|X)=\prod_{n=0}^{N-1}y_n^{t_n}(1-y_n)^{1-t_n} $ ** _, which represents the likelihood. (The Yamagata graph on page 214) _ ** </ font>