[Python] Seriously think about the M-1 winning method.

1. Contents

• The M-1 Grand Prix is exciting every year, but I enjoy watching it every year. It is also clear that it is an important grand prix that greatly influences the career after that for comedians.
• But I'm a little worried when I watch it every year. There are years when a butch-cut candidate wins, but there are also years when a combination wins by a small margin, and in this case, it may be felt that the judges' tastes are strong.
• Originally, there would be no absolute thing about the skill of comedians, and that's why I think that ** compatibility with judges is important **.
• If that happens, I wonder if there is a way to go to the judges and earn points efficiently according to the tastes of the judges.
• I'm not particularly familiar with comedy, and I only know about comedians who are often seen in variety shows. I'm sorry if I insisted that I was wrong in the detailed comedy theory.

2. This year's M-1 data

• I referred to Stable Wikipedia. (Https://ja.wikipedia.org/wiki/M-1%E3%82%B0%E3%83%A9%E3%83%B3%E3%83%97%E3%83%AA)

• ||kaminuma|matsumoto|reiji|tomizawa|shiraku|hanawa|kyojin| |:--|--:|--:|--:|--:|--:|--:|--:| |milkboy|98|97|96|97|97|99|97| |kamaitachi|95|95|94|93|95|95|93| |pekopa|96|94|92|94|91|94|93| |wagyu|92|92|93|91|96|96|92| |mitorizu|94|91|93|91|94|92|94| |karashi|94|90|93|90|89|90|93| |ozuwarudo|94|90|94|91|89|89|91| |suwe|92|89|91|90|92|91|92| |indians|94|88|92|90|87|89|92| |newyork|90|82|88|88|90|91|87|

• If you want to copy and use it, please see below.

data = pd.DataFrame([[98,97,96,97,97,99,97],
                     [95,95,94,93,95,95,93],
                     [96,94,92,94,91,94,93],
                     [92,92,93,91,96,96,92],
                     [94,91,93,91,94,92,94],
                     [94,90,93,90,89,90,93],
                     [94,90,94,91,89,89,91],
                     [92,89,91,90,92,91,92],
                     [94,88,92,90,87,89,92],
                     [90,82,88,88,90,91,87]],
                   columns=['kaminuma','matsumoto','reiji','tomizawa','shiraku','hanawa','kyojin'],
                   index=['milkboy','kamaitachi','pekopa','wagyu','mitorizu','karashi','ozuwarudo','suwe','indians','newyork'])

3. Data overview

Mean and standard deviation

data.describe()

• Looking at the average, Kaminuma's 93.9 is the highest and Matsumoto's 90.8 is the lowest, but this is not a very meaningful number. This is because if you get a high score from someone who gives you a high score, you will not be different from your rivals.
• The standard deviation ('std' in the table) is highest at 4.18 in Matsumoto and lowest at 2.12 in Reiji. It can be understood that while it is easy to make a difference to rivals if you are addicted to Matsumoto, it is difficult to make a difference even if you are addicted to Reiji.

Correlation coefficient

sns.heatmap(data.corr(), annot=True)
plt.show()

• It seems that Kaminuma and Shiraku have quite different tastes, and it can be seen that the correlation coefficient is quite low.

data.corr().sum(axis=0)

• Matsumoto and Tomizawa seem to have the same likes and dislikes as other judges. In other words, it seems that it is not the type that gives out its own perspective, only the evaluation that can be consistent with the evaluation of other judges.
• On the other hand, the correlation with other judges is not so high, probably because Shiraku and Hanawa score according to their own monosashi.

Pair plot

sns.pairplot(data)
plt.show()

• The content is generally consistent with the correlation coefficient numbers.
• For example, looking at the evaluation of Kaminuma at the top, if other judges give high evaluations, Kaminuma will also give a high score, but there is a lot of variation in the evaluation of Shiraku and Hanawa, and the preference is You can see that it will be quite different.

4. Model

strategy

• By now, it has been found that there are judges who may have their own perspectives, and that many judges tend to give similar evaluations to other judges. If it becomes *, the next question is ** which judge will like it most efficiently to earn points **.
• The first thing I thought about was normal linear regression. For example, regarding the evaluation of Kaminuma and Matsumoto,
  $ \ qquad Score_ {matsumoto} = \ beta Score_ {kaminuma} + \ epsilon_ {kaminuma}, \ quad \ epsilon_ {kaminuma} \ sim N (0, \ sigma_ {kaminuma) } ^ 2) If you go back to the form of $
  , it will be
  $ \ qquad \ Delta Score_ {matsumoto} = \ beta \ Delta Score_ {kaminuma} $
  , and Kaminuma's score will be one unit. By going up, I thought that it would be possible to model how much Matsumoto's score can be expected to rise accordingly.
• However, what I was interested in here is whether it is okay to assume that $ \ epsilon_ {person} $ are independent of each other when this is expanded to multiple judges. In other words, in estimating the score of Matsumoto and Reiji from the degree of addiction to Kaminuma, there is some scale that Matsumoto and Reiji emphasize, although Uenuma does not care, $ \ epsilon_ {matsumoto} $ and $ \ epsilon_ Is there a tendency for {reiji} $ to be numbers in the same direction?
• In order to capture this, we decided to analyze the variance of the residuals when performing linear regression.
  $ \ qquad Var (X + Y) = Var (X) + Var (Y) + 2 Cov (X, Y) $
  $ \ qquad Cov (X + Y, Z) = Cov (X, Z) ) + Cov (Y, Z) $
  
  $ \ qquad \ sum_ {person} Score_ {person} = \ sum_ {person} \ beta_ {person} Score_ {kaminuma} + \ epsilon_ {all}, \ quad \ epsilon_ {all} \ sim N (0, \ sigma_ {all} ^ 2) $
  $ \ qquad where \ quad \ sigma_ {all} ^ 2 = \ sum_ {person} Var (residual_ {person}) + \ sum_ {person_1} \ sum_ {person_2} Cov (residual_ {person_1}, residual_ {person_2}) $
  .

Implementation

• The code actually used for the analysis is as follows.

from sklearn.linear_model import LinearRegression
X = data.values
coefs = np.zeros((7,7))
V = np.zeros(7)
model = LinearRegression()
for i in range(X.shape[1]):
    residuals = np.zeros((6,10))
    coef = np.zeros(6)
    x = X[:,i].reshape(-1,1)
    y = np.delete(X, i, 1)
    for j in range(y.shape[1]):
        y_ = y[:,j].reshape(-1,1)
        model.fit(x, y_)
        coef[j] = model.coef_
        residuals[j,:] = (model.predict(x) - y_).flatten()
    coef = np.insert(coef, i, 1)
    coefs[i,:] = coef
    cov_mat = np.cov(residuals, bias=True)
    V[i] = cov_mat.sum()

result

• The sensitivities of other judges to each judge's score, $ \ beta_ {person} $, are as follows. The data is included in each line, and the sensitivity of other judges to Kaminuma's score is the first line.

coefs_df = pd.DataFrame(coefs, 
                        columns=['kaminuma','matsumoto','reiji','tomizawa','shiraku','hanawa','kyojin'],
                        index=['kaminuma','matsumoto','reiji','tomizawa','shiraku','hanawa','kyojin'])
sns.heatmap(coefs_df, annot=True)
plt.title('regression coefficient')
plt.show()

• The standard deviation of the residuals is as follows.

std_df = pd.DataFrame(pow(V,0.5),
                      columns=['residual std'],
                      index=['kaminuma','matsumoto','reiji','tomizawa','shiraku','hanawa','kyojin'])

5. Consideration

• When considering which judge's liking is most efficient, rather than thinking about what happens to the points from other judges when one judge gives an additional point, each judge Considering that the standard deviations of the scores were different, ** think about what happens to the scores from other judges when you receive an additional score for 1 standard deviation ($ 1 \ sigma $) **. I think it's better.

sensitivity_df = pd.DataFrame(data.std(axis=0)*coefs.sum(axis=1), columns=['sensitivity'])
sensitivity_df = pd.concat([sensitivity_df, std_df], axis=1)

• As far as you can see, if you aim for ** Matsumoto **, you can easily expect a score increase from other judges, and you can see that you can earn points most efficiently.
• On the other hand, the standard deviation of the residual is the smallest in Matsumoto and the largest in Shiraku. Even if you are addicted to it, other judges evaluate it from a different point of view and it is difficult to directly improve the overall score, but the score tends to fluctuate depending on whether you are addicted to another measure of other judges, and the residual It can be understood that the standard deviation of is large.
• From the above ** First of all, after raising the overall score to Matsumoto's favorite manzai, if you can afford it, we will try to differentiate from rivals by interweaving elements that will get caught in your will ** to win the M-1 It is thought that it will help to do so.

6. Summary

• As far as the first round of 2019 is concerned, it was suggested that the manzai that Matsumoto likes would be the key to winning the first round.
• I can't give a concrete point about what kind of manzai Matsumoto likes. It is the limit of comedy amateurs.
• Similar analysis for other years and finals is likely to yield interesting implications.
• Well, even if there is a situation where such a tricky strategy works, it will be limited to a very small difference. I think the best way to enjoy it is to look at it comfortably and laugh without thinking about difficult things.