Python operator overload precedence

Overview

――The other day, I had the opportunity to announce at PyCon JP 2016, so I announced it. --Presentation material "Metaprogramming Python" ――At that time, I was asked about the priority of operator Overload, but I couldn't answer it at once, so I summarized it briefly.

Question

-Which method takes precedence over object A, which defines ** __ add__ **, or object B, which defines ** __radd__ **?

class A:

  def __init__(self, value):
      self.value = value

  def __add__(self, other):
      self.value += other.value
      return self

class B:

  def __init__(self, value):
      self.value = value

  def __radd__(self, other):
      self.value += other.value
      return self

A(1) + B(2) #=>Of A__add__Does it work? B's__radd__Does it work?

Answer

--The ** ʻA.__ add__** on the left has priority and is executed. --Basically, **radd ** seems to be a method for fallback when ** add` ** is not implemented. ――I didn't understand this area properly, so I gave a wrong explanation. Excuse me.

A(1) + B(2) #=> A(3)

#=>For A class__add__Is implemented, so it works first

What does that mean?

In Document, ** __ rxxx__ ** system method explanation has the following explanation. ..

Calling these methods, the operand of the binomial arithmetic operation is reflected(Was replaced)Implement stuff.
These functions are only called if the operands on the left do not support the corresponding operations and the non-operators are of different types.
For example, y is__rsub__()If it is an instance of a class with a method
Expression x-When y is evaluated, x.__sub__(y)Y when returns NotImplemented.__rsub__(x)Is called.

# refs http://docs.python.jp/3/reference/datamodel.html#object.__ror__

It is also written as an annotation as follows.

Note The operand type on the right is a subclass of the operand type on the left,
If a reflection method is defined for this subclass method
This method is called before the non-reflective method of the operand on the left is called.
This behavior allows subclasses to override the parent's operation.

Let's check this in order.

Note

-Not Implemented is "a built-in type for returning when a comparison operation or binary operation is performed on an unsupported type"
- __add__Against__radd__A method like this is called a reflection method.

1. Execute __radd__ if __add__ is not implemented

A + B

This code runs in the following order:

1. 「A.__add__Is executed and "B" is added.
2.If "Not Implemented" is returned in 1, "B".__radd__To add "A"

If ** NotImplemented ** is not returned at the timings 1 and 2, respectively, the added result is returned and the process ends. ** TypeError ** is returned when ** NotImplemented ** is finally returned in the operation result.

TypeError: unsupported operand type(s) for +: 'A' and 'B'

Let's check this with the code.

class A:

  def __add__(self, other):
      print('1. A.__add__ return NotImplemented')
      return NotImplemented

class B:

  def __radd__(self, other):
      print('2. B.__radd__ return NotImplemented')
      return NotImplemented

A() + B()

When I run this code, it looks like this:

Execution result

1. A.__add__ return NotImplemented
2. B.__radd__ return NotImplemented
Traceback (most recent call last):
  File "oeprator_loading.py", line 32, in <module>
    A() + B()
TypeError: unsupported operand type(s) for +: 'A' and 'B'

Apparently it works in the order I wrote above. But where is the exception ** TypeError ** thrown? Both ** ʻA.__ add__** and **B.radd ** are just returning ** NotImplemented` **?

So let's look for the contents of the CPython code. Then I found the following code.

static PyObject *
binary_op(PyObject *v, PyObject *w, const int op_slot, const char *op_name)
{
    PyObject *result = binary_op1(v, w, op_slot);
    if (result == Py_NotImplemented) {
        Py_DECREF(result);
        return binop_type_error(v, w, op_name);
    }
    return result;
}
// refs  https://github.com/python/cpython/blob/3.6/Objects/abstract.c#L806

It seems that ** binop_type_error ** actually spits ** TypeError **, and the flow is as follows.

1. +Operator processing is performed
2. 「binary_In "op1"__add__ or  __radd__And finally receive "Not Implemented"
3.When you receive "Not Implemented", "binop"_type_"Error" throws "TypeError"

Now you can see the flow. Next, check the annotation part

2. If it is a subclass, __radd__ will be executed first

Note The operand type on the right is a subclass of the operand type on the left,
If a reflection method is defined for this subclass method
This method is called before the non-reflective method of the operand on the left is called.
This behavior allows subclasses to override the parent's operation.

Check if it really works like this. Consider the code below.

class A:

  def __add__(self, other):
      print('1. A.__add__ return 10')
      return 10

class B:

  def __radd__(self, other):
      print('2. B.__radd__ return NotImplemented')
      return NotImplemented

A() + B()

result

1. A.__add__ return 10

In this case ** ʻA.__ add__** only returns **return 10**, so **B.radd** will not be executed. Now let's inherit the **B ** class from the ** ʻA ** class.

class A:

  def __add__(self, other):
      print('1. A.__add__ return 10')
      return 10

class B(A): # <-Inherit A class

  def __radd__(self, other):
      print('2. B.__radd__ return NotImplemented')
      return NotImplemented

A() + B()

result

2. B.__radd__ return NotImplemented
1. A.__add__ return 10

Then ** B.__radd__ ** is executed first, and it is confirmed that" If the right side is a subclass on the left side of the ** operator, the reflection method is preferentially executed ** ". It's done.

Also, just because ** B.__radd__ ** is executed preferentially, if ** NotImplemented ** is returned, it will fall back to ** ʻA.__ add__` **. understood.

3. What is __iadd__ is?

Call these methods for cumulative arithmetic assignment(+=, -=, *=, @=, /=, //=, %=, **=, <<=, >>=, &=, ^=, |=)To implement.
These methods perform operations in-place(Change self)Try to do,
as a result(It doesn't have to be, but it can be self)Must be returned.
If a particular method is not defined, its cumulative arithmetic operation falls back to a regular method.
For example, x is__iadd__()If it is an instance of a class that has a method, x+=y is x= x.__iadd__(y)Is equivalent to
If not, x+x as well as y rating.__add__(y)And y.__radd__(x)Is considered.
In certain situations, cumulative assignment may result in unexpected errors
(Why is it added a_tuple[i] += [‘item’]Throws an exception?Please refer to)But,
This behavior is actually part of the behavior of the data model.

# refs http://docs.python.jp/3/reference/datamodel.html#object.__ior__

You can overload the processing of "** Cumulative Arithmetic Substitution **". In other words, you can customize the operation of "` ʻA + = B``".

Summary

--Operator Overload is executed in the order of "** 1. Operator method-> 2. Reflection method " - Reflection method ** is executed only when the operator method returns ** NotImplemented ** --When the operation result becomes ** NotImplemented , TypeError: unsupported operand type (s) for .. is sent. - There is a special method (__ixxx__) ** that can only overload ** cumulative arithmetic assignment **

I got some other questions, but I'll write them somewhere later.

End

reference

-3.3.7. Emulate numeric types