Technology that supports jupyter: traitlets (story of trying to decipher)

Python Advent Calendar 2019 Day 23 article.

Introduction

Do you know a library called traitlets? I was looking at the implementation of jupyter notebook a while ago and found out about its existence. It seems to be a library that was originally born and separated from the development of IPython. So, if you are using IPython or jupyter notebook, I am indebted to you, and I use it without knowing anything.

This is because the jupyter notebook and IPython config files are loaded using traitlets, for example by editing jupyter_notebook_config.py [^ 1] and ʻipython_config.py` [^ 2].

#Basic commented out
c.Application.log_datefmt = '%Y-%m-%d %H:%M:%S'

There may be people who have seen or edited the description like this.

[^ 1]: The generated one is in the directory pointed to by config: (basically .jupyter directly under the home), which is typed in jupyter --path. If you haven't set it yet, you can generate it with jupyter notebook --generate-config.

[^ 2]: If there is a profile directory (starting with profile) under the directory that appears in ʻipython locate, it is there. If you haven't created it, you can create it with ʻipython profile create (if you don't specify a name).

Actually, the mysterious c that appears here is an instance of the Config class of traitlets. And when you write c.Application.log_datefmt = ..., this is called log_datefmt of the Application class managed by the Configurable class that reads the configuration file in which this description is written (actually, the Application class that bundles them?). The member variable is assigned the value ....

In fact, the NotebookApp class (definition), which can be said to be the core class of jupyter notebook, is the JupyterApp class ([definition] of the jupyter_core module. Definition](https://github.com/jupyter/jupyter_core/blob/b129001bb88abf558df5dfab791a5aeeba79e25c/jupyter_core/application.py#L61), but this JupyterApp class inherits the Application class of traitlets ([here]](https://github.com/jupyter/jupyter_core/blob/b129001bb88abf558df5dfab791a5aeeba79e25c/jupyter_core/application.py#L30)).

I did a little research on what kind of library these traitlets are, but I couldn't understand them properly because the documentation was so rough that I wrote this article for the purpose of spreading the existence.

And the following is a translated + content diluted document of How to use.

How to use Traitlets

Since the so-called "type" is dynamically determined in python, it is possible to assign arbitrary values to class attributes (member variables) unless explicitly stated. I understand that one of the roles provided by traitlets is to properly type the attributes of this class so that more detailed check functions can be called easily. Actually, reading the configuration file that is also used in jupyter and ipython implementation seems to be the main function ...

Type check function

Define the HasTraits subclass Foo as follows:

from traitlets import HasTraits, Int

class Foo(HasTraits):
    bar = Int()

It gives a class called Foo an attribute called bar, just like a regular class. However, unlike regular class variables, this is a special attribute called ** trait **. In particular, this bar is a type of trait called int, which, as the name implies, stores an integer value.

Let's actually create an instance:

> foo = Foo(bar=3)
> print(foo.bar)
3
> foo.bar = 6
> print(foo.bar)
6

And foo has an attribute called bar of integer value "type", and it is possible to change the value.

On the other hand, what about giving a string?

> foo = Foo(bar="3")
TraitError: The 'bar' trait of a Foo instance must be an int, but a value of '3' <class 'str'> was specified.

You should get an error message stating that such a type assignment is incorrect. This eliminates the need to implement type checking yourself, such as with __setattr__.

Only Int type is introduced here, but some are prepared including container type such as List, and you can define it yourself. Please refer to the documentation for details.

Default value setting

The traitlet allows you to dynamically specify default values when instantiating. By the way, in the trait type ʻInt` above, 0 is set as the default value if nothing is specified:

> foo = Foo()
> print(foo.bar)
0

The following example stores today's date in a trait called today.

from traitlets import Tuple

class Foo(HasTraits):
    today = Tuple(Int(), Int(), Int())

    @default("today")
    def default_today(self):
        import datetime
        today_ = datetime.datetime.today()
        return (today_.year, today_.month, today_.day)

> foo = Foo()
> foo.today
(2019, 12, 22)

By the way, as you can see from the code, today's trait type is a tuple consisting of three integer values. Note that the default value of Tuple is(), so if you do not specify the default value or specify the value at the time of instantiation, the type will be different and an allocation error will occur.

I think this is probably equivalent to writing the following, but the former is clearly easier to read from the perspective of logic isolation:

class Foo(HasTraits):
    today = Tuple(Int(), Int(), Int())

    def __init__(self):
        import datetime
        today_ = datetime.datetime.today()
        self.today = (today_.year, today_.month, today_.day)

Value validation

Next, we will introduce the value assignment verification function. Even with type checking, I'm not sure if the value is correct. For example, ʻInt` alone is not enough when a trait (let's say it represents the number of something) is required to be a non-negative integer.

For that matter, this limitation may depend on another trait. For example, if you have a month that stores a month and a day that stores a day, the range of days allowed depends on the value of month. It is validate that performs such a check.

Here, I will implement it only in November and December.

from traitlets import validate

class Foo(HasTraits):
    today = Tuple(Int(), Int(), Int())

    @validate('today')
    def _valid_month_day(self, proposal):
        year, month, day = proposal['value']
        if month not in [11,12]:
            raise TraitError('invalid month')
        if month == 11 and day not in range(1,31):
            raise TraitError('invalid day')
        elif month == 12 and day not in range(1,32):
            raise TraitError('invalid day')
        return proposal['value']

> foo = Foo(today=(2000,12,1))
> foo.today
(2000, 12, 1)
> foo.today = (2000,13,1)
TraitError: invalid month
> foo.today = (2000,12,31)
> foo.today = (2000,12,32)
TraitError: invalid day

If multiple trait variables cross-reference, changing one value may cause a verification error on the way. In such cases, you should skip validation until all traits have changed. This can be achieved within the hold_trait_notifications scope. Let's look at the following example:

class Foo(HasTraits):
    a, b = Int(), Int()
    @validate('a')
    def _valid_a(self, proposal):
        if proposal['value'] * self.b <= 0:
            raise TraitError("invalid a")
        return proposal['value']
    @validate('b')
    def _valid_b(self, proposal):
        if proposal['value'] * self.a <= 0:
            raise TraitError("invalid b")
        return proposal['value']

> foo = Foo(a=1,b=1)
> foo.a = -1
> foo.b = -1
TraitError: invalid a
> with foo.hold_trait_notifications():
>     foo.a = -1
>     foo.b = -1
> print(foo.a, foo.b)
-1 -1

In this example, two traits a and b are defined, but their product is required to be non-negative. Then, even if both values are negative, this verification will pass, but if only one is changed, a verification error will occur. On the other hand, if you change the values of both a and b traits in hold_trait_notifications, delay verification will be performed at the end of this scope, so you don't have to worry about that.

Change notification

Finally, I would like to introduce the function to implement the observer pattern in the trait. This allows you to do something when the specified trait value is rewritten (event occurs).

class Foo(HasTraits):
    bar = Int()

    @observe('bar')
    def _observe_bar(self, change):
        ...

Is no longer complete code, but when the value of the trait bar changes, the function _observe_bar is executed.


As mentioned above, the content is insanely thin, but please forgive me because it is the first posting of a programming language system. Also, if you are familiar with traitlets, please enrich the lonely documentation and examples ...

Recommended Posts

Technology that supports jupyter: traitlets (story of trying to decipher)
The story of trying to reconnect the client
Story of trying to use tensorboard with pytorch
Technology that supports poop makers ~ Does state transition dream of poop?
Story of trying competitive programming 2
The story that the version of python 3.7.7 was not adapted to Heroku
A story that struggled to handle the Python package of PocketSphinx
A story that supports electronic scoring of exams with image recognition
Story of trying competitive programming Part 1
The story of trying to push SSH_AUTH_SOCK obsolete on screen with LD_PRELOAD
A story that suffered from OS differences when trying to implement a dissertation
Try to make a kernel of Jupyter
The story of adding MeCab to ubuntu 16.04
Technology that supports Python Descriptor edition #pyconjp
The story of trying deep3d and losing
Jupyter Notebook Basics of how to use
The story of pep8 changing to pycodestyle
The story of IPv6 address that I want to keep at a minimum
Library for "I want to do that" of data science on Jupyter Notebook
A story of a high school graduate technician trying to predict the survival of the Titanic