The story that Kivy's Japanese input characters are displayed

In July 2018, the latest version of Kivy (1.10.1) was released. Here is the IME display The content pull request has been adopted, and IME is operating and the Japanese being input is displayed.

Contents

Although it is a GUI library "Kivy" of Python, there is a problem that Japanese (IME) is not displayed while inputting Japanese in textinput of windows / MacOS environment. This time, we investigated the reason why it was not displayed and took measures to make it visible. The verification OS is windows.

Why Japanese is not displayed

First of all, Kivy, as shown below, the low layer realizes various functions by libraries such as OpenGL.

architecture.png

Among them, character input is realized using SDL.

Realization of text input in SDL

The source is excerpted from Tutorial of SDL text input.


    SDL_StartTextInput();
    while (!done) {
        SDL_Event event;

        if (SDL_PollEvent(&event)) {
            switch (event.type) {
                case SDL_QUIT:
                    /*End*/
                    done = SDL_TRUE;
                    break;
                case SDL_TEXTINPUT:
                    /*Add new text to the end of the text*/
                    strcat(text, event.text.text);
                    break;
                case SDL_TEXTEDITING:
                    /*
Update unconverted text.
Update the cursor position.
Convert the length of selection(if possible).
                    */
                    composition = event.edit.text;
                    cursor = event.edit.start;
                    selection_len = event.edit.length;
                    break;
            }
        }
        Redraw();
    }

}

If you look at this, you can see that the event "SDL_TEXTINPUT" is issued when entering characters, and the event "SDL_TEXTEDITING" is issued when operating IME.

About receiving events from SDL inside Kivy

With this in mind, let's take a look at kivy's github source code. It is core / window / _window_sdl2.pyx that receives the event from SDL and processes it. The "poll" function of pyx). The poll function receives the event from SDL and issues the event with the _event_loop function in window_sdl2.py. Looking at the code of the "poll" function, it looks like this:

    def poll(self):
        cdef SDL_Event event
        cdef int rv

        with nogil:
            rv = SDL_PollEvent(&event)
        if rv == 0:
            return False

        action = None
        if event.type == SDL_QUIT:
            return ('quit', )
        elif event.type == SDL_DROPFILE:
            return ('dropfile', event.drop.file)
        elif event.type == SDL_MOUSEMOTION:
            x = event.motion.x
            y = event.motion.y
            return ('mousemotion', x, y)
            
~ Omitted ~

        elif event.type == SDL_KEYDOWN or event.type == SDL_KEYUP:
            action = 'keydown' if event.type == SDL_KEYDOWN else 'keyup'
            mod = event.key.keysym.mod
            scancode = event.key.keysym.scancode
            key = event.key.keysym.sym
            return (action, mod, key, scancode, None)
        elif event.type == SDL_TEXTINPUT: ★
            s = event.text.text.decode('utf-8')
            return ('textinput', s)
        else:
            #    print('receive unknown sdl event', event.type)
            pass

As you can see, ** The character input and the "textinput" event are returned in response to the event "SDL_TEXTINPUT" when the character input is confirmed, but it is issued from SDL during input (IME). There is no processing to be performed in response to the "SDL_TEXTEDIT" event. ** ** This is the reason why characters are not displayed while inputting Japanese.

Make it possible to display the characters being entered in Japan

Now that I know the reason, I'll think about what to do next. In terms of processing, I think that the following procedure should be added first.

About the actual source

@souan added pull request to kivy itself. Also, as a sample program, it is placed on github.

Example of issuing a textedit event: https://github.com/Adachinski/kivy/tree/textedit_patch_smple Example of use: https://github.com/Adachinski/kivy_textedit_sample

The following is about the actual source.

Addition of "SDL_TEXTEDIT" event

Source: https://github.com/kivy/kivy/pull/5109

Added the process when SDL_TEXTEDIT is received to the poll function of the kivy / core / window / _window_sdl2.pyx file.

    def poll(self):
        cdef SDL_Event event
        cdef int rv

        with nogil:
            rv = SDL_PollEvent(&event)
        if rv == 0:
            return False


~ Omitted ~

        elif event.type == SDL_TEXTINPUT: 
            s = event.text.text.decode('utf-8')
            return ('textinput', s)
            
        elif event.type == SDL_TEXTEDITING:★ Add event
            s = event.edit.text.decode('utf-8')★ Enter the characters of IME
            return ('textedit', s)
        else:
            #    print('receive unknown sdl event', event.type)
            pass

By the way, this file is a pyx file, so you need to build it with cython and recreate the library. Since kivy is OpenGL + SDL, it was difficult to build the environment and I was building the build environment on windows, but it took 2 to 3 days. Details on building the environment will be posted at a later date.

Postscript: I made an article about how to build an environment for Japanese input.

Add "textedit" event

Source: https://github.com/Adachinski/kivy/tree/textedit_patch_smple

To do this, modify window_sdl2.py. https://github.com/Adachinski/kivy/commits/textedit_patch_smple/kivy/core/window/window_sdl2.py

The correction points are as follows.

class WindowSDL(WindowBase):

    __events__ = ('on_textedit',)★ Addition
 
~ Omitted ~
        def _mainloop(self):
        EventLoop.idle()

        # for android/iOS, we don't want to have any event nor executing our
        # main loop while the pause is going on. This loop wait any event (not
        # handled by the event filter), and remove them from the queue.
        # Nothing happen during the pause on iOS, except gyroscope value sent
        # over joystick. So it's safe.
        while self._pause_loop:
            self._win.wait_event()
            if not self._pause_loop:
                break
            self._win.poll()

        while True:
            event = self._win.poll()
            if event is False:
                break
            if event is None:
                continue

            action, args = event[0], event[1:]
            if action == 'quit':
                if self.dispatch('on_request_close'):
                    continue
                EventLoop.quit = True
                self.close()
                break
~ Omitted ~


            elif action == 'textinput':
                text = args[0]
                self.dispatch('on_textinput', text)
            elif action == 'textedit':★ Added textedit
                text = args[0]
                self.dispatch('on_textedit', text)

            # unhandled event !
            else:
                Logger.trace('WindowSDL: Unhandled event %s' % str(event))
    

~ Omitted ~
    def on_textedit(self, text):★ Addition
        pass

Actual usage example using "textedit" event

Source: https://github.com/Adachinski/kivy_textedit_sample

Inherit the "TextInput" class in TextInputIME.py and add a new one I have created a "TextInputIME" class. We also declare the variable testtext as a StringProperty. When the processing of the textedit event comes, the character being input is stored in testtext. After that, in the kv file, the contents of testtext are displayed in Label.

The actual operation screen is as follows.

Entering

editing.png

As you type, the characters you type appear on the IME label (gray area).

After confirming the input

complete.png When you press the enter key and type, the characters you type will be displayed in textinput.

What is not currently possible

I managed to display the Japanese I was typing, but I couldn't do the following due to a problem (specification?) On the SDL side.

However, this content itself is not a very critical factor for me personally, so the current response is undecided.

Recommended Posts

The story that Kivy's Japanese input characters are displayed
The story that Japanese output was confused with Django
[Note] Japanese characters are garbled with atom-runner
The story that FastAPI may take supremacy
The story that scipy suddenly stopped loading
The story that XGBoost was finally installed
How to deal with the problem that Japanese characters are garbled when outputting logs using JSON log formatter
What is the reason why the basic commands are not displayed in Japanese in man?
Bugs that static files are not displayed on the App Engine development server
The story that fits in with pip installation
The story when I was using IntelliJ on Linux and could not input Japanese