How to communicate asynchronously only with Touch Designer

Introduction

TouchDesigner (TD) can execute python scripts, If processing such as HTTP communication takes time, it will freeze a little until the processing is completed. At this time, ** the timeline of the entire TD will also stop and the drawing will not be updated **, so To prevent this, we decided to communicate asynchronously.

If you just want to get the value asynchronously, you can use node etc. in addition to TD, but Due to the circumstances of the environment, I wanted to complete it with only TD, so I considered the following method.

--Use Web DAT --Use the threading module

I will explain each of these.

Sample code

This sample is uploaded below. https://github.com/genkitoyama/TD_with_asynchronous_communication

Use Web DAT

This is the easiest. With the parameter ʻAsynchronous Fetchturned on, Enter the URL and pulse theFetch` button, The TD timeline doesn't stop and communicates behind the scenes (maybe curl?). Reference: https://docs.derivative.ca/Web_DAT

Dec-15-2019 14-18-35.gif (In the above example, API for finding the altitude of the Geographical Survey Institute is used.)

In the case of GET, it is OK if you enter it directly in the URL, In the case of POST, it is OK if you pulse the Submit button with the table of the data you want to send connected to the upper inlet ʻInput 0`. (By the way, you can put a custom HTTP header in the lower inlet)

Then, according to the format of the returned one, pull the required value using the XML DAT or json module, and you're done.

Reference: Parse JSON to DAT Table with TouchDesigner

Use the threading module

threading module Is a module that performs parallel processing with multiple threads. It is included as standard in python3.5 inside TD.

By putting the function you want to execute in another thread and the argument of that function in the argument of threading.Thread (), it will be executed in another thread.

Here, requests is used as the module for HTTP communication. requests is also included as standard in python inside TD. (There are several other modules that communicate with HTTP, but I feel that you can use whatever you like.)


import threading
import requests

class HttpRequest():
	def __init__(self):
		self.url = op('URL').text
		self.city_id = op('CITY_ID').text
	
    #Functions you want to execute asynchronously
	def request(self):
		response = requests.get(self.url, params={'city':self.city_id})
		
	def start_request(self):
		myThread = threading.Thread(target=self.request)
		myThread.start()

This is a great reference to Article by Dr. Matthew Ragan.

Disadvantages and countermeasures

The threading module is also very useful, but it has some drawbacks.

--Cannot detect when the process in another thread ended --Cannot refer to TD operator while processing another thread

Unable to detect when processing in another thread ended

I can send a stop command from the main thread, It is not possible to detect when the processing in another thread is completed. You can use the join () method to wait for the end of another thread, During that time, the main thread will stop, so the TD will eventually stop. ..

Therefore, it is not easy to do something like "execute this process when the value is returned by hitting the URL in another thread".

Cannot refer to TD operator while processing another thread

This is natural if you think about it, but since the process is run in another thread, Unable to access the operator that resides in the TD's main thread.

Therefore, it is not possible to "parse the result returned by hitting the URL in another thread to Table DAT as it is in the same function ".

(By the way, if you do that, you will get a dialog like this) multi_threading_alert.png

Countermeasures

So, it's quite a skill, but I decided to use Timer CHOP to monitor whether the value is returned (≒ whether the processing of another thread is completed) ** at regular intervals.

I would like you to see the sample for details, but roughly speaking, the flow is as follows.

  1. Write the process to store the obtained value in the member variable in the function executed in another thread.
  2. Check if the variable contains a value in ʻonDone ()of theTimer CHOP` callback.
  3. If it is, the timer will end and proceed to another process.
  4. If not, restart the timer

Dec-15-2019 14-28-46.gif

In the above example, livedoor's Weather Hacks is used. With this API, the result will be returned in an instant, so even if you do it this way, it will not be very delicious, but if it is the original API you used before, it will take 7 to 8 seconds to return due to circumstances. So I devised this method.

python tried to create RequestManager class and HttpRequest class as follows.


import http_request
import json

class RequestManager():
	def __init__(self):
		self.http = http_request.HttpRequest()
		self.can_start_request = True			#May I make a request
		self.max_confirmation_count = 10		#Maximum confirmation count
		self.current_confirmation_count = 0		#Current confirmation count
		self.timer = op('timer1')
	
	def init_timer(self):
		print('init http request timer')
		
	def start_timer(self):
		#If you can make a request, throw a request, if not, wait
		if self.can_start_request == True:
			print('start http request timer')
			self.http.start_request()
		else:
			print('restart timer')
			
	def done_timer(self):
		print('done http request timer')
		#Check if the value is returned
		output = self.http.get_data()
		#If there is no value, increase the confirmation count and try again
		if output is None:
			print('http response is none')
			self.can_start_request = False
			self.current_confirmation_count += 1
			#When the confirmation count reaches the maximum, it is judged that communication has failed.
			if self.current_confirmation_count >= self.max_confirmation_count:
				print('http out of request')
				self.timer.par.initialize.pulse()
				self.http = http_request.HttpRequest()
				self.current_confirmation_count = 0
			else:
				self.timer.par.start.pulse()
		else:
		#If there is a value, json is parsed and stored
			self.can_start_request = True
			out_json = json.loads(output)
			op('out_json')[0,0].val = out_json['forecasts'][0]
			
			self.current_confirmation_count = 0
			self.timer.par.initialize.pulse()

import threading
import requests

class HttpRequest():
	def __init__(self):
		self.url = op('URL').text
		self.city_id = op('CITY_ID').text
		self.out_data = None
		print('http request init')
	
	def request(self):
		get_data = {'city':self.city_id}
		response = requests.get(self.url, params=get_data)
		self.out_data = response.text
		print(self.out_data)
		
	def start_request(self):
		self.out_data = None
		myThread = threading.Thread(target=self.request)
		myThread.start()
		
	def get_data(self):
		return self.out_data

in conclusion

――We have summarized two types of asynchronous communication methods using only TouchDesigner. ――Because I'm a python beginner, I would be grateful if you could comment on the last part of the brute force, especially if you have any ideas for a better method ...!

bonus

――This time, I used TouchPlayer to run each on multiple terminals, but since I can't see the Textport with TouchPlayer, I used the method of spitting out the log to an external file and checking it if an error occurred. -(There is also an option to open TouchDesigner as ReadOnly.) ――At that time, loguru was very convenient as a module to spit out logs, so I recommend it.

Referenced links

Recommended Posts

How to communicate asynchronously only with Touch Designer
How to use Qt Designer
How to update with SQLAlchemy?
How to cast with Theano
How to Alter with SQLAlchemy?
How to separate strings with','
How to RDP with Fedora31
How to Delete with SQLAlchemy?
How to cancel RT with tweepy
Python: How to use async with
How to use virtualenv with PowerShell
How to deal with imbalanced data
How to install python-pip with ubuntu20.04LTS
How to deal with imbalanced data
How to get started with Scrapy
How to get started with Python
How to deal with DistributionNotFound errors
How to get started with Django
How to Data Augmentation with PyTorch
How to use FTP with Python
How to calculate date with python
How to install mysql-connector with pip3
How to INNER JOIN with SQLAlchemy
How to install Anaconda with pyenv
How to authenticate with Django Part 2
How to authenticate with Django Part 3
How to change Django's SQLite3 uploaded to python anywhere with GUI only
How to do arithmetic with Django template
[Blender] How to set shape_key with script
How to title multiple figures with matplotlib
How to get parent id with sqlalchemy
Python to remember only with hello, worlds
How to add a package with PyCharm
How to install DLIB with 2020 / CUDA enabled
How to use ManyToManyField with Django's Admin
How to use Cmder with PyCharm (Windows)
How to prevent package updates with apt
How to work with BigQuery in Python
How to use Ass / Alembic with HtoA
How to deal with enum compatibility errors
How to use Japanese with NLTK plot
How to do portmanteau test with python
How to search Google Drive with Google Colaboratory
How to display python Japanese with lolipop
How to download youtube videos with youtube-dl
How to use jupyter notebook with ABCI
How to power off Linux with Ultra96-V2
"How to pass PATH" to learn with homebrew
How to scrape websites created with SPA
How to use CUT command (with sample)
How to enter Japanese with Python curses
[Python] How to deal with module errors
How to install zsh (with .zshrc customization)
How to read problem data with paiza
How to use SQLAlchemy / Connect with aiomysql
How to get started with laravel (Linux)
How to group volumes together with LVM
How to install python3 with docker centos
How to use JDBC driver with Redash
How to selectively delete past tweets with Tweepy
How to upload with Heroku, Flask, Python, Git (4)