If you want to make a Windows application (exe) that can be actually used now using only Python
Introduction
This article is based on the theme of Qiita Summer Festival 2020 "If you want to make a △△ (app) using only 〇〇 (language)" The contents are in line with it.
On July 5, 2020, my work "VMD sizing ver5.00 (exe ｜ code)" Has been released. The theme of this tool is to "regenerate VMD (MMD motion data) with the appropriate head and body for the specified model", well, it sounds to most of you who are reading this article. It's a hobby app.
** Nga! !! !! ** **
Most of the people who enjoy MMD (MikuMikuDance) are very ordinary people who have nothing to do with Python or programs. If anyone responds with "Snake ?!", I think it's rather valuable. How can such people use their own apps? This article is about the theme of how to make a ** "App that adjusts the UI to something nice after a while" **, which has come to the fore after such anguish, trial and error, and a lot of trouble. , It is a summary of my own answers. By the way, "VMD sizing" has been used to the extent that the cumulative total exceeds DL8500. (Approximately 9600 DL when combined with the 32-bit version)
Python + pyinstaller = exe is not so rare, but it requires some ingenuity to bring it down to a level that can withstand actual operation.
<"Isn't it okay to make it in C?" ** I like Python. ** (Because I don't know C ...)
1. Environment construction
1.1. Installing Anaconda
First, let's prepare the development environment.
<"It's not a reason for machine learning, isn't it okay to leave it raw?" **No way! !! ** **
A common problem in pyinstaller articles is that "extra libraries are included and the exe file becomes large". It's common to try out new libraries during development. However, if you create an exe as it is, there is a high possibility that unnecessary libraries will be included in the exe. ** Let's separate the development environment and the release environment exactly. ** **
Download the installer from Anaconda Official. If it is made from now on, it would be better to use 3 series.
After DL, please follow the steps to install.
1.2. Building a development environment
First, let's build a virtual environment for development.
conda create -n pytest_env pip python=3.7
Once you have a development environment, let's ʻactivate`. By the way, let's also create a source code management directory.
1.3. Building a release environment
Similarly, create a release environment,
conda create -n pytest_release pip python=3.7
1.4. Library installation
Once you have the management directory, move to it and install the required libraries. Here is one trick.
** pyinstaller installs only in release environment **
By installing pyinstaller only in the release environment, you can prevent accidental release in the development environment.
Since it's a big deal, let's introduce numpy.
Installation command for development environment
pip install numpy wxPython
Installation command for release environment
pip install numpy wxPython pypiwin32 pyinstaller
pypiwin32 seems to be the library needed to run pyinstaller on Windows.
The GUI is easy to create using WxFormBuilder. However, there are some things that are difficult to understand the automatic naming convention, that parts cannot be reused, and that there are some things that are not enough to create an actual operation application, so I output it when it is in a certain form, and after that I have to do it myself. I recommend it.
Reference: GUI (WxFormBuilder) in Python (mm_sys) https://qiita.com/mm_sys/items/716cb159ea8c9e634300
After this, we will proceed in a reverse lookup format. Please see the section you are interested in.
3. exe reverse lookup TIPS collection with python
3.1. How to run a logic thread while running a GUI thread-with a suspend function
I think most of the people who are interested in this article are interested in it. I would love to know if there is a correct answer. So I skipped various things and brought it to the beginning.
--Start a logic thread that runs for a long time while keeping the GUI thread as it is --Multiprocess can also be executed in the logic thread --Logic threads do not increase processes --When the GUI thread is terminated, the logic thread is also terminated. --End the logic thread with the suspend button --The button for logic thread is valid only for single click (double click is disabled)
The code that meets the above requirements is below.
executor.py
# -*- coding: utf-8 -*-
#
import wx
import sys
import argparse
import numpy as np
import multiprocessing
from pathlib import Path
from form.MainFrame import MainFrame
from utils.MLogger import MLogger
VERSION_NAME = "ver1.00"
#No exponential notation, omitted if the number of effective decimal places exceeds 6 or 30, and the number of characters in one line is 200.
np.set_printoptions(suppress=True, precision=6, threshold=30, linewidth=200)
#Windows multi-process measures
multiprocessing.freeze_support()
if __name__ == '__main__':
#Argument interpretation
parser = argparse.ArgumentParser()
parser.add_argument("--verbose", default=20, type=int)
args = parser.parse_args()
#Logger initialization
MLogger.initialize(level=args.verbose, is_file=False)
#GUI startup
app = wx.App(False)
frame = MainFrame(None, VERSION_NAME, args.verbose)
frame.Show(True)
app.MainLoop()
First, the caller ʻexecutor.py`. Launch the GUI from here.
MainFrame.py
# -*- coding: utf-8 -*-
#
from time import sleep
from worker.LongLogicWorker import LongLogicWorker
from form.ConsoleCtrl import ConsoleCtrl
from utils.MLogger import MLogger
import os
import sys
import wx
import wx.lib.newevent
logger = MLogger(__name__)
TIMER_ID = wx.NewId()
(LongThreadEvent, EVT_LONG_THREAD) = wx.lib.newevent.NewEvent()
#Main GUI
class MainFrame(wx.Frame):
def __init__(self, parent, version_name: str, logging_level: int):
self.version_name = version_name
self.logging_level = logging_level
self.elapsed_time = 0
self.worker = None
#Initialization
wx.Frame.__init__(self, parent, id=wx.ID_ANY, title=u"c01 Long Logic {0}".format(self.version_name), \
pos=wx.DefaultPosition, size=wx.Size(600, 650), style=wx.DEFAULT_FRAME_STYLE)
self.sizer = wx.BoxSizer(wx.VERTICAL)
#processing time
self.loop_cnt_ctrl = wx.SpinCtrl(self, id=wx.ID_ANY, size=wx.Size(100, -1), value="2", min=1, max=999, initial=2)
self.loop_cnt_ctrl.SetToolTip(u"processing time")
self.sizer.Add(self.loop_cnt_ctrl, 0, wx.ALL, 5)
#Check box for parallel processing
self.multi_process_ctrl = wx.CheckBox(self, id=wx.ID_ANY, label="If you want to execute parallel processing, please check it.")
self.sizer.Add(self.multi_process_ctrl, 0, wx.ALL, 5)
#Button Sizer
self.btn_sizer = wx.BoxSizer(wx.HORIZONTAL)
#Run button
self.exec_btn_ctrl = wx.Button(self, wx.ID_ANY, u"Start long logic processing", wx.DefaultPosition, wx.Size(200, 50), 0)
#Binding with mouse left click event [Point.01】
self.exec_btn_ctrl.Bind(wx.EVT_LEFT_DOWN, self.on_exec_click)
#Binding with mouse left double click event [Point.03】
self.exec_btn_ctrl.Bind(wx.EVT_LEFT_DCLICK, self.on_doubleclick)
self.btn_sizer.Add(self.exec_btn_ctrl, 0, wx.ALIGN_CENTER, 5)
#Suspend button
self.kill_btn_ctrl = wx.Button(self, wx.ID_ANY, u"Long logic processing interruption", wx.DefaultPosition, wx.Size(200, 50), 0)
#Bind with mouse left click event
self.kill_btn_ctrl.Bind(wx.EVT_LEFT_DOWN, self.on_kill_click)
#Bind with mouse left double click event
self.kill_btn_ctrl.Bind(wx.EVT_LEFT_DCLICK, self.on_doubleclick)
#Initial state is inactive
self.kill_btn_ctrl.Disable()
self.btn_sizer.Add(self.kill_btn_ctrl, 0, wx.ALIGN_CENTER, 5)
self.sizer.Add(self.btn_sizer, 0, wx.ALIGN_CENTER | wx.SHAPED, 0)
#Console [Point.06】
self.console_ctrl = ConsoleCtrl(self)
self.sizer.Add(self.console_ctrl, 1, wx.ALL | wx.EXPAND, 5)
#print Output destination is console [Point.05】
sys.stdout = self.console_ctrl
#Progress gauge
self.gauge_ctrl = wx.Gauge(self, wx.ID_ANY, 100, wx.DefaultPosition, wx.DefaultSize, wx.GA_HORIZONTAL)
self.gauge_ctrl.SetValue(0)
self.sizer.Add(self.gauge_ctrl, 0, wx.ALL | wx.EXPAND, 5)
#Event bind [Point.05】
self.Bind(EVT_LONG_THREAD, self.on_exec_result)
self.SetSizer(self.sizer)
self.Layout()
#Display in the center of the screen
self.Centre(wx.BOTH)
#Double click invalidation process
def on_doubleclick(self, event: wx.Event):
self.timer.Stop()
logger.warning("It was double-clicked.", decoration=MLogger.DECORATION_BOX)
event.Skip(False)
return False
#Execute 1 Processing when clicked
def on_exec_click(self, event: wx.Event):
#Start with a slight delay with a timer (avoid batting with double click) [Point.04】
self.timer = wx.Timer(self, TIMER_ID)
self.timer.StartOnce(200)
self.Bind(wx.EVT_TIMER, self.on_exec, id=TIMER_ID)
#Interruption 1 Click processing
def on_kill_click(self, event: wx.Event):
self.timer = wx.Timer(self, TIMER_ID)
self.timer.StartOnce(200)
self.Bind(wx.EVT_TIMER, self.on_kill, id=TIMER_ID)
#Processing execution
def on_exec(self, event: wx.Event):
self.timer.Stop()
if not self.worker:
#Console clear
self.console_ctrl.Clear()
#Disable execute button
self.exec_btn_ctrl.Disable()
#Suspend button enabled
self.kill_btn_ctrl.Enable()
#Execute in another thread [Point.09】
self.worker = LongLogicWorker(self, LongThreadEvent, self.loop_cnt_ctrl.GetValue(), self.multi_process_ctrl.GetValue())
self.worker.start()
event.Skip(False)
#Suspend processing execution
def on_kill(self, event: wx.Event):
self.timer.Stop()
if self.worker:
#When the button is pressed in the stopped state, it stops
self.worker.stop()
logger.warning("Interrupts long logic processing.", decoration=MLogger.DECORATION_BOX)
#Worker end
self.worker = None
#Run button enabled
self.exec_btn_ctrl.Enable()
#Disable suspend button
self.kill_btn_ctrl.Disable()
#Hide progress
self.gauge_ctrl.SetValue(0)
event.Skip(False)
#Processing after long logic is over
def on_exec_result(self, event: wx.Event):
# 【Point.12] Make the logic end explicitly known
self.sound_finish()
#Run button enabled
self.exec_btn_ctrl.Enable()
#Disable suspend button
self.kill_btn_ctrl.Disable()
if not event.result:
event.Skip(False)
return False
self.elapsed_time += event.elapsed_time
logger.info("\n Processing time: %s", self.show_worked_time())
#Worker end
self.worker = None
#Hide progress
self.gauge_ctrl.SetValue(0)
def sound_finish(self):
#Sound the end sound
if os.name == "nt":
# Windows
try:
import winsound
winsound.PlaySound("SystemAsterisk", winsound.SND_ALIAS)
except Exception:
pass
def show_worked_time(self):
#Convert elapsed seconds to hours, minutes and seconds
td_m, td_s = divmod(self.elapsed_time, 60)
if td_m == 0:
worked_time = "{0:02d}Seconds".format(int(td_s))
else:
worked_time = "{0:02d}Minutes{1:02d}Seconds".format(int(td_m), int(td_s))
return worked_time
Point.01: Bind mouse left click event and execution method
First of all, let's bind the mouse left-click event on the button and the method to be executed.
There are several binding methods, but I personally like to write Parts.Bind (event type, firing method) for binding GUI parts and events.
Point.02: Start the left-click event with a timer slightly delayed
If you pick up the left-click event and execute it as it is, it will fire at the same time as the double-click event, and as a result, the double-click event will run.
(In terms of processing, the double-click event is part of the single-click event, so the two events fire at the same time.)
Therefore, by setting a timer in ʻon_exec_click and ʻon_kill_click fired from a single click and executing it with a slight delay, the execution method bound to the double-click event will be executed first.
Point.03: Bind mouse left double click event and execution method
Bind the left double-click event in the same procedure as Point ①. You can prevent double processing by picking up double clicks here.
Point.04: Stop the timer event with the mouse left double-click execution method
The double-click event will stop the timer event executed at Point②. You can now disable double-clicking.
Only single-click event fires … The corresponding event will be executed with a slight delay
When the double click event fires … The single-click event is not executed because the timer is stopped.
Point.05: Set the output destination of print to console control
print is a wrapper for sys.stdout.write, so if you set the output destination to a console control, the output destination of print will be inside the control.
Point.06: Define console control in subclass
So, what is that console control? It's a subclass of wx.TextCtrl.
ConsoleCtrl.py
# -*- coding: utf-8 -*-
#
import wx
from utils.MLogger import MLogger # noqa
logger = MLogger(__name__)
class ConsoleCtrl(wx.TextCtrl):
def __init__(self, parent):
#Multiple lines allowed, read-only, no border, vertical scrolling, horizontal scrolling, key event acquisition
super().__init__(parent, wx.ID_ANY, wx.EmptyString, wx.DefaultPosition, wx.Size(-1, -1), \
wx.TE_MULTILINE | wx.TE_READONLY | wx.BORDER_NONE | wx.HSCROLL | wx.VSCROLL | wx.WANTS_CHARS)
self.SetBackgroundColour(wx.SystemSettings.GetColour(wx.SYS_COLOUR_3DLIGHT))
#Keyboard event binding
self.Bind(wx.EVT_CHAR, lambda event: self.on_select_all(event, self.console_ctrl))
#Output processing of console part [Point.07】
def write(self, text):
try:
wx.CallAfter(self.AppendText, text)
except: # noqa
pass
#All selection process of console part [Point.08】
def on_select_all(event, target_ctrl):
keyInput = event.GetKeyCode()
if keyInput == 1: # 1 stands for 'ctrl+a'
target_ctrl.SelectAll()
event.Skip()
Point.07: Perform additional processing in the write method
Call the ʻAppendText method with CallAfter, assuming that it will be called from a logic thread different from the GUI thread. This will also stabilize the print` output from the logic thread.
Point.08: Add method for select event in console control
If there are letters, it is the human saga that makes you want to copy them. Therefore, the all-selection process is executed in the all-selection event (combination of keyboard events).
Point.09: Execute the logic thread in another thread
I finally got into the main subject.
Perform logic processing within LongLogicWorker.
Reference source: https://doloopwhile.hatenablog.com/entry/20090627/1275175850
LongLogicWorker.py
# -*- coding: utf-8 -*-
#
import os
import wx
import time
from worker.BaseWorker import BaseWorker, task_takes_time
from service.MOptions import MOptions
from service.LongLogicService import LongLogicService
class LongLogicWorker(BaseWorker):
def __init__(self, frame: wx.Frame, result_event: wx.Event, loop_cnt: int, is_multi_process: bool):
#processing time
self.loop_cnt = loop_cnt
#Whether to run in multiple processes
self.is_multi_process = is_multi_process
super().__init__(frame, result_event)
@task_takes_time
def thread_event(self):
start = time.time()
#Para and options stuffing
# max_The maximum value of workers is Python3.Based on the default value of 8
options = MOptions(self.frame.version_name, self.frame.logging_level, self.loop_cnt, max_workers=(1 if not self.is_multi_process else min(32, os.cpu_count() + 4)))
#Logic service execution
LongLogicService(options).execute()
#elapsed time
self.elapsed_time = time.time() - start
def post_event(self):
#Call and execute the event after the logic processing is completed [Point.11】
wx.PostEvent(self.frame, self.result_event(result=self.result and not self.is_killed, elapsed_time=self.elapsed_time))
LongLogicWorker inherits from BaseWorker.
BaseWorker.py
# -*- coding: utf-8 -*-
#
import wx
import wx.xrc
from abc import ABCMeta, abstractmethod
from threading import Thread
from functools import wraps
import time
import threading
from utils.MLogger import MLogger # noqa
logger = MLogger(__name__)
# https://wiki.wxpython.org/LongRunningTasks
# https://teratail.com/questions/158458
# http://nobunaga.hatenablog.jp/entry/2016/06/03/204450
class BaseWorker(metaclass=ABCMeta):
"""Worker Thread Class."""
def __init__(self, frame, result_event):
"""Init Worker Thread Class."""
#Parent GUI
self.frame = frame
#elapsed time
self.elapsed_time = 0
#Call event after the thread ends
self.result_event = result_event
#Progress gauge
self.gauge_ctrl = frame.gauge_ctrl
#Successful processing
self.result = True
#With or without stop command
self.is_killed = False
#Thread start
def start(self):
self.run()
#Thread stop
def stop(self):
#Turn on interrupt FLG
self.is_killed = True
def run(self):
#Thread execution
self.thread_event()
#Post-processing execution
self.post_event()
def post_event(self):
wx.PostEvent(self.frame, self.result_event(result=self.result))
@abstractmethod
def thread_event(self):
pass
# https://doloopwhile.hatenablog.com/entry/20090627/1275175850
class SimpleThread(Thread):
"""A thread that just executes a callable object (such as a function)"""
def __init__(self, base_worker, acallable):
#Processing in another thread
self.base_worker = base_worker
#Methods to run in the function decorator
self.acallable = acallable
#Function decorator results
self._result = None
#Suspended FLG=Initialize in the OFF state
super(SimpleThread, self).__init__(name="simple_thread", kwargs={"is_killed": False})
def run(self):
self._result = self.acallable(self.base_worker)
def result(self):
return self._result
def task_takes_time(acallable):
"""
Function Decorator [Point.10】
While executing the original processing of acallable in another thread
Update window wx.Keep calling YieldIfNeeded
"""
@wraps(acallable)
def f(base_worker):
t = SimpleThread(base_worker, acallable)
#A demon that kills a child when a parent dies
t.daemon = True
t.start()
#Keep updating window drawings for the life of the thread
while t.is_alive():
#Twirl the progress gauge
base_worker.gauge_ctrl.Pulse()
#Refresh the window if necessary
wx.YieldIfNeeded()
#Wait a little
time.sleep(0.01)
if base_worker.is_killed:
# 【Point.23] If the caller issues a stop command, you(GUI)Termination command to all threads except
for th in threading.enumerate():
if th.ident != threading.current_thread().ident and "_kwargs" in dir(th):
th._kwargs["is_killed"] = True
break
return t.result()
return f
Point.10: Run a GUI thread and another thread with the function decorator
This is a story I have already received from the reference site, but I will continue to update the drawing of the GUI thread while running another thread with the function decorator.
In SimpleThread, execute ʻacallable. At this time, the reason for holding BaseWorkeris to pass the suspension flag. WhileSimpleThread` is alive, the GUI thread only updates the drawing and accepts interruptions.
(This area will be shown later)
Point.11: Call and execute the event after the logic processing is completed
Since the call event after processing was passed when the worker was initialized in advance, execute it with wx.PostEvent.
This will bring you back to the process in the GUI.
Point.12: Make the logic end explicitly known
Until the logic is finished, not all users are stuck on the PC all the time, so to make it easier to understand when it is finished, sizing makes an INFO sound. Depending on the Windows environment and other environments such as Linux, an error may occur, so it seems better to make it sound only when it can be sounded with try-except. By the way, if you give the elapsed time, the feeling of how long it took will be quantified, so I feel that it is easy to understand.
LongLogicService.py
# -*- coding: utf-8 -*-
#
import logging
from time import sleep
import concurrent.futures
from concurrent.futures import ThreadPoolExecutor
from service.MOptions import MOptions
from utils.MException import MLogicException, MKilledException
from utils.MLogger import MLogger # noqa
logger = MLogger(__name__)
class LongLogicService():
def __init__(self, options: MOptions):
self.options = options
def execute(self):
logging.basicConfig(level=self.options.logging_level, format="%(message)s [%(module_name)s]")
# 【Point.13] try the whole-Enclose in except and output the error content
try:
#It is OK to put logic normally
self.execute_inner(-1)
logger.info("--------------")
#It is OK to distribute with parallel tasks
futures = []
# 【Point.14] Give names to parallel tasks
with ThreadPoolExecutor(thread_name_prefix="long_logic", max_workers=self.options.max_workers) as executor:
for n in range(self.options.loop_cnt):
futures.append(executor.submit(self.execute_inner, n))
#【Point.15] Parallel tasks wait for completion after being issued in bulk
concurrent.futures.wait(futures, timeout=None, return_when=concurrent.futures.FIRST_EXCEPTION)
for f in futures:
if not f.result():
return False
logger.info("End of long logic processing", decoration=MLogger.DECORATION_BOX, title="Logic end")
return True
except MKilledException:
#In case of termination by suspend option, only the result is returned as it is
return False
except MLogicException as se:
#Incomplete data error
logger.error("It ended with data that cannot be processed.\n\n%s", se.message, decoration=MLogger.DECORATION_BOX)
return False
except Exception as e:
#Other errors
logger.critical("The process ended with an unintended error.", e, decoration=MLogger.DECORATION_BOX)
return False
finally:
logging.shutdown()
def execute_inner(self, n: int):
for m in range(5):
logger.info("n: %s - m: %s", n, m)
sleep(1)
return True
Point.13: In the logic thread, enclose the whole with try-except
Since the threads are separated, if you do not exclude the error properly, you may end up suddenly though you do not know what it is. It's hard to chase afterwards, so let's exclude it and log it.
Point.14: Give names to parallel tasks
When executing parallel tasks, it is easier to debug if you add a prefix so that you can easily understand which process is the problem.
Point.15: Parallel tasks wait for the end after being issued in a batch
Concurrent tasks are first issued with ʻexecutor.submitand then waited withconcurrent.futures.waituntil all processing is complete. At that time, theconcurrent.futures.FIRST_EXCEPTION` option is added so that processing will be interrupted if any Exception occurs.
MLogger.py
# -*- coding: utf-8 -*-
#
from datetime import datetime
import logging
import traceback
import threading
from utils.MException import MKilledException
# 【Point.16] Implement your own logger
class MLogger():
DECORATION_IN_BOX = "in_box"
DECORATION_BOX = "box"
DECORATION_LINE = "line"
DEFAULT_FORMAT = "%(message)s [%(funcName)s][P-%(process)s](%(asctime)s)"
DEBUG_FULL = 2
TEST = 5
TIMER = 12
FULL = 15
INFO_DEBUG = 22
DEBUG = logging.DEBUG
INFO = logging.INFO
WARNING = logging.WARNING
ERROR = logging.ERROR
CRITICAL = logging.CRITICAL
total_level = logging.INFO
is_file = False
outout_datetime = ""
logger = None
#Initialization
# 【Point.17] Be able to define the minimum output level for each module
def __init__(self, module_name, level=logging.INFO):
self.module_name = module_name
self.default_level = level
#Logger
self.logger = logging.getLogger("PyLogicSample").getChild(self.module_name)
#Standard output handler
sh = logging.StreamHandler()
sh.setLevel(level)
self.logger.addHandler(sh)
# 【Point.18] Prepare a log method with a lower level than debug
def test(self, msg, *args, **kwargs):
if not kwargs:
kwargs = {}
kwargs["level"] = self.TEST
kwargs["time"] = True
self.print_logger(msg, *args, **kwargs)
def debug(self, msg, *args, **kwargs):
if not kwargs:
kwargs = {}
kwargs["level"] = logging.DEBUG
kwargs["time"] = True
self.print_logger(msg, *args, **kwargs)
def info(self, msg, *args, **kwargs):
if not kwargs:
kwargs = {}
kwargs["level"] = logging.INFO
self.print_logger(msg, *args, **kwargs)
def warning(self, msg, *args, **kwargs):
if not kwargs:
kwargs = {}
kwargs["level"] = logging.WARNING
self.print_logger(msg, *args, **kwargs)
def error(self, msg, *args, **kwargs):
if not kwargs:
kwargs = {}
kwargs["level"] = logging.ERROR
self.print_logger(msg, *args, **kwargs)
def critical(self, msg, *args, **kwargs):
if not kwargs:
kwargs = {}
kwargs["level"] = logging.CRITICAL
self.print_logger(msg, *args, **kwargs)
#Actual output
def print_logger(self, msg, *args, **kwargs):
#【Point.22] Suspend FLG on the thread currently running=If ON is set, an interruption error will occur.
if "is_killed" in threading.current_thread()._kwargs and threading.current_thread()._kwargs["is_killed"]:
#If a stop command is issued, an error
raise MKilledException()
target_level = kwargs.pop("level", logging.INFO)
#Output only when both application and module log levels are met
if self.total_level <= target_level and self.default_level <= target_level:
if self.is_file:
for f in self.logger.handlers:
if isinstance(f, logging.FileHandler):
#Delete all existing file handlers
self.logger.removeHandler(f)
#If there is file output, handler association
#File output handler
fh = logging.FileHandler("log/PyLogic_{0}.log".format(self.outout_datetime))
fh.setLevel(self.default_level)
fh.setFormatter(logging.Formatter(self.DEFAULT_FORMAT))
self.logger.addHandler(fh)
#Added to output module name
extra_args = {}
extra_args["module_name"] = self.module_name
#Log record generation
if args and isinstance(args[0], Exception):
# 【Point.19] When an Exception is received, a stack trace is output.
log_record = self.logger.makeRecord('name', target_level, "(unknown file)", 0, "{0}\n\n{1}".format(msg, traceback.format_exc()), None, None, self.module_name)
else:
log_record = self.logger.makeRecord('name', target_level, "(unknown file)", 0, msg, args, None, self.module_name)
target_decoration = kwargs.pop("decoration", None)
title = kwargs.pop("title", None)
print_msg = "{message}".format(message=log_record.getMessage())
# 【Point.20] Decorate log messages with parameters
if target_decoration:
if target_decoration == MLogger.DECORATION_BOX:
output_msg = self.create_box_message(print_msg, target_level, title)
elif target_decoration == MLogger.DECORATION_LINE:
output_msg = self.create_line_message(print_msg, target_level, title)
elif target_decoration == MLogger.DECORATION_IN_BOX:
output_msg = self.create_in_box_message(print_msg, target_level, title)
else:
output_msg = self.create_simple_message(print_msg, target_level, title)
else:
output_msg = self.create_simple_message(print_msg, target_level, title)
#output
try:
if self.is_file:
#If there is file output, regenerate the record and output both console and GUI
log_record = self.logger.makeRecord('name', target_level, "(unknown file)", 0, output_msg, None, None, self.module_name)
self.logger.handle(log_record)
else:
# 【Point.21] Logic thread is output separately for print and logger
print(output_msg)
self.logger.handle(log_record)
except Exception as e:
raise e
def create_box_message(self, msg, level, title=None):
msg_block = []
msg_block.append("■■■■■■■■■■■■■■■■■")
if level == logging.CRITICAL:
msg_block.append("■ **CRITICAL** ")
if level == logging.ERROR:
msg_block.append("■ **ERROR** ")
if level == logging.WARNING:
msg_block.append("■ **WARNING** ")
if level <= logging.INFO and title:
msg_block.append("■ **{0}** ".format(title))
for msg_line in msg.split("\n"):
msg_block.append("■ {0}".format(msg_line))
msg_block.append("■■■■■■■■■■■■■■■■■")
return "\n".join(msg_block)
def create_line_message(self, msg, level, title=None):
msg_block = []
for msg_line in msg.split("\n"):
msg_block.append("■■ {0} --------------------".format(msg_line))
return "\n".join(msg_block)
def create_in_box_message(self, msg, level, title=None):
msg_block = []
for msg_line in msg.split("\n"):
msg_block.append("■ {0}".format(msg_line))
return "\n".join(msg_block)
def create_simple_message(self, msg, level, title=None):
msg_block = []
for msg_line in msg.split("\n"):
# msg_block.append("[{0}] {1}".format(logging.getLevelName(level)[0], msg_line))
msg_block.append(msg_line)
return "\n".join(msg_block)
@classmethod
def initialize(cls, level=logging.INFO, is_file=False):
# logging.basicConfig(level=level)
logging.basicConfig(level=level, format=cls.DEFAULT_FORMAT)
cls.total_level = level
cls.is_file = is_file
cls.outout_datetime = "{0:%Y%m%d_%H%M%S}".format(datetime.now())
Point.16: Implement your own logger
Perhaps the logger part is the one that I have the most ingenuity. It is easier for users to output the log according to a certain format, but it is very troublesome to define it one by one. You can decorate the message box and ruled lines with a single flag. And above all, the logger is used to judge the interruption flag. (Details will be described later)
Point.17: Be able to define the minimum output level for each module
By defining the minimum output level for each module, you can suppress the debug log of utility methods in particular. Physically erasing the debug log or commenting it out can be a hassle to check if there is a problem. By raising or lowering the minimum level for each module, you can control the output log level, which will lead to easier debugging.
Point.18: Prepare a log method with a lower level than debug
Although it is paired with 17, it is easier to suppress the output by preparing a low-level method.
Point.19: Outputs a stack trace when an Exception is received
This is mainly useful when you get an unhandled exception. Also, since the logger is handled by both the GUI thread and the logic thread, there is no need to adjust the output at the source of the logic thread.
Point.20: Decorate log messages with parameters
Since we've made the console control a regular text control, we've used a lot of message blocking for clarity. Since the amount of messages is variable, it was not possible to assign a fixed character string, so blocking is called for each given parameter. I think there is a way to specify the method at the caller, but I think that this is easier to manage in terms of meaning. Even when decorating the text, the amount of code will be less if it is handled in one place in this way rather than being separated by the caller.
Point.21: Execute print and logger.handle separately during output processing
To print to the console control, you need the output from print, and to print to the stream, you need the output from logger.handle.
Both messages output the same information, and information such as the output module and output time is added to the output to the stream to make it easier to follow.
Point.22: If the thread currently running is set to suspend FLG = ON, an interruption error will occur.
This was where I was most worried ... The following are some of Python's thread-like practices. --Do not kill threads from the outside --Hold the suspend parameter and refer to the parameter every time it is processed internally --If the suspend parameter is ON, exit from the inside
Reference: https://stackoverflow.com/questions/323972/is-there-any-way-to-kill-a-thread
Even if you look at the interruption parameters internally, you don't want to see them for each logic process, and you don't want to carry around the parameters ... So, isn't it a logger that always passes any logic? So, if you can see it on the logger and you can change it from the GUI thread, it's about "current thread" ... By saying that, it became like this.
I don't know if it's a good method, but I like it because logic processing doesn't care about interruptions at all.
Point.23: When interrupt is instructed on the GUI side, turn on all thread interruption FLG except GUI.
The suspend FLG is set in the BaseWorker function decorator.
Suspend of all living threads other than GUI thread By turning on FLG, you can see the interruption by looking at any thread.
Now, when you try to output the log, an error will occur and you will be returned to the GUI.
Normal termination route
Suspended end route
3.2. Debug from VSCode
Now that we've created the environment, let's make it run from VS Code.
In the Python Path field of the workspace, specify the full path of ʻAnaconda> envs> Development Environment> python.exe`.
Use launch to specify the execution of the exe.
{
"folders": [
{
"path": "src"
}
],
"settings": {
"python.pythonPath": "C:\\Development\\Anaconda3\\envs\\pytest_env\\python.exe"
},
"launch": {
"version": "0.2.0",
"configurations": [
{
"name": "Python: debug",
"type": "python",
"request": "launch",
"program": "${workspaceFolder}/executor.py",
"console": "integratedTerminal",
"pythonPath": "${command:python.interpreterPath}",
"stopOnEntry": false,
"args": [
// "--verbose", "1", //minimum
// "--verbose", "2", // DEBUG_FULL
// "--verbose", "15", // FULL
"--verbose", "10", // TEST
// "--verbose", "20", // INFO
]
}
]
}
}
You can now launch the GUI from VS Code.
3.3. Create an exe
I've put together a lot of code, but in the end I have to make it an exe. So, here are the batches and config files that create PythonExe.
pyinstaller64.bat
@echo off
rem ---
rem ---Generate exe
rem ---
rem ---Change the current directory to the execution destination
cd /d %~dp0
cls
rem ---After switching to the release environment, run pyinstaller
rem ---Return to development environment when finished
activate pytest_release && pyinstaller --clean pytest64.spec && activate pytest_env
pytest64.spec
# -*- coding: utf-8 -*-
# -*- mode: python -*-
#PythonExe sample 64bit version
block_cipher = None
a = Analysis(['src\\executor.py'],
pathex=[],
binaries=[],
datas=[],
#Hidden library import
hiddenimports=['wx._adv', 'wx._html', 'pkg_resources.py2_warn'],
hookspath=[],
runtime_hooks=[],
#Excluded libraries
excludes=['mkl','libopenblas'],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
a.binaries,
a.zipfiles,
a.datas,
[],
#app name
name='PythonExeSample.exe',
#Whether to display the debug log when making an exe
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
upx_exclude=[],
runtime_tmpdir=None,
#Whether to display the console
console=False )
If you want to stick to an exe, just run the batch, switch to the release environment, run pyinstaller, and then back to the development environment.
Now you don't have to worry about inadvertently putting extra libraries in your release environment.
The spec file is a configuration file for pyinstaller, but it is a setting added by a comment line.
hiddenimports
pyinstaller basically automatically includes the libraries called inside the code, but there are some libraries that cannot be included as they are.
It is hidden imports that explicitly imports it.
The only way to find this is to change debug = False at the bottom to True and look for the part that is causing the error, which is a very simple task.
excludes
On the contrary, if you want to exclude it because the file size becomes large if it is bundled together, specify it with ʻexcludes. In this case, mkl and libopenblas` are excluded by referring to https://www.reddit.com/r/pygame/comments/aelypb/why_is_my_pyinstaller_executable_180_mb_large/.
The completed exe is about 30M. (Even if excluded, this size ...
3.4. What you want to add
--About icons --About external memory (json)
I may add it when my energy recovers. How are you doing here with VMD sizing? Questions are also welcome.
4. Source code
All of the above code can be found at https://github.com/miu200521358/PythonExeSample. If you are interested, please fork and take a look at the contents.
Recommended Posts