Calculation time measurement using maf

When I make a new program, I often experiment with various parameters and measure what kind of parameter and how long it takes to calculate.

In order to measure this, it is necessary to extract and collect time from the output of each result. You can do this manually if you have a few, but if you have a lot of parameter combinations, you can't do it by hand, so use a script. To do this, you have to write scripts for parameter generation, result parsing, and result collection. I feel that such processing is often used in common, so I found a maf in the place where I was wondering if I could make a library and tried using it.

Since there was no sample of calculation time measurement in the current maf sample, I made it.

About maf

For details, it is best to read here (http://research.preferred.jp/2013/12/maf/), but in simple terms, it is made by extending the build system written in python called waf. It is a tool for supporting experiments.

With this, you can easily enumerate the combinations of parameters and easily summarize the experimental results. Wonderful!

wscript and experimental results

Like waf, maf needs to write Makefile in the make command called wscript. wscript etc. are in the following. https://github.com/shu65/maf_measure_computing_time

In this wscript, "sleep 0", "sleep 1", "sleep 2" are executed 3 times each, the time is measured using the time command, and the average calculation time is calculated for each parameter, which is a very simple sample. It has become.

The execution method after git clone is as follows.

python


cd maf_measure_computing_time
./waf configure
./waf experiment -j 1 

Note that without the final "-j 1", multiple combinations of parameter experiments cannot be performed simultaneously and accurate calculation time cannot be measured.

Finally

Currently, I don't want to execute in parallel only for measuring time, but I can't specify it with wscript, so I'd like to improve this area and send a Pull Request in the future ...

Recommended Posts

Calculation time measurement using maf
Time measurement using a clock
measurement of time
python time measurement
Age calculation using python
Measurement of execution time
Calculation problem using mod
Python execution time measurement memo
# 1 [python3] Simple calculation using variables
Time calculation operator datetime timedelta
Execution time measurement with Python With
Write processing time measurement a little easier using the with clause
Calculation of time series customer loyalty
Calculation of normal vector using convolution
Collective measurement of Volume using FSL
[FSL] Image measurement using ROI (VOI)
python> Processing time measurement> time.time () --start_time