I forgot to evaluate the performance because of the habit of skipping "I tried to aim for the fastest FizzBuzz"

Introduction

I forgot to evaluate the performance in the article I wrote earlier, "I tried to aim for the fastest FizzBuzz". I'm an idiot. For that reason, we evaluated the performance of the missing program.

Evaluation method

The last FizzBuzz program I created

$ time -p ./fizzbuzz >/dev/null
real 0.00
user 0.00
sys 0.00
$ 

Even if you execute such as, it is not useful because the execution time of the program is too short to measure with the time command. With a little ingenuity

 $ time -p (for i in `seq 1 10000`; do ./fizzbuzz; done) >/dev/null
real 6.76
user 4.55
sys 4.41

A method of executing multiple times and dividing the execution time by the number of executions can be considered, but this method is not adopted because the repetition by the shell seems to be more expensive than the execution time of the program. This time, I decided to make up a short program that repeatedly executes the same program and use it.

repeat.c

#include <stdlib.h>
#include <stdio.h>
#include <fcntl.h>
#include <stdbool.h>
#include <unistd.h>
#include <errno.h>
#include <spawn.h>
#include <sys/wait.h>

int main(int argc, char* argv[])
{
    int opt;
    int n = 1;
    bool f = false;
    int fd = -1;
    opterr = 0;
    while ((opt = getopt(argc, argv, "hfn:")) != -1) {
        switch (opt) {
        case 'h':
        default:
            fprintf(stderr, "Usage: repeat [options]... command\n");
            exit(EXIT_FAILURE);
            break;
        case 'f':
            f = true;
            break;
        case 'n':
            n = atoi(optarg);
            break;
        }
    }
    if (f) {
        fd = open("/proc/sys/vm/drop_caches", O_WRONLY);
    }
    for (int i = 1; i <= n; i++) {
        if (f && fd >= 0) {
            while (write(fd, "1", 1) == -1 && errno == EINTR) {
                ;
            }
        }
        pid_t pid;
        posix_spawn(&pid, argv[optind], NULL, NULL, &argv[optind], NULL);
        int status;
        wait(&status);
        if (status) {
            fprintf(stderr, "pid %d: %d\n", pid, status);
            exit(status);
        }
    }
    if (f && fd >= 0) {
        close(fd);
    }
}

this

$ gcc -Wall -Wextra -O2 repeat.c -o repeat

Compile as and

python

$ time -p ./repeat -n 10000 ./fizzbuzz >/dev/null
real 1.84
user 1.44
sys 0.48
$ 

Use as. The command line option \ -n value'specifies the number of times the program is executed repeatedly, and the number output by the time command is divided by the number of executions to obtain the execution time per execution. Once the program is read from the disc, its contents are registered in the page cache, and it will not go to read the disc during repeated executions. Since we also want to evaluate the load of reading the program from the disk, we have provided a function to execute the program from the state where the page cache is cleared by specifying the command line option \ -f'. The page cache is cleared by writing 1 to / proc / sys / vm / drop_caches, and since administrator privileges are required, it is necessary to use sudo together.

$ sudo time -p ./repeat -n 10000 -f ./fizzbuzz >/dev/null
real 85.90
user 3.26
sys 78.85
$ 

This is also the execution time including the program load time by dividing the output number by the number of repetitions as in the previous example.

Measurement result

No cache clear (* The number is the time (seconds) executed 10,000 times)

There is cash clear (* same as above)

Executable file size (* unit is bytes)

Evaluation

• The result was that the assembler version was the fastest with and without cache clear (for the time being, I'm relieved).
• Under the condition without clearing the cache, the disk is read only for the first time, so the executable file size has almost no effect on the execution time (should). The difference that occurs seems to be due to the following two points.
• Time required to start up a program written in C (C language static link version, C language dynac link version)
• Time required for dynamic link (C language dynamic link version)
• Under the condition without cache clear, user + sys ≒ real holds, but under the condition with cache clear, there is a difference between the values of user + sys and real, and the difference seems to be the time required to wait for disk reading. Therefore, under the condition that the cache is cleared, it is considered that the larger the executable file size, the greater the influence on the real number.

in conclusion

The end.