Write O_SYNC file in C and Python

Write O_SYNC file in C and Python

Overview

Compare the time cost of writing a file in C and Python.

The average time required for each writing when the appropriate data ('COMMAND') is written from the beginning of the file to the existing appropriate file ('DRIVER') the appropriate number of times (10000 times). Measure the cost. (File open / close is not included in the time cost.)

The reason for writing data from the beginning of the file each time is because of Background described later.

background

31313 I run ev3dev on Mindstorm EV3 and send instructions to the device file when controlling the motor, but I was wondering if there is a difference in speed between C and Python only for that process.

--31313 Mindstorm EV3 (https://www.lego.com/ja-jp/mindstorms/products/mindstorms-ev3-31313 mindstorms-ev3-31313)))

• ev3dev (http://www.ev3dev.org/)

Preparation

The environment is as follows.

- macOS: 10.12.4
  software:
    - cc: Apple LLVM version 8.1.0 (clang-802.0.41)
    - python: 3.6.1

The created code is as follows. There were several Python file manipulation APIs, so I'll try three types (io.FileIO, io.BaseIO, os).

test_syncw.c

// cc -Weverything -Ofast test_syncw.c -o test_syncw

#include <stdlib.h>
#include <stdio.h>
#include <time.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

#define TEST_COUNT 10000

#define DRIVER_FILE "DRIVER"
#define COMMAND "COMMAND"

int main(__attribute__((unused)) int argc, __attribute__((unused)) char* argv[]) {
    int fd = open(DRIVER_FILE, O_WRONLY | O_SYNC);
    do {
        clock_t elapsed_time;
        do {
            clock_t start_clock = clock();
            for (int i = 0; i < TEST_COUNT; ++i) {
                lseek(fd, 0, SEEK_SET);
                write(fd, COMMAND, sizeof(COMMAND));
            }
            elapsed_time = clock() - start_clock;
        } while(0);
        printf("%lfus\n", (double) elapsed_time / TEST_COUNT * (1000000.0 / CLOCKS_PER_SEC));
    } while(0);
    close(fd);
    return EXIT_SUCCESS;
}

test_syncw.py

import os
import io
import datetime
import time


TEST_COUNT = 10000
DRIVER_FILE = 'DRIVER'
COMMAND = b'COMMAND'


def test_with_fileio():
    elapsed_time = -1
    with io.FileIO(DRIVER_FILE, mode='w') as file:
        start = time.time()
        for i in range(TEST_COUNT):
            file.seek(0)
            file.write(COMMAND)
            file.flush()
        elapsed_time = (time.time() - start) * 1000000
        file.close()
    print(f'io.FileIO: {elapsed_time / TEST_COUNT}us')


def test_with_iobase():
    elapsed_time = -1
    with open(DRIVER_FILE, mode='wb', buffering=0) as file:
        start = time.time()
        for i in range(TEST_COUNT):
            file.seek(0)
            file.write(COMMAND)
            file.flush()
        elapsed_time = (time.time() - start) * 1000000
        file.close()
    print(f'io.IOBase: {elapsed_time / TEST_COUNT}us')


def test_with_os():
    elapsed_time = -1
    fd = os.open(DRIVER_FILE, flags=(os.O_WRONLY | os.O_SYNC))
    start = time.time()
    for i in range(TEST_COUNT):
        os.lseek(fd, 0, os.SEEK_SET)
        os.write(fd, COMMAND)
    os.close(fd)
    elapsed_time = (time.time() - start) * 1000000
    print(f'os: {elapsed_time / TEST_COUNT}us')


if __name__ == '__main__':
    test_with_fileio()
    test_with_iobase()
    test_with_os()

Experiment

Try running the code. A suitable file ('DRIVER') needs to be created in advance.

$ touch DRIVER
$ cc -Weverything -Ofast test_syncw.c -o test_syncw
$ ./test_syncw
30.814900us
$ python test_syncw.py
io.FileIO: 3.4329us
io.IOBase: 3.3681us
os: 61.7254us

Python's io.FileIO and io.IOBase are not able to SYNC in the first place. Python's os is slower than C, but this is a time cost to compare.

I repeated it 5 times just in case.

Python takes about twice as long as C, but I don't think it's an absolute level of concern.