I'm doing Atcoder (competitive programming) as a hobby, but it's been sluggish recently. I thought it would be nice to automate the test work during the contest, I made a test automation tool with python.
Below are some of the more versatile tools than the ones I've introduced in this article.
Please also compare the above when using it.
Source code https://github.com/tks3210/autoJudge
Operating environment
>python autojudge.py [contest_name] [quest] If you specify the contest name (abc143) and the question name (abc143_a) as shown in ↑, It builds automatically, gets sample I / O data, and tests it. I also tried to create an atmosphere with the original family. The above three test cases are https://atcoder.jp/contests/abc143/tasks/abc143_a It is automatically obtained from the input example and output example of.
I can't bother Atcoder's server with my own tests, so Access is once for each problem. (The acquired test case is saved locally so that it will be referred to in the next and subsequent tests.)
When you write the wrong code, it will tell you that it is wrong.
** In case of WA (wrong answer) ** ** In case of TLE (calculation time over) ** In the current setting, if ** 2 seconds ** is exceeded until the result is output, it is treated as TLE. Even if you run in an infinite loop, it will be cut off in about 3 seconds and the next test will be done. ** For CE (compile error) ** In the first place, it doesn't matter that it doesn't compile, so it ends without testing.
If you run this test during the contest, you will not be able to get the question page without logging in. By setting the user name / password etc. with the -i option, automatic authentication and test execution during the contest are enabled.
>python autojudge.py abc143 abc143_d -i
Atcoder Username:tks_fj * Atcoder account user name
Atcoder Password:******** Atcoder account password
Src Directory(Ex. ./aaa/abc140/abc140.cpp => input "./aaa"):../../02_contest
Judging abc143/abc143_d...
testcase 1: AC
testcase 2: AC
testcase 3: AC
result: AC
** This configuration information is stored in the local configuration file (setting.conf), so you only need to do this once. ** ** (This work can also be done by directly rewriting the configuration file.)
You can also add new test cases interactively In order to support input and output of multiple lines, "quit" + Enter is used to exit.
(input:20 7 Output:Add 6)
>python autojudge.py abc143 abc143_a -a
type test input(exit by "quit")
20 7
quit
type test output(exit by "quit")
6
quit
If you test it again, it will be added to testcase4.
>python autojudge.py abc143 abc143_a
Judging abc143/abc143_a...
testcase 1: AC
testcase 2: AC
testcase 3: AC
testcase 4: AC
result: AC
** For initial settings, refer to autoJudge: Initial Settings or the next chapter. ** **
I will also introduce the design and implementation inside, which also serves as a memorandum.
The following artifacts are managed in a repository called ** autoJudge **.
It is assumed that ** autoJudge ** will be cloned in the directory where the source code (.cpp) to be tested is stored. It is recommended to operate with the following directory structure and naming convention. (But it works even if it is not ↓ (see 4. Supplementary -p option))
.
├── autoJudge * The deliverable created this time
│ ├── setting.conf * Configuration file
│ ├── autojudge.py * Program body
│ ├── testcase * List of test cases(Generated during test)
│ │ ├── abc142@abc142_a.txt
│ │ └── abc143@abc143_a.txt
│ └── design.pu * Class diagram
├── abc142 * Contest name
│ ├── abc142_a.cpp * Problem name.cpp
│ ├── abc142_b.cpp
│ ├── abc142_c.cpp
│ ├── abc142_d.cpp
│ ├── abc142_e.cpp
│ └── abc142_f.cpp
├── abc143
:
This is a configuration file used for authentication and searching for the source location.
setting.conf
username:tks_fj
password:*******
srcpath:../
It can be set with the -i option introduced in ** 2.3 Automatic authentication **, but of course it is OK to rewrite directly.
The program consists of the following two classes.
autojudge.py
def RegisterUser(self):
"""user settings(First time)"""
print("Atcoder Username:", end="")
username = input().rstrip('\r\n')
print("Atcoder Password:", end="")
password = input().rstrip('\r\n')
print("Src Directory(Ex. ./aaa/abc140/abc140.cpp => input \"./aaa\"):", end="")
srcpath = input().rstrip('\r\n')
with open(CONF_FILE, "w") as f:
f.write("username:" + username + "\n")
f.write("password:" + password + "\n")
f.write("srcpath:" + srcpath + "\n")
autojudge.py
def GetTestCases(self, test_name, islogin = False):
"""Get test cases from the specified question name and return a list"""
self.__UpdateConf()
file_name = self.contest + "@" + test_name + ".txt"
testinfo = [{"contest":self.contest, "testname":test_name}]
#To reduce the server load, do not scrape the acquisition of the same information.
if file_name in os.listdir(TESTCASES_PATH):
testcases = self.__ReadFile(file_name)
else:
testcases = self.__ScrapePage(test_name, islogin)
self.__WriteFile(file_name, testcases)
return testinfo + testcases
autojudge.py
def AddTestCases(self, test_name):
"""Add your own test case to the acquired test case"""
self.__UpdateConf()
testcase = {}
print("type test input(exit by \"quit\")")
testcase["input"] = ""
while(1):
line = input()
if (line == "quit"):
break;
testcase["input"] += line + "\n"
print("type test output(exit by \"quit\")")
testcase["output"] = ""
while(1):
line = input()
if (line == "quit"):
break;
testcase["output"] += line + "\n"
file_name = self.contest + "@" + test_name + ".txt"
if file_name in os.listdir(TESTCASES_PATH):
testcases = self.__ReadFile(file_name)
testcases.append(testcase)
self.__WriteFile(file_name, testcases)
autojudge.py
def Execute(self, srcpath = ""):
"""Run the test"""
print(YELLOW + "Judging " + self.testinfo["contest"] + "/" + self.testinfo["testname"] + "..." + COLORRESET)
if (srcpath == ""):
srcpath = self.__GetPath()
self.__Build(srcpath)
if (self.result["build"] == 0):
self.__Run()
self.__Result()
abc143@abc143_a.txt
[test case 0]
---input---
12 4
---output---
4
---fin---
[test case 1]
---input---
20 15
---output---
0
---fin---
The following modules are required. You can put it in immediately with pip, pip3, conda, etc.
pip3 install requests
pip3 install bs4, lxml
>python autojudge.py --help
usage: autojudge.py [-h] [-p PATH] [-a] [-i] contest_name question
positional arguments:
contest_name set contest name(ex. abc143)
question set question name(ex. abc143_a)
optional arguments:
-h, --help show this help message and exit
-p PATH, --path PATH set path of source code
-a, --addtest add testcase
-i, --init set configuration
When the directory structure of the source code is not "3.1 Directory structure" (For example, in the following cases)
.
├── 02_contest
│ ├── abc143
│ │ ├── abc143_a.cpp
│ │ ├── abc143_b.cpp
│ │ └── abc143_c.cpp
├── 04_autotest
│ ├── autoJudge
│ │ ├── README.md
│ │ ├── autoJudge.py
│ │ ├── autojudge.pu
│ │ ├── setting.conf
│ │ └── testcase
You can specify the source code location directly with the -p option.
>python autojudge.py abc143 abc143_a -p ../../02_contest/abc143/abc143_a.cpp
Judging abc143/abc143_a...
testcase 1: AC
testcase 2: AC
testcase 3: AC
testcase 4: AC
result: AC
Recommended Posts