NEWAY

愿识乾坤大，仍怜草木青

0%

Autograder简单的中文教程

Posted on Edited on In Valine:

前段时间想给一个作业写一个autograder，翻遍网上发现没有中文教程，遂作此文。（也是给自己留个备份 方便之后直接看）

首先准备一个文件夹 autograder_folder。注意这个文件夹里的内容会被上传到服务器中的/autograder/source

第一步

创建setup.sh，这个是autograder server第一个执行的脚本，一般来说，你可以用来给autograder server装环境。例子：

1
2
3
4
5
6
7
8
9
10
11
#!/usr/bin/env bash

apt-get install -y python3 python3-pip python3-dev

pip3 install -r /autograder/source/requirements.txt # /autograder/source 就是你创建的autograder_folder

cd /autograder/source/cframe/

python3 setup.py install

cd /

第二步

创建一个脚本run_autograder， 其中的内容大概就是

  1. 复制学生的submission（在/autograder/submission 里面）到某个你需要的地方
  2. 跑学生的submission
  3. grading

例子

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#!/usr/bin/env bash

# Set up autograder files

find /autograder/submission -name "imply.py" -exec cp {} /autograder/source/imply.py \; # 这里就是把文件转移到source文件夹

cd /autograder/source
mkdir logs
# mv project2/imply.py imply.py
BENCH=/autograder/source/bench
echo "running user tests"
timeout 10s python3 imply.py $BENCH/c7.bench $BENCH/c7.1.impl  result/c7.1
echo "finished running user"
echo "------------------------"

echo "running scoring system"
cd /autograder/source/

python3 run_all_submission.py。#

echo "finished running scoring system"
echo "------------------------"


python3 run_tests.py

Grading?

grade需要两个文件，run_tests.py

1
2
3
4
5
6
7
import unittest
from gradescope_utils.autograder_utils.json_test_runner import JSONTestRunner

if __name__ == '__main__':
    suite = unittest.defaultTestLoader.discover('tests') # 这里就是看tests文件夹里面所有的test.py
    with open('/autograder/results/results.json', 'w') as f:
        JSONTestRunner(visibility='visible', stream=f,stdout_visibility = 'visible').run(suite)

和 tests/test.py
下面是我的例子，主要看下每个func 的前缀，很好理解。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
import unittest
import random
from gradescope_utils.autograder_utils.decorators import partial_credit, visibility
import os
import re

class TestPartialCredit(unittest.TestCase):
    def setUp(self):
        pass

    @visibility('after_due_date')
    @partial_credit(70.0)
    def test_fanout_free(self, set_score=None):
            with open(f"./result_sort/log", 'r') as IN:
                fanout_free = 0
                fanout = 0
                xlist = 0
                unique = 0
                for line in IN:
                    line = line.strip()
                    score = 0
                    if ":" in line:
                        score = float(line.split(":")[1].strip())
                    if re.match(r'^c7\.(\d)', line):
                        fanout_free += score
                    elif re.match(r'^nand\.(\d)', line):
                        fanout_free += score
                    elif re.match(r'^or\.(\d)', line):
                        fanout_free += score
                    else:
                        continue
                    print(f"line={line}, this benchmark's score={score}, fanout_free cumulative score={fanout_free}")
                    
                fanout_free = fanout_free * 10
                """Sets partial credit"""
                set_score(fanout_free + 10)


    @partial_credit(30.0)
    @visibility('after_due_date')
    def test_fanout(self, set_score=None):
            with open(f"./result_sort/log", 'r') as IN:
                fanout_free = 0
                fanout = 0
                xlist = 0
                unique = 0
                for line in IN:
                    line = line.strip()
                    score = 0
                    if ":" in line:
                        score = float(line.split(":")[1].strip())
                    if re.match(r'^c7\.(\d)', line):
                        fanout_free += score
                    elif re.match(r'^nand\.(\d)', line):
                        fanout_free += score
                    elif re.match(r'^or\.(\d)', line):
                        fanout_free += score
                    elif "xlist" in line:
                        xlist += score
                        xlist = xlist * 10
                    elif "unique" in line:
                        unique += score
                        unique = unique * 10
                    else:
                        fanout += score
                        print(f"line={line}, this benchmark's score={score}, fanout cumulative score={fanout}")
                    

                fanout = fanout * 5
                set_score(fanout)
    
    @partial_credit(20.0)
    @visibility('after_due_date')
    def test_extra(self, set_score=None):
            with open(f"./result_sort/log", 'r') as IN:
                fanout_free = 0
                fanout = 0
                xlist = 0
                unique = 0
                for line in IN:
                    line = line.strip()
                    score = 0
                    if ":" in line:
                        score = float(line.split(":")[1].strip())
                    if re.match(r'^c7\.(\d)', line):
                        fanout_free += score
                    elif re.match(r'^nand\.(\d)', line):
                        fanout_free += score
                    elif re.match(r'^or\.(\d)', line):
                        fanout_free += score
                    elif "xlist" in line:
                        xlist += score
                        xlist = xlist * 10
                        print(f"line={line}, this benchmark's score={score}, xlist cumulative score={xlist}, unique cumulative score={unique}")
                    elif "unique" in line:
                        unique += score
                        unique = unique * 10
                        print(f"line={line}, this benchmark's score={score}, xlist cumulative score={xlist}, unique cumulative score={unique}")
                    else:
                        fanout += score
                extra = xlist + unique
            set_score(extra)
    
    @partial_credit(10.0)
    @visibility('after_due_date')
    def test_clarity(self, set_score=None):
        set_score(10)

最后

注意不要压缩autograder_folder 这个文件夹，
要选中autograder_folder里面的所有文件然后压缩