Autotesting Specs
&runspecs - autotesting specification follows after this directive until end of file
Refer to the give manual for further documentation
Web interface to help you set up the autotesting specfication.
&testmode mode
where mode can be automark, notest
Examples:
Pre-test phase
&compile rm -f *.c rm -f a.out f=grid.c classify -v $1 dos2unix $f list $f options="-O -Wall -Werror" echo "Compiling with $options ..." echo gcc $options $f compile_status=$? if [ $compile_status != 0 ] then echo echo "** No good. Exiting..." exit 1 fi cp ~/bin/trim.pl . cp $A/gridTest[1-6] . &end &timelim 3 &linelim 120 20 &exec ./a.out
&compile - enter the script to compile submission and to print code listings, terminate with &end
&prep - enter the script to run before each test, terminate with &end
&linelim m n - print the first m output lines and the last n output lines
&timelim s - cpu time limit
&exec command - $exec anywhere in the test suite will be substituted by the command
Test phase
&test 1 &mark 1.5 gridTest1 no valid row col input &invoke $exec < gridTest1 | ./trim.pl &match < Student output and > Sample output diff -iw $stdout $stdres &showdiff -120w -iWt &test 2 &mark 1.5 gridTest2 middle cell &invoke $exec < gridTest2 | ./trim.pl &match < Student output and > Sample output diff -iw $stdout $stdres &showdiff -120w -iWt
&test n
&mark mark description
&invoke command
&match description - put the command to match the student output and the sample output on the following line
&show file
&showdata optional_command
&showdiff arguments
&showdiff -nw sdiff_arguments - translate into sdiff command and arguments
&showdiff -w diff_arguments - translate into diff command and arguments
&special mark description - enter the script to define testing that can't be defined using the standard testing directives, terminate with &end
Post-test phase
&performance 10 &subjective 5 Human Marker 5 &end &totalmark 15
&performance total_testing_mark n
can use &auto instead of &performance
total_testing_mark is the maximum mark for passing all the tests, each failed test deducts mark from this maximum
n is either abort_test_number or deferred_test_number depending on the testmode
abort_test_number: if testmode is automarked, the autotesting aborts after failing n tests. This is described very well here
- deferred_test_number: if testmode is deferred, the first n tests are immediate autotesting, thereafter the autotesting is deferred until the human marker is present.
&subjective total_subjective_mark
description TAB mark
description TAB mark
- ...
&end
Used by Xmark (classrun -mark).
You must insert a TAB character between description and mark in order that Xmark parses the subjective assessment correctly.
&totalmark total_assignment_mark