#acl All:read

== Autotesting Specs ==

 * '''&runspecs''' - autotesting specification follows after this directive until end of file

   * Refer to the [[https://www.cse.unsw.edu.au/~give/Documents/Manual/give_html/node54.html|give manual]] for further documentation
   * [[https://cgi.cse.unsw.edu.au/~give/Autom/index.php| Web interface ]] to help you set up the autotesting specfication. 

 * '''&testmode''' ''mode''
 . ~-''where `mode` can be '''`automark`''', '''`notest`''' ''-~

 * '''Examples:'''
   * [[https://www.cse.unsw.edu.au/~give/Documents/Examples/9021-03s1-A.give.spec|9021-03s1-A]]
   * [[https://www.cse.unsw.edu.au/~give/Documents/Examples/9021-03s1-B.give.spec|9021-03s1-B]]
   * [[https://www.cse.unsw.edu.au/~give/Documents/Examples/9021-03s1-C.give.spec|9021-03s1-C]]
   * [[https://www.cse.unsw.edu.au/~give/Documents/Examples/9021-03s1-D.give.spec|9021-03s1-D]]
   * [[give.spec/Examples/notest]]
   * [[give.spec/Examples/special2]]

=== Pre-test phase ===
 {{{
&compile
rm -f *.c
rm -f a.out

f=grid.c
classify -v $1
dos2unix $f
list $f

options="-O -Wall -Werror"
echo "Compiling with $options ..."
echo
gcc $options $f
compile_status=$?
if [ $compile_status != 0 ]
then
    echo
    echo "** No good. Exiting..."
    exit 1
fi

cp ~/bin/trim.pl .
cp $A/gridTest[1-6] .

&end

&timelim 3
&linelim 120 20
&exec ./a.out

 }}}
 * '''&compile''' - enter the script to compile submission and to print code listings, terminate with '''&end'''
 * '''&prep''' - enter the script to run before each test, terminate with '''&end'''
 * '''&linelim''' ''m'' ''n'' - print the first m output lines and the last n output lines
 * '''&timelim''' ''s'' - cpu time limit
 * '''&exec''' ''command'' - ''$exec'' anywhere in the test suite will be substituted by the command

=== Test phase ===
 {{{
&test 1
&mark 1.5  gridTest1 no valid row col input
&invoke $exec < gridTest1 | ./trim.pl
&match < Student output and > Sample output
diff -iw $stdout $stdres
&showdiff -120w -iWt

&test 2
&mark 1.5  gridTest2 middle cell
&invoke $exec < gridTest2 | ./trim.pl
&match < Student output and > Sample output
diff -iw $stdout $stdres
&showdiff -120w -iWt
 }}}
 * '''&test''' ''n''
 * '''&mark''' ''mark'' ''description''
 * '''&invoke''' ''command''
 * '''&match''' ''description'' - put the command to match the student output and the sample output on the following line
 * '''&show''' ''file''
 * '''&showdata''' ''optional_command''
 * '''&showdiff''' ''arguments''
   * &showdiff -''n''w ''sdiff_arguments'' - translate into sdiff command and arguments
   * &showdiff -w ''diff_arguments'' - translate into diff command and arguments

 * '''&special''' ''mark'' ''description'' - enter the script to define testing that can't be defined using the standard testing directives, terminate with '''&end''' 
   * [[give.spec/Examples/special]]


=== Post-test phase ===
 {{{
&performance 10

&subjective 5
Human Marker    5
&end

&totalmark 15
 }}}
 * '''&performance''' ''total_testing_mark'' ''n'' 
 . can use &auto instead of &performance
   * ''total_testing_mark'' is the maximum mark for passing all the tests, each failed test deducts mark from this maximum
   * ''n'' is either abort_test_number or deferred_test_number depending on the testmode
     * abort_test_number: if testmode is automarked, the autotesting aborts after failing ''n'' tests. This is described very well [[https://www.cse.unsw.edu.au/~give/Documents/Manual/give_html/node64.html|here]]
     * deferred_test_number: if testmode is deferred, the first n tests are immediate autotesting, thereafter the autotesting is deferred until the human marker is present.

 * '''&subjective''' ''total_subjective_mark''
   . ''description'' TAB ''mark''
   . ''description'' TAB ''mark''
   . ...
 '''&end'''
 . Used by [[http://www.cse.unsw.edu.au/~give/Admindata/tutors/xmark.html|Xmark]] (classrun -mark).

 /!\ You must insert a ~+`TAB`+~ character between ''description'' and ''mark'' in order that Xmark parses the subjective assessment correctly.
 * '''&totalmark''' ''total_assignment_mark''

-----
CategoryGiveSpec