Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code quality changes - Shellcheck #24

Open
wants to merge 17 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions examples/arithmetic/data/generator.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# shellcheck source=/dev/null
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is something we want in examples. What does it do?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It ensures that the shellcheck tool doesn't try to follow the path, since this path does not lead to a valid file in almost all cases.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does it not lead to a valid file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I thought this was on the other example files in the root, where the source paths there reference something that probably doesn't exist. Should those files even be there since these examples are given?

The issue here is working directory related. Running shellcheck from the scripts directory without this comment won't report any issues because the file exists.
The correct way to amend this is:

# shellcheck source-path=SCRIPTDIR

Which tells shellcheck to treat the path as if it were relative from the script's directory.

. ../../../gen.sh

use_solution arithmetic.cpp
Expand Down
2 changes: 1 addition & 1 deletion examples/codforces/data/generator.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env bash

# shellcheck source=/dev/null
. ../../../gen.sh

ulimit -s unlimited
Expand Down
2 changes: 1 addition & 1 deletion examples/codforces/data/testdata.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
range: 0 100
on_reject: continue
range: 0 100
grader_flags: ignore_sample
6 changes: 3 additions & 3 deletions examples/lampswitches/data/generator.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env bash

# shellcheck source=/dev/null
. ../../../gen.sh

use_solution sl.cpp
Expand Down Expand Up @@ -74,9 +74,9 @@ wait
if which dot >/dev/null; then
echo Creating visualizations...
for F in sample/*.in secret/group1/*.in; do
N=$(head -n 1 $F)
N=$(head -n 1 "$F")
if [[ $N -lt 25 ]]; then
./visualize.py <$F | dot -T png -o ${F%.in}.png
./visualize.py <"$F" | dot -T png -o "${F%.in}".png
fi
done
fi
2 changes: 1 addition & 1 deletion examples/lampswitches/data/testdata.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
range: 0 100
on_reject: continue
range: 0 100
grader_flags: ignore_sample
1 change: 1 addition & 0 deletions examples/ninetynine/data/empty.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
#!/usr/bin/env bash
3 changes: 2 additions & 1 deletion examples/ninetynine/data/generator.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# shellcheck source=/dev/null
. ../../../gen.sh

use_solution ../../data/empty.sh # empty .ans files
Expand All @@ -25,5 +26,5 @@ tc two-4 echo two
group group3 40
limits mode=random
for A in {1..20}; do
tc random-$A echo random
tc random-"$A" echo random
Tagl marked this conversation as resolved.
Show resolved Hide resolved
done
2 changes: 1 addition & 1 deletion examples/ninetynine/data/testdata.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
range: 0 100
on_reject: continue
range: 0 100
grader_flags: ignore_sample
99 changes: 54 additions & 45 deletions gen.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# shellcheck shell=bash
# This file provides support functions for generating testdata, primarily for
# scoring problems with test groups. It has some niceties like automatically
# passing deterministic random seeds to the generator, and generating test
Expand Down Expand Up @@ -63,7 +64,7 @@ _get_ext () {

_base () {
ext=$(_get_ext "$1")
echo $(basename "$1" .$ext)
basename "$1" ".$ext"
}

_error () {
Expand Down Expand Up @@ -97,64 +98,64 @@ add_program cat "bash -c cat<\$0"
# Compile a C++ program to run.
# Arguments: file opts
compile_cpp () {
echo Compiling $1...
echo Compiling "$1"...
Tagl marked this conversation as resolved.
Show resolved Hide resolved
if [[ $2 == *"opt"* || "$(uname -s)" != Linux* ]]; then
g++ -O2 -Wall -std=gnu++20 -DGENERATING_TEST_DATA -o $(_base $1) $1
g++ -O2 -Wall -std=gnu++20 -DGENERATING_TEST_DATA -o "$(_base "$1")" "$1"
else
g++ -O2 -fsanitize=undefined -fsanitize=address -Wall -std=gnu++20 -DGENERATING_TEST_DATA -o $(_base $1) $1
g++ -O2 -fsanitize=undefined -fsanitize=address -Wall -std=gnu++20 -DGENERATING_TEST_DATA -o "$(_base "$1")" "$1"
fi
add_program $(_base $1) "./$(_base $1)"
add_cleanup $(_base $1)
add_program "$(_base "$1")" "./$(_base "$1")"
add_cleanup "$(_base "$1")"
}

# Compile a Java program to run.
# Arguments: file
compile_java () {
javac $1
if ! [ $(pwd) -ef $(dirname $1) ] # unless $(dirname $1) is the same dir as $(pwd)
javac "$1"
if ! [ "$(pwd)" -ef "$(dirname "$1")" ] # unless $(dirname $1) is the same dir as $(pwd)
then
cp $(dirname $1)/*.class .
cp "$(dirname "$1")"/*.class .
fi
add_program $(_base $1) "java $(_base $1)"
add_cleanup $(_base $1)
add_program "$(_base "$1")" "java $(_base "$1")"
add_cleanup "$(_base "$1")"
}

# Compile a Python program to run.
# Arguments: file opts
compile_py () {
if [[ $2 == *"cpython3"* ]]; then
add_program $(_base $1) "python3 $1"
add_program "$(_base "$1")" "python3 $1"
elif [[ $2 == *"cpython2"* ]]; then
add_program $(_base $1) "python2 $1"
add_program "$(_base "$1")" "python2 $1"
elif [[ $2 == *"pypy2"* ]]; then
add_program $(_base $1) "pypy $1"
add_program "$(_base "$1")" "pypy $1"
else
add_program $(_base $1) "pypy3 $1"
add_program "$(_base "$1")" "pypy3 $1"
fi
}

# Compile a bash program to run.
# Arguments: file
compile_sh () {
add_program $(_base $1) "bash $1"
add_program "$(_base "$1")" "bash $1"
}

# Compile a program
# Arguments: file opts
compile () {
ext=$(_get_ext $1)
if [ $ext == "java" ]
ext=$(_get_ext "$1")
if [ "$ext" == "java" ]
then
compile_java $1
elif [ $ext == "cpp" -o $ext == "cc" ]
compile_java "$1"
elif [ "$ext" == "cpp" ] || [ "$ext" == "cc" ]
then
compile_cpp $1 $2
elif [ $ext == "py" ]
compile_cpp "$1" "$2"
elif [ "$ext" == "py" ]
then
compile_py $1 $2
elif [ $ext == "sh" ]
compile_py "$1" "$2"
elif [ "$ext" == "sh" ]
then
compile_sh $1 $2
compile_sh "$1" "$2"
else
echo "Unsupported program: $1"
exit 1
Expand All @@ -175,7 +176,7 @@ grader_flags: ignore_sample" > testdata.yaml
# Arguments: testcase path
solve () {
local execmd=${programs[$SOLUTION]}
$execmd < $1.in > $1.ans
$execmd < "$1".in > "$1".ans
Tagl marked this conversation as resolved.
Show resolved Hide resolved
}

CURGROUP_NAME=.
Expand All @@ -186,8 +187,8 @@ CURTEST=
# Arguments: solution name
use_solution () {
path=$SOLUTION_BASE/$1
SOLUTION=$(_base $path)
compile $path $2
SOLUTION="$(_base "$path")"
compile "$path" "$2"
}


Expand Down Expand Up @@ -262,31 +263,33 @@ grader_flags: min" > "$CURGROUP_DIR/testdata.yaml"
# Arguments: parameters sent to input validator
limits () {
if [[ $USE_SCORING == 1 ]]; then
echo "input_validator_flags: $@" >> "$CURGROUP_DIR/testdata.yaml"
echo "input_validator_flags: $*" >> "$CURGROUP_DIR/testdata.yaml"
else
echo "input_validator_flags: $@" >> testdata.yaml
echo "input_validator_flags: $*" >> testdata.yaml
fi
}

output_validator_flags () {
if [[ $USE_SCORING == 1 ]]; then
echo "output_validator_flags: $@" >> "$CURGROUP_DIR/testdata.yaml"
echo "output_validator_flags: $*" >> "$CURGROUP_DIR/testdata.yaml"
else
echo "output_validator_flags: $@" >> testdata.yaml
echo "output_validator_flags: $*" >> testdata.yaml
fi
}

_check_missing_samples () {
for INF in sample/*.in; do
local name=$(basename "$INF" .in)
local name
name=$(basename "$INF" .in)
if [[ "$name" != '*' && ${cases[$name]} != sample* ]]; then
_error "missing sample or sample_manual directive for sample/$name.in"
fi
done

local any=0
for INF in sample/*.in; do
local name=$(basename "$INF" .in)
local name
name=$(basename "$INF" .in)
if [[ "$name" != '*' && ${cases[$name]} = sample* && ${latestdir[$name]} = "sample" && $REQUIRE_SAMPLE_REUSE = 1 ]]; then
_error "sample/$name must be included in some secret test group; add the line \"tc $name\""
any=1
Expand Down Expand Up @@ -336,6 +339,7 @@ _handle_err() {

_par_tc () {
set -E
# shellcheck disable=SC2064
trap "_handle_err $1" ERR
_do_tc "$@"
}
Expand Down Expand Up @@ -364,12 +368,16 @@ tc () {
PARALLELISM_ACTIVE=1
LN="cp "
fi
local path="$CURGROUP_DIR/$(_base ${cases[$name]})"
${LN}${cases[$name]}.in "$path.in"
${LN}${cases[$name]}.ans "$path.ans"
local path
path="$CURGROUP_DIR/$(_base "${cases[$name]}")"
# shellcheck disable=2086
${LN}"${cases[$name]}".in "$path.in"
# shellcheck disable=2086
${LN}"${cases[$name]}".ans "$path.ans"
for ext in {hint,desc}; do
if [ -f ${cases[$name]}.$ext ]; then
${LN}${cases[$name]}.$ext "$path.$ext"
if [ -f "${cases[$name]}.$ext" ]; then
# shellcheck disable=2086
${LN}"${cases[$name]}.$ext" "$path.$ext"
fi
done
latestdir[$name]="$CURGROUP_DIR"
Expand All @@ -389,8 +397,9 @@ tc () {
fi

# Add an index to the test case name, to enforce evaluation order.
local path="$CURGROUP_DIR/$(printf '%03d' $TC_INDEX)-$name"
let TC_INDEX++
local path
path="$CURGROUP_DIR/$(printf '%03d' $TC_INDEX)-$name"
(( TC_INDEX++ ))
CURTEST="$path"
cases[$name]="$path"
latestdir[$name]="$CURGROUP_DIR"
Expand All @@ -410,9 +419,9 @@ tc () {
if [[ $PARALLELISM_ACTIVE = 5 ]]; then
# wait after every 4 cases
wait
let PARALLELISM_ACTIVE=1
PARALLELISM_ACTIVE=1
fi
let PARALLELISM_ACTIVE++
(( PARALLELISM_ACTIVE++ ))
_par_tc "$nicename" "$name" "$path" "$program" "${@:3}" &
fi
}
Expand All @@ -423,7 +432,7 @@ tc_manual () {
if [[ $# == 1 ]]; then
name=$(_base "$1")
fi
tc $(_base "$1") cat "$1"
tc "$(_base "$1")" cat "$1"
}

# Include all testcases in another group
Expand Down Expand Up @@ -460,7 +469,7 @@ _cleanup_programs () {
rm -f "$x"
done
rm -rf __pycache__
rm -rf *.class
rm -rf ./*.class

_check_missing_samples

Expand Down
1 change: 1 addition & 0 deletions generator_example.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/usr/bin/env bash
# shellcheck source=/dev/null
. ../../testdata_tools/gen.sh

# Example test data generator, with comments to explain what each line does.
Expand Down
2 changes: 2 additions & 0 deletions generator_example_acm.sh
Tagl marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
#!/usr/bin/env bash
export USE_SCORING
USE_SCORING=0
# shellcheck source=/dev/null
. ../../testdata_tools/gen.sh

# Example test data generator for non-scoring problems.
Expand Down