diff --git a/README.md b/README.md index 16902d79..52fb6de0 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# tcframe 1.1.0 +# tcframe 1.2.0 [![GitHub Release](https://img.shields.io/github/release/tcframe/tcframe.svg)](https://github.com/tcframe/tcframe) [![Build Status](https://img.shields.io/travis/tcframe/tcframe/master.svg)](https://travis-ci.org/tcframe/tcframe) diff --git a/docs/api-ref/api-ref.rst b/docs/api-ref/api-ref.rst index 16fc9f36..08cb8601 100644 --- a/docs/api-ref/api-ref.rst +++ b/docs/api-ref/api-ref.rst @@ -225,6 +225,38 @@ The following macros are exposed to define input/output formats: ---- +.. _api-ref_styles: + +Problem styles +************** + +.. sourcecode:: cpp + + virtual void StyleConfig() {} + +Defines the options to enable for problem styles. The following methods are exposed: + +.. cpp:function:: CustomScorer() + + Declares that the problem needs a custom scorer. + +.. cpp:function:: NoOutput() + + Declares that the problem does not need test case output files. + +See :ref:`styles` for more details. + +Example: + +.. sourcecode:: cpp + + void StyleConfig() { + CustomScorer(); + NoOutput(); + } + +---- + .. _api-ref_constraints: Constraints and subtasks @@ -524,6 +556,10 @@ Test cases generation The solution command to use for generating output files. Default: ``./solution``. +.. py:function:: --scorer= + + The custom scorer command to use for checking sample output strings in problem spec class. Default: ``./scorer``. + .. py:function:: --seed= The seed for random number generator ``rnd`` in the test spec. Default: ``0``. @@ -543,6 +579,9 @@ Local grading The solution command to grade. Default: ``./solution``. +.. py:function:: --scorer= + + The custom scorer command to use. Default: ``./scorer``. .. py:function:: --time-limit= diff --git a/docs/conf.py b/docs/conf.py index a8d28010..926aff0b 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -53,9 +53,9 @@ # built documents. # # The short X.Y version. -version = '1.1' +version = '1.2' # The full version, including alpha/beta/rc tags. -release = '1.1.0' +release = '1.2.0' # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/docs/getting-started/getting-started.rst b/docs/getting-started/getting-started.rst index e71461a5..603481f5 100644 --- a/docs/getting-started/getting-started.rst +++ b/docs/getting-started/getting-started.rst @@ -44,7 +44,7 @@ Installation Firstly, we must get **tcframe** on our system. It consists of C++ header files and a few helper scripts. -Download the latest **tcframe** `here `_. Extract the zip file somewhere on your system; for example, ``~/tcframe``. We will call this directory "**tcframe** home". Confirm that you extracted it correctly by verifying that the directory ``include`` exists directly inside **tcframe** home. +Download the latest **tcframe** `here `_. Extract the zip file somewhere on your system; for example, ``~/tcframe``. We will call this directory "**tcframe** home". Confirm that you extracted it correctly by verifying that the directory ``include`` exists directly inside **tcframe** home. Then, add the following lines to your ``~/.bashrc``. This will set the environment variable ``TCFRAME_HOME`` to our **tcframe** home directory, and make ``tcframe`` command available everywhere. diff --git a/docs/release-notes/1_2_0.rst b/docs/release-notes/1_2_0.rst new file mode 100644 index 00000000..d620c37c --- /dev/null +++ b/docs/release-notes/1_2_0.rst @@ -0,0 +1,19 @@ +.. _v1_2_0: + +1.2.0 +===== + +December 16, 2016 + +Bugfixes +-------- + +- Fixed a regression when the final verdict of local grading of problems without subtasks is always AC. + +New features +------------ + +- It is now possible to write a custom program that checks whether the contestant's output is correct (instead of just comparing it with the official output). This is called custom scorer program. +- It is now possible not to generate test case output (``.out``) files. + +See :ref:`styles` for details on how to enable the above options. diff --git a/docs/release-notes/release-notes.rst b/docs/release-notes/release-notes.rst index 4c030f90..33c6e136 100644 --- a/docs/release-notes/release-notes.rst +++ b/docs/release-notes/release-notes.rst @@ -6,6 +6,7 @@ Release Notes .. toctree:: :maxdepth: 1 + 1_2_0 1_1_0 1_0_1 1_0_0 diff --git a/docs/topic-guides/grading.rst b/docs/topic-guides/grading.rst index 083f6c60..f9ff741b 100644 --- a/docs/topic-guides/grading.rst +++ b/docs/topic-guides/grading.rst @@ -39,6 +39,9 @@ Runtime Error (RTE) Time Limit Exceeded (TLE) The solution did not stop within the time limit, if specified. +Internal Error (ERR) + Custom scorer (if any) crashed or did not give valid verdict. + The verdict of each subtask will be also shown. The verdict of a subtask is the worst verdict of all verdicts of test cases that are assigned to it. Here, RTE is worse than WA, and WA is worse than AC. Here is a sample output of a local grading for problems with subtasks. diff --git a/docs/topic-guides/styles.rst b/docs/topic-guides/styles.rst new file mode 100644 index 00000000..e2966927 --- /dev/null +++ b/docs/topic-guides/styles.rst @@ -0,0 +1,84 @@ +.. _styles: + +Problem Styles +============== + +Currently, **tcframe** supports only **batch**-style problems, where the solution is expected to read the test cases from the standard input and write the answers to the standard output. There are some configurable options to this behavior, which can be specified in the ``StyleConfig()`` method of the problem spec class. + +.. sourcecode:: cpp + + void StyleConfig() { + // option specifications + } + +The available options are as follows. + +Custom scorer +------------- + +Enabled by calling ``CustomScorer()`` inside ``StyleConfig()``. + +A scorer is a program which decides the verdict of a test case. By default, the scorer is the simple ``diff`` program. If custom scorer is enabled, then you must provide the custom scorer program. + +The custom scorer will receive the following arguments: + +- argv[1]: test case input filename +- argv[2]: test case output filename +- argv[3]: contestant's produced output filename + +The custom scorer must print the test case verdict to the standard output, which is a line consisting of either: + +- ``AC``: indicates that the contestant's output is correct +- ``WA``: indicates that the contestant's output is incorrect + +The custom scorer must be compiled prior test cases generation/local grading, and the execution command should be passed to the runner program as the ``--scorer`` option. For example: + +:: + + ./runner grade --solution=./solution_alt --scorer=./my_custom_scorer + +The default scorer command is ``./scorer`` if not specified. + +Here is an example custom scorer which gives AC if the contestant's output differs not more than 1e-9 with the official output. + +.. sourcecode:: cpp + + #include + using namespace std; + + int wa() { + cout << "WA" << endl; + return 0; + } + + int ac() { + cout << "AC" << endl; + return 0; + } + + int main(int argc, char* argv[]) { + ifstream tc_in(argv[1]); + ifstream tc_out(argv[2]); + ifstream con_out(argv[3]); + + double tc_ans; + tc_out >> tc_ans; + + double con_ans; + if (!(con_out >> con_ans)) { + return wa(); + } + + if (abs(tc_ans - con_ans) < 1e-9) { + return ac(); + } else { + return wa(); + } + } + +No output +--------- + +Enabled by calling ``NoOutput()`` inside ``StyleConfig()``. + +Sometimes, a problem does not need test case output files (``.out``) because the scoring is done by a custom score alone. If this option is enabled, then ``.out`` files will not be generated, and it is not allowed to specify ``Output()`` in sample test cases. diff --git a/docs/topic-guides/test-cases.rst b/docs/topic-guides/test-cases.rst index 5780af1f..9b214581 100644 --- a/docs/topic-guides/test-cases.rst +++ b/docs/topic-guides/test-cases.rst @@ -134,6 +134,8 @@ can be translated to: The ``Output()`` part of a sample test case definition is optional, and if not present, the solution will be run to produce the output. However, it is only for easier migration from **tcframe** 0.x. You should always specify both input and output, so that you are sure you are typing the output correctly in the problem statement by only looking at the spec file (no need to check with the actual produced output file). + Of course, if ``NoOutput()`` is enabled (see :ref:`styles`), then ``Output()`` is not allowed to be specified. + :ref:`If your problem has subtasks `, you also need to assign each sample test case to a set of subtasks, by calling ``Subtasks()`` at the beginning of each ``SampleTestCaseX()`` with the set of subtask numbers, as follows. .. sourcecode:: cpp diff --git a/docs/topic-guides/topic-guides.rst b/docs/topic-guides/topic-guides.rst index 91dae889..7192abc9 100644 --- a/docs/topic-guides/topic-guides.rst +++ b/docs/topic-guides/topic-guides.rst @@ -12,6 +12,7 @@ In this guide, we will learn each aspect of **tcframe** more thoroughly. Each se Spec and runner I/O variables I/O formats + Problem styles Constraints Subtasks Test cases diff --git a/scripts/tcframe b/scripts/tcframe index 1957d0ae..1a3332d4 100755 --- a/scripts/tcframe +++ b/scripts/tcframe @@ -21,7 +21,7 @@ build() { } version() { - echo "tcframe 1.1.0" + echo "tcframe 1.2.0" } if [ -z "$TCFRAME_HOME" ]; then