Rename input_visualizer to test_case_visualizer; add *_visualizer_args

examples/problems/maximal/data/sample/test_group.yaml

@@ -0,0 +1,2 @@
+test_case_visualizer_args: [sample]
+output_visualizer_args: [sample]

examples/problems/maximal/data/secret/test_group.yaml

@@ -0,0 +1,2 @@
+test_case_visualizer_args: [secret]
+output_visualizer_args: [secret]

examples/problems/maximal/output_visualizer/output_visualizer.py

@@ -0,0 +1,16 @@
+#!/usr/bin/env python3
+import sys
+from xml.sax.saxutils import escape
+
+input_file = open(sys.argv[1]).read().strip()
+answer_file = open(sys.argv[2]).read().strip()
+feedback_dir = sys.argv[3]
+args = sys.argv[4:]
+with open(f"{feedback_dir}/judgeimage.svg", "w") as f:
+    print(
+        "<svg>"
+        + f"<text y='20'>args: {escape(' '.join(args))}</text>"
+        + f"<text y='40'>team output: {escape(input())}</text>"
+        + "</svg>",
+        file=f,
+    )

examples/problems/maximal/problem.yaml

@@ -1,8 +1,8 @@
 problem_format_version: 2023-07-draft # or `legacy` for the previous version of the Problem Package Format
 type: pass-fail
 name: # May be just a string if there's only a problem statement for one language
-  en: Sample problem
-  sv: Exempel problem
+  en: Sample Problem
+  sv: Exempel Problem
 uuid: b2792508-b645-480a-8357-da047d57a17d
 credits:
   authors:
@@ -15,13 +15,14 @@ credits:
   translators:
     sv: Sven Translator <sven@translator.com>
   acknowledgements: [Inspirational Speaker 1, Inspirational Speaker 2]
-source: My Contest 2024
-source_url: https://my.contest.com/2024
+source:
+  name: My Contest 2024
+  url: https://my.contest.com/2024
 license: cc by-sa
 rights_owner: My Contest
 limits: # There are many more limits that can be set...
   time_limit: 10
-keywords: []
+keywords: [example]
 languages: [python3, cpp]
 constants:
   example_constant: 1

examples/problems/maximal/test_case_visualizer/README.md

@@ -0,0 +1,2 @@
+This test case visualizer is intended for use with BAPCtools' `bt generate`.
+The visualizer should be invoked as `python3 input_visualizer.py <input_file_path> <answer_file_path> <...test_case_visualizer_args>` and should write a `testcase.<ext>` file.

examples/problems/maximal/test_case_visualizer/input_visualizer.py

@@ -0,0 +1,9 @@
+#!/usr/bin/env python3
+import sys
+from xml.sax.saxutils import escape
+
+input_file = open(sys.argv[1]).read().strip()
+answer_file = open(sys.argv[2]).read().strip()
+args = sys.argv[3:]
+with open("testcase.svg", "w") as f:
+    print(f"<svg><text y='20'>args: {escape(' '.join(args))}</text></svg>", file=f)

spec/2023-07-draft.md

@@ -62,32 +62,32 @@ that is, they must be of length at least 1, at most 255, consist solely of lower
 
 The following table summarizes the elements of a problem package described in this specification:
 
-File or Folder         | Required? | Described In                                  | Description
----------------------- | --------- | --------------------------------------------- | -----------
-`problem.yaml`         | Yes       | [Problem Metadata](#problem-metadata)         | Metadata about the problem (e.g., source, license, limits)
-`statement/`           | Yes       | [Problem Statements](#problem-statements)     | Problem statement files
-`attachments/`         | No        | [Attachments](#attachments)                   | Files available to teams other than the problem statement and sample test data
-`solution/`            | No        | [Solution Description](#solution-description) | Written explanations of how to solve the problem
-`data/sample/`         | No        | [Test Data](#test-data)                       | Sample test data
-`data/secret/`         | Yes       | [Test Data](#test-data)                       | Secret test data
-`data/invalid_input/`  | No        | [Invalid Test Cases](#invalid-test-cases)     | Invalid test case input for testing input validation
-`data/invalid_output/` | No        | [Invalid Test Cases](#invalid-test-cases)     | Invalid test case output for testing output validation
-`data/valid_output/`   | No        | [Valid Output](#valid-output)                 | Valid test case output for testing output validation
-`generators/`          | No        | [Generators](#generators)                     | Scripts and documentation about how test cases were automatically generated
-`include/`             | No        | [Included Files](#included-files)             | Files appended to all submitted solutions
-`submissions/`         | Yes       | [Example Submissions](#example-submissions)   | Correct and incorrect judge solutions of the problem
-`input_validators/`    | Yes       | [Input Validators](#input-validators)         | Programs that verifies correctness of the test data inputs
-`static_validator/`    | No        | [Static Validator](#static-validator)         | Custom program for judging solutions with source files as input
-`output_validator/`    | No        | [Output Validator](#output-validator)         | Custom program for judging solutions
-`input_visualizer/`    | No        | [Input Visualizer](#input-visualizer)         | Scripts and documentation about how test case illustrations were generated
-`output_visualizer/`   | No        | [Output Visualizer](#output-visualizer)       | Program to generate images illustrating submission output
+File or Folder          | Required? | Described In                                  | Description
+----------------------- | --------- | --------------------------------------------- | -----------
+`problem.yaml`          | Yes       | [Problem Metadata](#problem-metadata)         | Metadata about the problem (e.g., source, license, limits)
+`statement/`            | Yes       | [Problem Statements](#problem-statements)     | Problem statement files
+`attachments/`          | No        | [Attachments](#attachments)                   | Files available to teams other than the problem statement and sample test data
+`solution/`             | No        | [Solution Description](#solution-description) | Written explanations of how to solve the problem
+`data/sample/`          | No        | [Test Data](#test-data)                       | Sample test data
+`data/secret/`          | Yes       | [Test Data](#test-data)                       | Secret test data
+`data/invalid_input/`   | No        | [Invalid Test Cases](#invalid-test-cases)     | Invalid test case input for testing input validation
+`data/invalid_output/`  | No        | [Invalid Test Cases](#invalid-test-cases)     | Invalid test case output for testing output validation
+`data/valid_output/`    | No        | [Valid Output](#valid-output)                 | Valid test case output for testing output validation
+`generators/`           | No        | [Generators](#generators)                     | Scripts and documentation about how test cases were automatically generated
+`include/`              | No        | [Included Files](#included-files)             | Files appended to all submitted solutions
+`submissions/`          | Yes       | [Example Submissions](#example-submissions)   | Correct and incorrect judge solutions of the problem
+`input_validators/`     | Yes       | [Input Validators](#input-validators)         | Programs that verifies correctness of the test data inputs
+`static_validator/`     | No        | [Static Validator](#static-validator)         | Custom program for judging solutions with source files as input
+`output_validator/`     | No        | [Output Validator](#output-validator)         | Custom program for judging solutions
+`test_case_visualizer/` | No        | [Test Case Visualizer](#test-case-visualizer) | Scripts and documentation about how test case illustrations were generated
+`output_visualizer/`    | No        | [Output Visualizer](#output-visualizer)       | Program to generate images illustrating submission output
 
 A minimal problem package must contain `problem.yaml`, a problem statement, a secret test case, an accepted judge solution, and an input validator.
 
 ### Programs
 
 There are a number of different kinds of programs that may be provided in the problem package: 
-submissions, input validators, output validators, and output visualizers.
+submissions, input validators, output validators, test case visualizers, and output visualizers.
 All programs are always represented by a single file or directory.
 In other words, if a program consists of several files, these must be provided in a single directory.
 In the case that a program is a single file, it is treated as if a directory with the same name takes its place, which contains only that file.
@@ -569,13 +569,15 @@ In `data/sample/`, `data/secret` and each [test data group](#test-data-groups),
 
 The format of `test_group.yaml` is as follows:
 
-Key                     | Type                                | Default                                        | Comments
------------------------ | ----------------------------------- | ---------------------------------------------- | --------
-`scoring`               | Map                                 | [See Result Aggregation](#result-aggregation)  | Description of how the results of the group test cases and subgroups should be aggregated. This key is only permitted for the `secret` group and its subgroups.
-`input_validator_args`  | Sequence of strings or map of strings to sequences of strings | empty string         | See [Test Case Configuration](#test-case-configuration).
-`output_validator_args` | Sequence of strings                 | empty sequence                                 | See [Test Case Configuration](#test-case-configuration).
-`static_validation`     | Map or boolean                      | false                                          | Configuration of static validation test data node. See [Static Validator](#static-validator)
-`full_feedback`         | Boolean                             | `false` in `secret`, `true` in `sample`        | See [Test Case Configuration](#test-case-configuration).
+Key                         | Type                                | Default                                        | Comments
+--------------------------- | ----------------------------------- | ---------------------------------------------- | --------
+`scoring`                   | Map                                 | [See Result Aggregation](#result-aggregation)  | Description of how the results of the group test cases and subgroups should be aggregated. This key is only permitted for the `secret` group and its subgroups.
+`input_validator_args`      | Sequence of strings or map of strings to sequences of strings | empty sequence       | See [Test Case Configuration](#test-case-configuration).
+`output_validator_args`     | Sequence of strings                 | empty sequence                                 | See [Test Case Configuration](#test-case-configuration).
+`test_case_visualizer_args` | Sequence of strings                 | empty sequence                                 | See [Test Case Configuration](#test-case-configuration).
+`output_visualizer_args`    | Sequence of strings                 | empty sequence                                 | See [Test Case Configuration](#test-case-configuration).
+`static_validation`         | Map or boolean                      | false                                          | Configuration of static validation test data node. See [Static Validator](#static-validator)
+`full_feedback`             | Boolean                             | `false` in `secret`, `true` in `sample`        | See [Test Case Configuration](#test-case-configuration).
 
 ### Test Data Groups
 
@@ -614,18 +616,19 @@ The allowed keys are defined as follows.
 Keys are optional unless explicitly stated.
 Any unknown keys should be treated as an error.
 
-Key                   | Type                | Default
---------------------- | ------------------- | -------
-args                  | Sequence of strings | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to empty sequence
-output_validator_args | Sequence of strings | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to empty sequence
-input_validator_args  | Sequence of strings or map of strings to sequences of strings | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to empty sequence
-full_feedback         | Boolean             | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to `false` in `secret` and `true` in `sample`
-hint                  | String              | 
-description           | String              | 
+Key                         | Type                | Default
+--------------------------- | ------------------- | -------
+`args`                      | Sequence of strings | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to empty sequence
+`input_validator_args`      | Sequence of strings or map of strings to sequences of strings | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to empty sequence
+`output_validator_args`     | Sequence of strings | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to empty sequence
+`test_case_visualizer_args` | Sequence of strings | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to empty sequence
+`output_visualizer_args`    | Sequence of strings | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to empty sequence
+`full_feedback`             | Boolean             | Inherited from [`test_group.yaml`](#test-data-groups), which defaults to `false` in `secret` and `true` in `sample`
+`hint`                      | String              | 
+`description`               | String              | 
 
 For each test case:
 - `args` defines arguments passed to the submission for this test case.
-- `output_validator_args` defines arguments passed to the output validator for the test case.
 - `input_validator_args` defines arguments passed to each input validator for the test case.
   If a sequence of strings, then those are the arguments that will be passed to each input validator for this the case.
   If a map, then each key is the name of the input validator and the value is the arguments to pass to that input validator for the test case.
@@ -636,8 +639,11 @@ For each test case:
     - any error messages (stderr),
     - the illustration created by the output visualizer (if applicable),
     - the expected output.
-- A _hint_ provides feedback for solving a test case to, e.g., somebody whose submission didn't pass.
-- A _description_ conveys the purpose of a test case.
+- `output_validator_args` defines arguments passed to the output validator for the test case.
+- `test_case_visualizer_args` defines arguments passed to the test case visualizer for the test case.
+- `output_visualizer_args` defines arguments passed to the output visualizer for the test case.
+- `hint` provides feedback for solving a test case to, e.g., somebody whose submission didn't pass.
+- `description` conveys the purpose of a test case.
   It is an explanation of what aspect or edge case of the solution the input file is meant to test.
 
 ### Illustrations
@@ -1242,15 +1248,16 @@ Almost all test cases failed — are you even trying to solve the problem?
 A validator program is allowed to write any kind of debug information to its standard error pipe.
 This information may be displayed to the user upon invocation of the validator.
 
-## Input Visualizer
+## Test Case Visualizer
 
 If a tool was used to automate creating test case illustration annotations,
-it is recommended to include the input visualizer source code in the directory `input_visualizer/` along with invocation instructions in a file such as `input_visualizer/README.txt`.
+it is recommended to include the test case visualizer source code in the directory `test_case_visualizer/` along with invocation instructions in a file such as `test_case_visualizer/README.txt`.
+Tooling may make use of the `test_case_visualizer_args` key in the [test data configuration](#test-data-configuration) and [test case configuration](#test-case-configuration).
 
 ## Output Visualizer
 
 An output visualizer is an optional [program](#programs) that is run after every invocation of the output validator in order to generate images illustrating the submission output.
-A visualizer program must be an application (executable or interpreted) capable of being invoked with a command line call. It is invoked using the same arguments as the output validator.
+A visualizer program must be an application (executable or interpreted) capable of being invoked with a command line call. It is invoked using the same arguments as the output validator, except that `output_validator_args` is replaced with `output_visualizer_args`.
 It must be provided as a program (as specified [above](#programs)) in the directory `output_visualizer/`.
 
 All files written to the feedback directory by the output validator are accessible to the visualizer.