Investigate randomization

[Willem3141]

No description provided.

[Willem3141]

Some thoughts, partially from my meeting with Arthur on Tuesday, partially my own.

• The basics. We need support for questions that presents in the question a randomized graph to the student.

  • Make a separate question type "Randomized GraphChecker", which contains the new features developed for randomization. The existing "GraphChecker" question type should not get these features (even though it works from the same codebase) to prevent user confusion.

• What to randomize? Generating a randomized graph-like structure in a sensible way is difficult, because each question has its own requirements on which graphs should be used. For example, some questions may need edge weights, some may not; some may need positive edge weights, or graphs with negative edge weights but no negative cycles, and so on. Thus, there would need to be a comprehensive interface for the question author to specify all of this, which is very far out of scope. (And besides the generation, one would also need to take care of reliably layouting the generated graph, and making sure that all generated graphs have about the same degree of difficulty.) Solution: offer a library of graphs, and let the question author select a subset of these from a dialog with little pictures, divided into categories. The question author can select entire categories, or individual graphs from each category if so desired.

  • Allow each type to store a randomization library (a list of categories with example graphs in them).
  • Make a script to render each example graph into a small PNG image.
  • Allow each question to store randomization options: a subset of graphs from the randomization library.
  • Create UI in the question author form to select graphs from the randomization library.

• Question setup. At runtime, GraphChecker randomly picks a graph from the subset chosen. We call this graph the input (we should find a better name for this; Moodle's built-in question type varnumeric calls these variables but that isn't clearer in my opinion). We should support a placeholder such as {{ input }} in the question text, which gets replaced at runtime by a read-only graph editor showing the input. Note that CodeRunner allows something similar by means of its Twig support.

  • Set up randomization: check how Moodle built-in randomized questions do it, and do the same thing.
  • In the question text, replace {{ input }} by a read-only graph editor displaying the input.

• Checking. Of course, when checking the student answer, the checks need access to the input (otherwise the input would be irrelevant, as the outcome would be the same no matter the input). At this moment, checks which require a graph parameter let the question author set that parameter to some constant value using a graph editor, but it should be possible to use the input instead (and that should probably be the default).

  • If the value of a graph parameter is "input", then replace it on the server side by the generated input.
  • Do the same for the tester.
  • Change the Check UI only in randomized mode to hide the graph edit button, and instead just say "input". (Rationale: it is error-prone, when editing a randomized question, to check against anything other than the input.)

• Input type and answer type. It is often useful that the answer type (the type of graph the student needs to draw) may differ from the input type (the type of graph we're randomizing for the question). For example, consider a question where a randomized PDA is given, and the student is asked to draw an equivalent DFA. But this implies that checks may receive a different type of graph parameter than the answer type. Hence, we need a mechanism to "type-check" graph parameters.

  • For each check that requires a parameter of type graph, mark it with which type of graph it needs. (Currently that is just the answer type.)
  • Ensure that the graph editor opened by the Check UI is initialized with the correct type for the check.
  • When saving the question, if the parameter is set to Input, validate that it accepts the currently set input type.
  • Optional: hide checks from the check list if they have the wrong type (this may be clearer in some cases, but it is also confusing as the check list then depends on both the input type and the answer type). We'll decide on this later.

• Preload. The question author should have the option to put the input as preload into the graph editor.

  • If in randomized mode, replace the preload graph editor by a checkbox "Preload the input" (or some more user-friendly wording).

• Text answers. As an extension of randomization, it makes sense to start thinking about questions that require text instead of a graph as the answer. For example, a question that presents a randomized DFA and asks the student for a regex describing the language accepted by the DFA.

  • Add a simple textarea UI plugin.
  • Right now, using the graph UI plugin is hardcoded in several places. Instead, consult the ui_plugin key in type.json to figure out which UI plugin to load.
  • Make sure that the Graph UI-specific settings aren't shown in the author form if the Graph UI isn't active. (Bonus points: make a generic system to handle this.)
  • Do the same for the tester.
  • As an example, add a new regex answertype.

---

Other things to think about: vertex/edge highlighting (is this a separate question type?), adding things like edge weights randomly as post-processing on the randomly picked graph, checks with graph parameters that allow more than one type simultaneously, automatically generate sample answer, teachers can make their own graph library.