Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify context not executed message #342

Closed
pdawyndt opened this issue Apr 21, 2023 · 1 comment · Fixed by #444
Closed

Clarify context not executed message #342

pdawyndt opened this issue Apr 21, 2023 · 1 comment · Fixed by #444
Labels
enhancement New feature or request

Comments

@pdawyndt
Copy link
Contributor

pdawyndt commented Apr 21, 2023

Submission: 13852943

Not clear for student what this message means, and what causes the message.

If I run the first test case locally, this is the stack trace of the exception that is raised

54:5 Uncaught RangeError: Invalid string length
    at hoofdletter (<code>:5:25)
    at groepen (<code>:13:24)

so student should get to see this message as feedback.

niknetniko added a commit that referenced this issue Apr 21, 2023
Fixes the underlying issue of #342
@pdawyndt
Copy link
Contributor Author

pdawyndt commented Apr 22, 2023

@niknetniko Thanks for solving the underlying issue of not properly reporting the "memory limit exceeded" case.

I still believe Dodona should have a better way to report test cases and contexts that have not been executed and evaluated by the judge due to some critical event (e.g. an exceeded time or memory limit in earlier contexts/test cases).

image

Using the more granular status (correct, wrong, runtime error, time limit exceeded, memory limit exceeded) instead of the binary status (correct/wrong) in rendering the feedback could improve the interpretability for the students by better explaining why something went wrong. We could use different color codes and different icons to make this differentiation, and extend the more granular status with an additional "not executed/evaluated" status.

image

Bonus: If no output (in the above case return value) is generated, we could avoid that Dodona puts effort in diffing the expected/generated output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant