Feature request, show green checkmarks in feedback when partial feedback is hidden but answers are all correct.

[somiaj]

I have a problem with $showPartialCorrectAnswers=0 which contains four drop down menus. I expected to see green checkmarks if everything was correct, but instead these were also hidden. Here is the result, with the submit earning 100%.

[somiaj]

Here is what I came up with to show all green checkmarks when all parts are correct, but to keep the feedback hidden when $showPartialCorrectAnswers is false.

diff --git a/macros/PG.pl b/macros/PG.pl
index e3f4a802..dfef13e4 100644
--- a/macros/PG.pl
+++ b/macros/PG.pl
@@ -1035,6 +1035,13 @@ sub ENDDOCUMENT {
                                && !$rh_envir->{showAttemptPreviews}
                                && !$rh_envir->{showMessages};
 
+                       # Compute $numCorrect here, since we need to know if everything is correct when deciding
+                       # to display correct feedback when $showPartialCorrectAnswers is false.
+                       for my $answerLabel (@answerNames) {
+                               my $answerScore = $PG->{PG_ANSWERS_HASH}{$answerLabel}{ans_eval}{rh_ans}{score} // 0;
+                               ++$numCorrect if $answerScore >= 1;
+                       }
+
                        for my $answerLabel (@answerNames) {
                                my $response_obj = $PG->{PG_ANSWERS_HASH}{$answerLabel}->response_obj;
                                my $ansHash      = $PG->{PG_ANSWERS_HASH}{$answerLabel}{ans_eval}{rh_ans};
@@ -1080,8 +1087,13 @@ sub ENDDOCUMENT {
                                        push(@{ $options{feedbackElements} }, @$elements);
                                }
 
-                               if (($rh_envir->{showAttemptResults} && $PG->{flags}{showPartialCorrectAnswers})
-                                       || $rh_envir->{forceShowAttemptResults})
+                               if (
+                                       (
+                                               $rh_envir->{showAttemptResults}
+                                               && ($PG->{flags}{showPartialCorrectAnswers} || $numCorrect == @answerNames)
+                                       )
+                                       || $rh_envir->{forceShowAttemptResults}
+                                       )
                                {
                                        if ($showCorrectOnly) {
                                                $options{resultClass} = 'correct-only';
@@ -1107,7 +1119,6 @@ sub ENDDOCUMENT {
                                # Update the counts.  This should be after the custom feedback_options call as that method can change
                                # some of the options.  (The draggableProof.pl macro changes the answerGiven option, and the
                                # PGessaymacros.pl macro changes the manuallyGraded and needsGrading options.)
-                               ++$numCorrect        if $answerScore >= 1;
                                ++$numManuallyGraded if $options{manuallyGraded};
                                $needsGrading = 1    if $options{needsGrading};
                                ++$numBlank unless $options{manuallyGraded} || $options{answerGiven} || $answerScore >= 1;

[drgrice1]

Note that how you are describing what happens is actually how it was designed to work. With the previous attempts table that is how it worked, and I didn't change the behavior with the new feedback. No per problem feedback is ever given when you set $showPartialCorrectAnswers = 0. Only the message at the top of the problem gives more information telling you how many (in this case all) are correct.

[somiaj]

@drgrice1 I thought the older feedback table would show green in all the rows when they were all correct, and the feedback was only hidden when the status for the full problem was less than 1. I don't have a 2.18 server I can quickly test for this though.

[drgrice1]

No it didn't. It wouldn't say correct or incorrect.

Now that I look at your screenshot closer I see that something is incorrect with your claim. If you submitted the answer it should not show the red incorrect or green correct buttons. It should show the black info buttons as in the following screenshot:

That is what a student working the problem will also see.

Also, if you click "Check Answers" as an instructor (when working the problem in a set or in the problem editor) it will show the red incorrect and green correct feedback buttons.

[somiaj]

I got a 2.18 server up just to remind myself how it worked. I see, it would only show a green message that everything was correct below the table, but still no feedback in the table. I was just use to seeing this more from an instructor side, and confused myself that everything being correct was part of the table. I do notice that the correct message is a bit brighter in 2.18 with the old feedback table than the success message is now. Not an issue, just didn't stand out as much.

The reason it is orange is due to my BSU theme using orange for secondary buttons in other places. I'll go look into that, thanks. The orange color is probably what threw me off.

[somiaj]

Anyways, I'll close this unless others think this could be a feature request.

[drgrice1]

By the way, here is a screen shot of the attempts table and what it did here from 2.17.

I think that you should leave this issue open. I wasn't saying that I liked the old behavior. I am just saying that is the way it worked, and was designed to work. We should discuss this, because I think there are other issues with the $showPartialCorrectAnswers = 0 behavior as well as this one.

[Alex-Jordan]

It has come up that the $showPartialCorrectAnswers = 0 behavior when the user has submitted all correct answers is problematic for a problem used in a PreTeXt project. When the user gets everything right, there is no indication at all of their success.

A related issue is when the user gets some answers correct, but not all, it is not clear to the user what is happening. They will see the preview feedback buttons, and not understand why their answers have not been assessed as correct or incorrect.

What I am thinking right now is that ideally,

• if all submitted answers are correct, the feedback buttons are the "correct" class buttons, as Jaimos's patch would do.
• otherwise, with showPartialCorrectAnswers = 0, the feedback button should come with a message like "some or all of your answers are incorrect"

[drgrice1]

The issue for PreTeXt is that the PreTeXt javascript does not show the message that is shown in WeBWorK that states the general assessment as to how many answers are correct or incorrect. That summary message is in the data that is returned and that the javascript has access to. So the PreTeXt javascript should be displaying that message.

I don't think that it is a good idea to show that for all feedback buttons. One reason for this is the extra processing that this would require. Currently the answers are processed and the individual feedback is generated. Once that is done then the data has been gathered as to how many answers are correct or not, and then the summary is generated from that data.

To put the message in each feedback button, you would have to massively restructure things so that all of the answers are processed first, then a second loop would be needed to generate the feedback buttons and summary. It would be much less efficient, and cumbersome.

I think that it would be nice change the behavior to show all green when a user gets all answers correct. I also don't like that behavior. However, this also would require multiple loops to implement.

[drgrice1]

Note that it is possible to handle the all or none correct behavior without "relooping". The problem state that is computed when answers are processed could be passed into the content post processor callbacks, and then the full problem score could be used from there.

However, that doesn't work for showing the number or correct or incorrect answers in all feedback buttons because even the problem state doesn't have that information. Although, I think that doing this is a bad idea not only because of the inefficiency, but because I don't think that information belongs in all feedback buttons. I think the PreTeXt javascript just needs to be updated to show the summary somewhere.

[Alex-Jordan]

OK, with the PTX js, I can look for whether or not the flag showPartialCorrectAnswers was set, and was set to 0. Then if the overall score is 1, I can put the "correct" class on all the feedback buttons. And if the overall score is less than 1, I can find something helpful to do.

[somiaj]

I would still like feedback updated in the case $showPartialCorrectAnswers = 0 but the full problem is correct. I think it sounds like at least this part is agreed upon, so maybe you only need to update the PTX js to deal with the case the full problem is not correct to inform them about the percent correct similar to what webwork2 already informs students of.

[somiaj]

It sounds like @drgrice1 might have a better solution to this than my attempt that doesn't require looping over all the inputs twice.

[drgrice1]

I put in a pull request with the change that I implemented.

[pstaabp]

Fixed in #1187