Text exercises: Fix an edge case with discarded Athena feedback suggestions for tutors

[maximiliansoelch]

Checklist

General

• This is a small issue that I tested locally and was confirmed by another developer on a test server.
• Language: I followed the guidelines for inclusive, diversity-sensitive, and appreciative language.
• I chose a title conforming to the naming conventions for pull requests.

Client

• Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data (e.g. using paging).
• I strictly followed the client coding and design guidelines.
• I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.

Motivation and Context

Currently, there is the edge case when two feedback items overlap and have the same end_index, but a different start index, which they reference.
This leads to issues as tutors do not see all of the feedback when they assess text exercises that use feedback suggestions from Athena. In case Athena is not used, the issue does not occur.
The overlapping feedback element is not shown in the client, and therefore also the score displayed to the tutor during the assessment is wrong (as it only incorporates the visible feedback).
The overall score on Artemis is still correct, and the feedback items persist, but the overlapping text blocks not.
However, it is not clear to the tutors that they may give the students unintendedly more points than they want.

Example:

Only three feedback items are displayed to the tutor during the assessment:

However, Athena generated four feedback suggestions (where two of them overlap):

Also, in the Artemis database four feedback items are saved:

Description

This PR adds a new if condition check to the addAutomaticTextBlockRef() function that processes incoming feedback suggestions from Athena.
The check is called if the existing block is starting after the new block, but both ending at the same index.
In addition, the corresponding test has been extended by two new suitable test case instantiations.

Steps for Testing

Prerequisites:

• 1 Instructor
• 1 Student
• 1 Text Exercise with Athena Feedback Suggestions enabled

1. Participate in the exercise as a student
2. After the exercise due date, assess the submission as the instructor
3. When the assessment editor loads, make sure that Athena provides feedback suggestions
4. Verify that the number of shown suggestions is the same as the number of generated suggestions (you can see the logs of Athena TS1 here: https://athena-test1.ase.cit.tum.de/logs | credentials are here: https://confluence.ase.in.tum.de/spaces/ArTEMiS/pages/212054239/Deployments+Test+Server#Deployments%26TestServer-APISecrets%26LogCredentials)

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.

Review Progress

Performance Review

• I (as a reviewer) confirm that the client changes (in particular related to REST calls and UI responsiveness) are implemented with a very good performance even for very large courses with more than 2000 students.

Code Review

• Code Review 1
• Code Review 2

Manual Tests

• Test 1
• Test 2

Test Coverage

Class/File	Line Coverage	Confirmation (assert/expect)
text-submission-assessment.component.ts	91%	✅

Summary by CodeRabbit

• Bug Fixes

  • Improved handling of text block references to prevent overlapping blocks
  • Enhanced logic for managing text block end indices

• Tests

  • Added test cases to verify correct behavior with overlapping text blocks
  • Expanded test coverage for edge cases in text submission assessment

[coderabbitai]

Walkthrough

This pull request modifies the text submission assessment component to improve handling of text block references, specifically addressing scenarios where text blocks have matching end indices. The changes enhance the logic for adding automatic text block references, ensuring more precise management of overlapping text blocks. A corresponding test suite has been updated to validate the new behavior, covering edge cases involving text block overlap and feedback suggestion management.

Changes

File	Change Summary
src/main/webapp/app/exercises/text/assess/text-submission-assessment.component.ts	Modified addAutomaticTextBlockRef method to handle scenarios with matching end indices, adjusting text block reference addition logic
src/test/javascript/spec/component/text-submission-assessment/text-submission-assessment.component.spec.ts	Added new test cases to verify handling of overlapping text blocks with shared end indices

Sequence Diagram

sequenceDiagram
    participant Component as TextSubmissionAssessmentComponent
    participant Method as addAutomaticTextBlockRef
    
    Component->>Method: Add text block reference
    Method-->>Method: Check end index overlap
    alt End indices match
        Method->>Method: Adjust block reference
    end
    Method-->>Component: Return updated text block references

Loading

Possibly related PRs

• Assessment: Do not display automatic as additional feedback in the online code editor #10120: Refactors feedback utility functions to separate manual and automatic feedback
• Modeling exercises: Fix manual feedback results not being shown #10132: Addresses manual feedback display issues

Suggested labels

bugfix, ready to merge, assessment, exercise

Suggested reviewers

• SimonEntholzer
• krusche
• b-fein
• ahmetsenturk
• Feras797

---

📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 0646b25 and ff0d46d.

📒 Files selected for processing (2)

• src/main/webapp/app/exercises/text/assess/text-submission-assessment.component.ts (1 hunks)
• src/test/javascript/spec/component/text-submission-assessment/text-submission-assessment.component.spec.ts (1 hunks)

🧰 Additional context used

📓 Path-based instructions (2)

src/main/webapp/app/exercises/text/assess/text-submission-assessment.component.ts (1)

Pattern src/main/webapp/**/*.ts: angular_style:https://angular.io/guide/styleguide;methods_in_html:false;lazy_loading:true;code_reuse:true;tests:meaningful;types:PascalCase;enums:PascalCase;funcs:camelCase;props:camelCase;no_priv_prefix:true;strings:single_quotes;localize:true;btns:functionality;links:navigation;icons_text:newline;labels:associate;code_style:arrow_funcs,curly_braces,open_braces_same_line,indent_4;memory_leak_prevention:true;routes:naming_schema;chart_framework:ngx-charts;responsive_layout:true

src/test/javascript/spec/component/text-submission-assessment/text-submission-assessment.component.spec.ts (1)

Pattern src/test/javascript/spec/**/*.ts: jest: true; mock: NgMocks; bad_practices: avoid_full_module_import; perf_improvements: mock_irrelevant_deps; service_testing: mock_http_for_logic; no_schema: avoid_NO_ERRORS_SCHEMA; expectation_specificity: true; solutions: {boolean: toBeTrue/False, reference: toBe, existence: toBeNull/NotNull, undefined: toBeUndefined, class_obj: toContainEntries/toEqual, spy_calls: {not_called: not.toHaveBeenCalled, once: toHaveBeenCalledOnce, with_value: toHaveBeenCalledWith|toHaveBeenCalledExactlyOnceWith}}

⏰ Context from checks skipped due to timeout of 90000ms (1)

• GitHub Check: Codacy Static Code Analysis

🔇 Additional comments (2)

src/main/webapp/app/exercises/text/assess/text-submission-assessment.component.ts (1)

295-301: LGTM! Good handling of overlapping text blocks.

The added code elegantly handles the edge case where two text blocks share the same end index but have different start indices. The solution to adjust the new text block's end index prevents overlapping blocks while ensuring all feedback is preserved, which directly addresses the issue of tutors not seeing all feedback during assessments.

src/test/javascript/spec/component/text-submission-assessment/text-submission-assessment.component.spec.ts (1)

562-583: LGTM! Comprehensive test coverage for the edge case.

The added test cases thoroughly verify the handling of text blocks with the same end index:

1. First case tests the basic scenario with blocks [377, 719] and [453, 719].
2. Second case ensures the solution works regardless of block order.

These tests will help prevent regressions in the overlapping feedback handling.

Finishing Touches

• 📝 Generate Docstrings (Beta)

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Other keywords and placeholders

• Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai or @coderabbitai title anywhere in the PR title to generate the title automatically.

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[LeonWehrhahn]

Code

[FelixTJDietrich]

Code looks good and makes sense, it is rather hard to test but still works as expected on TS1

[HawKhiem]

Tested on TS1. Works as described