Clearinghouse Connection Management and SD Document Job Handling

[jjeroch]

Summary:

This epic encompasses the development of a feature toggle to enable or disable the clearinghouse connectivity for Self-Description (SD) functionalities and the creation of a backend job to retrigger SD document creation for customers affected by the toggle's disablement. The need for such features arises from the current portal's implementation, which mandates the clearinghouse compliance process for creating signed json.ld for the SD legal person. Due to the instability and immaturity of the job handling SD creation, a toggle is planned for deactivation to allow users to continue with registration without the SD step. Additionally, a mechanism is needed to retrospectively generate SDs for customers who registered while the toggle was disabled.

Why:

The toggle provides an immediate workaround to bypass the current mandatory clearinghouse compliance process, which is not stable enough for reliable operation. By introducing this toggle, administrators can mitigate the impact of the job's instability on registration flows, ensuring business continuity. The subsequent job retriggering capability ensures that once the system is stable or the feature is re-enabled, the necessary SD documents can be generated for compliance and completeness.

Acceptance Criteria:

• Implement a feature toggle CLEARINGHOUSE_CONNECT_DISABLED to control the clearinghouse connectivity for SD processes.
• Ensure the system can handle application processes and connector registrations without clearinghouse connectivity when the toggle is off.
• Develop a GET API endpoint to identify active companies and connectors missing SD documents, secured with the proper authentication and authorization.
• Create an administrator-triggered job to initiate SD creation for business partners who lack SD documentation once the toggle is reactivated.
• Propose UI notification methods to inform users about the status of the clearinghouse SD connectivity.
• Documentation, validation scenarios, error handling, and logging must be updated and verified for the new functionalities.

Implementation Notes:

• The feature toggle should be accessible and modifiable by system administrators, with a system restart required for changes to take effect.
• A job should be designed to trigger the SD creation process retrospectively for affected customers, ensuring proper permission checks and error handling.
• Thorough testing is required for both the feature toggle and the backend job to prevent any unexpected issues.
• The design must be flexible to accommodate future changes to the SD document creation process.

Implementation Steps:

• Integrate the feature toggle within the system, making sure it is clearly documented and can be easily managed by administrators.
• Design and implement the GET API endpoint according to the defined requirements.
• Develop the logic for the administrator-triggered job, including error handling, logging, and proper permission checks.
• Write unit and integration tests to validate the new functionalities.
• Update system documentation to include information about the feature toggle, the API endpoint, and the job's operation.
• Review the implementation with the team and incorporate feedback to ensure robustness and reliability.

---

Test Cases for Feature Toggle CLEARINGHOUSE_CONNECT_DISABLED

1. Toggle Accessibility

   • Verify that the toggle is accessible only to system administrators.
   • Ensure that changes to the toggle require a system restart to take effect.

2. Toggle Functionality

   • Confirm that enabling the toggle disables the clearinghouse connectivity (incl SD Factory).
   • Confirm that disabling the toggle enables the clearinghouse connectivity (incl SD Factory).
   • Validate that application processes and connector registrations proceed without SD step when the toggle is off.
     • retest application approval process (use existing test case flow but exclude the test case for SD cretion for legal person) - run the test till the user can access the portal completely.
     • test the connector registration (managed and owned) and run the test till the connector is active. The existing test cases are good enough, the only difference is, that the connector gets no SD document and is automatically active.

3. System Stability

   • Test that the system remains stable when the toggle is switched on and off.
   • Ensure that no SD document creation jobs are triggered while the toggle is disabled. (verify via logs)

Test Cases for GET API Endpoint

1. API Accessibility and Security

   • Verify that the endpoints are accessible only to authenticated users with proper authorization. (CX Admin)
   • Verify that non-authorized users are not able to access the endpoints to retrieve the SD Status per company (anything beside CX Admin)

2. API Functionality

   • Confirm that the API correctly identifies active companies with missing LegalPerson SD documents.
   • Confirm that the API correctly identifies active connectors of active companies with missing SD documents.
   • Verify that the API response with an empty array if no missing SD document records are existing.

3. Error Handling

   • Ensure that the endpoint responds with proper error messages for bad requests or server errors.

Test Cases for Administrator-Triggered Job

1. Job Triggering

   • Verify that only administrators can trigger the job.
     Confirm that the job can be triggered only when the toggle is reactivated.

2. Job Execution

   • Test the job's ability to create SD documents for all affected business partners.
   • Validate that the job handles errors gracefully and logs them accordingly.
     • if a selection of the retriggered companies/objects did run unsuccessfully (decline by SD Factory, decline by clearinghouse, decline by portal due to existing SD)
     • if none of the retriggered companies/objects did run successsfully

Test Cases for UI Notifications

1. Notification Design and Delivery

   • Verify that the UI notifications about the status of the clearinghouse SD connectivity are clear and informative.
   • Confirm that notifications are delivered in a timely manner to the appropriate users.

2. Notification Content

   • Check that the notification content accurately reflects the current status of the clearinghouse SD connectivity.

Additional Test Scenarios

1. Scalability Testing
   • Verify that the system can handle a large number of SD document creation jobs (e.g. 200 missing documents).

---

Linked Tickets

High Prio

• sig#796: SST Clearinghouse - Implement Feature Toggle for Clearinghouse Self-Description Connectivity portal-backend#792
• sig#796: Connector registration fails if clearinghouseConnectDisabled gets disabled after onboarding portal-backend#887

Medium Prio

• sig#796: SD / Clearinghouse toggle - enable job to retrigger portal-backend#813
• sig#796: Ensure Application Status Endpoints Reflect Clearinghouse Feature Toggle Effects portal-backend#837
• sig#796: Explore CH runtime activation for CH Compliance Activation During Approval Flow portal-backend#855
• sig#796: Enhance SelfDescription retrigger process portal-backend#903
• sig#796: Overall documentation + review #834

Improve the system/user experience

• sig#796: Application Status Endpoint Update to Reflect Feature Toggle Impact portal-frontend#948
• sig#796: Admin UI for Managing SD Document Retriggering portal-frontend#949

[jjeroch]

Late delivered.
Reason: Go-Live Task Force decision with Association

Special Request!

[jjeroch]

@MaximilianHauer & @jjeroch Please add test cases
++ Deep Dive needed! Monday: 8am

[jjeroch]

Test Cases added ✅

[jjeroch]

Open discussion: with the deactivation the SD endpoint could get deactivated (e.g. managed by policy) - security is given by the workflow validations (CH is not allowed to submit data for a application in an invalid status - process worker)
@MaximilianHauer

++ further specification of the runtime impact (eclipse-tractusx/portal-backend#855)
++ re-trigger impact for closed application process workers/application checklists to be discussed/designed and correctly implemented. (eclipse-tractusx/portal-backend#813)

Detail sessions to be planned

[evegufy]

Committers:
@evegufy
@Phil91
@ntruchsess
@oyo

[Phil91]

@MaximilianHauer I've added the ticket eclipse-tractusx/portal-backend#903 to handle the retrigger process

[MaximilianHauer]

added :

• sig#796 SD Factory Toggle Notifications  portal-backend#1029 - Needs Refinement
• sig#796 - SD Factory Toggle Notifications Frontend Adjustments  portal-frontend#1138

[MaximilianHauer]

UI-Notifications will not be implemented.
Due to the reason that the whole process is activates/deactivated via configuration we do not have an automatic point where a notification would be triggered. This mean d the operator would need to call the endpoint manually. As this would reduce the flexibility in the communication to the customer we will not implement this functionality.

Hence the responsibility of communication will be on the operation anyway therefor we should ensure their flexibility on how to communicate it.

[MaximilianHauer]

Additional issue to enhance UI design of the Retrigger Page

• sig#796: Enhance the Admin UI Design portal-frontend#1216