I/O error when requesting catalog from other connector via EDC CE UI

[tmberthold]

We received a report that the following error message may occur when doing a catalog request using the sovity EDC CE UI:
An I/O error has occurred while writing a response message entity to the container output stream.

Unfortunately, it is not known how many DataOffers this Catalog Request should return, which run into this error, but this could be important for the following initial analysis.

As Paul wrote back in time in the context of the old IDS-protocol quoting such an error message:
we might still run into I/O errors when assembling the response takes too long, or the body size is too large
eclipse-edc/Connector#1515 (comment)

With regard to the Catalog Request and the different existing Postman Collections, the maximum of DataOffers to be returned by a Catalog Request is always limited by a QuerySpec-limit in the request body, when directly interacting with the Management-API instead of using a UI.

• Tractus-X EDC Postman Collection sets the limit to 100:
  https://github.com/eclipse-tractusx/tractusx-edc/blob/2498215fc8916de003667774c40f02342fdd47bd/docs/development/postman/collection.json#L338

• Our sovity EDC CE Postman Collection sets the limit to 10:
  https://github.com/sovity/edc-extensions/blob/78f939e1f163aec422de892e83cf940ca9a3a5d2/docs/postman_collection.json#L487

Our API-wrapper, on the other hand, which is relevant for the sovity EDC UI, does not use a limit, it rather sets it to the maximum:
https://github.com/sovity/edc-extensions/blob/78f939e1f163aec422de892e83cf940ca9a3a5d2/utils/catalog-parser/src/main/java/de/sovity/edc/utils/catalog/DspCatalogService.java#L46C79-L46C95

Is it possible that setting the QuerySpec to maximum instead of limiting the result can cause this I/O error under certain circumstances, be it because the providing EDC takes too long to fetch the DataOffers for the Catalog-Request-Response (for whatever reasons) or the response-body itself becoming too large?

A possible solution could be pagination within our UI for catalog requests of a single connector with adjustable number of offers to be displayed, so that limits and offsets need to be used, instead of requesting all DataOffer at once.

[jkbquabeck]

@tmberthold There was a second message containing their steps to reproduce:
https://mobility-dataspace.online/?wpsc-section=ticket-list&ticket-id=102

Hi Jakob,

We might have some steps to reproduce, or at least we noticed this pattern for a few times before sending this reply:

Create Asset
Create Contract Definition for the same asset
Check Asset is available on catalog browser
Delete Contract Definition
wait for 2hr (approx)
Check catalog browser for connector
It seems that it takes about 2 hours before encountering the issue after performing steps 1-4.

I hope this will help you reproduce the issue.

[tmberthold]

@tmberthold There was a second message containing their steps to reproduce
[...]
I hope this will help you reproduce the issue.

Tried, but I couldn't recreate the error with that steps with my local deployment.

[tmberthold]

I would close this since I'm quite positive that this is related to the catalog endpoint thingy.