Configuring experimental BOM Processing V2 fails

[muellerst-hg]

Current Behavior

The experimental configuration "BOM Processing V2" flag cannot be changed. The Web UI responses with a green flash message "Configuration saved", but the property stays unchanged. The response body contains: "The config property could not be found."

Version: 4.12.0-SNAPSHOT
Built-on: 2024-07-15T08:35:18Z

Steps to Reproduce

Web UI

1. Go to Administration->Configuration->Experimental
2. Set "BOM Processing V2" to enabled and click "Update" button
3. Reload page to find the setting unchanged

API:

curl -H "Content-Type: application/json" -X POST -H "X-Api-Key: **********" "https://**********/api/v1/configProperty/aggregate" --data '[{ "groupName": "experimental", "propertyName": "bom.processing.task.v2.enabled", "propertyValue": "true" }]'
["The config property could not be found."]

Expected Behavior

I expect the property to be changed.
I would also expect that a GET request to the API endpoint /api/v1/configProperty would contain an item with "groupName": "experimental" and "propertyName": "bom.processing.task.v2.enabled". However, it does not.

Dependency-Track Version

4.12.0-SNAPSHOT

Dependency-Track Distribution

Container Image

Database Server

PostgreSQL

Database Server Version

13

Browser

Mozilla Firefox

Checklist

• I have read and understand the contributing guidelines
• I have checked the existing issues for whether this defect was already reported

[nscuro]

Thanks for reporting, I removed the flag for v4.12 (#3722) but apparently forgot to do the same for the frontend. BOM processing V2 is the new default for v4.12 going forward.

[muellerst-hg]

Well, we have identified an issue with bom processing v2 in 4.11 which does not appear using the "old" processor. I was just about to test this with snapshot.
We're feeding the processor with a larger bom file (11MB) and ended up with an exception:

  using statement "INSERT INTO "PROJECT_METADATA" ("AUTHORS","PROJECT_ID","SUPPLIER") VALUES (?,?,?)" failed :
  ERROR: duplicate key value violates unique constraint "PROJECT_METADATA_PROJECT_ID_IDX"
  Detail: Key ("PROJECT_ID")=(1964) already exists. [bomSerialNumber=d9fff84d-e061-415f-b124-ab42babd3967, bomFormat=CycloneDX, bomUploadToken=80b7876e-dc1d-4f35-be02-0d0b309d4aa2, projectName=linux-elastic, bomSpecVersion=1.5, projectUuid=5b06171a-13a6-4a14-ac47-1937ac160c29, projectVersion=9118, bomVersion=1]

I will would create another issue, but need first need to fiddle with the bom file to provide something to reproduce and test if this is still happening with SNAPSHOT

[muellerst-hg]

It seems this exception has been reported already in #3895

[nscuro]

Thanks, will investigate.

#3895 mentions project cloning in the reproduction steps. Are you also making use of cloning before you upload the new BOM?

[muellerst-hg]

yes, we clone an existing project with the following script:

export DATA="{\"project\":\"${PROJECT_UUID}\",\"version\":\"${NEW_PROJECT_VERSION}\",\"includeTags\":true,\"includeProperties\":true,\"includeComponents\":false,\"includeServices\":true,\"includeAuditHistory\":true,\"includeACL\":true}"
curl \
      --url "${DEPENDENCY_TRACK_BASE_URL}/api/v1/project/clone" \
      --silent \
      --location \
      --fail \
      --header "X-API-Key: ${DEPENDENCY_TRACK_TOKEN}" \
      --header 'Content-Type: application/json' \
      --request PUT \
      --data-raw "${DATA}

Then wait for the new project to be crated, fetch the NEW_PROJECT_UUID and then upload the BOM using:

curl \
      --url "${DEPENDENCY_TRACK_BASE_URL}/api/v1/bom" \
      --silent \
      --show-error \
      --location \
      --fail \
      --header "X-API-Key: ${DEPENDENCY_TRACK_TOKEN}" \
      --header 'Content-Type: multipart/form-data' \
      --request POST \
      --form "project=${NEW_PROJECT_UUID}" \
      --form "[email protected]"

[muellerst-hg]

it pretty much looks like that it's related to cloning:
When using a new project and uploading the bom file, the processing does not throw this exception.
When cloning an existing project and uploading the bom file, I get the followibng in my logs:

2024-07-15 16:04:32,205 ERROR [Persist] Insert of object "org.dependencytrack.model.ProjectMetadata@7d1feb10" using statement "INSERT INTO "PROJECT_METADATA" ("AUTHORS","PROJECT_ID","SUPPLIER") VALUES (?,?,?)" failed : ERROR: duplicate key value violates unique constraint "PROJECT_METADATA_PROJECT_ID_IDX"

[nscuro]

That should be sufficient to reproduce and fix. Appreciate the detailed context.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.