Implement the externally managed bcl_convert service status tracking

[victorskl]

Context:

With ICAv2, we are leveraging ILMN managed bcl_convert workflow run. Unlike in our current v1 Portal, we do not drive the bcl conversion ourselves but to use off the shelve BSSH/basespace managed conversion.

This ILMN managed conversion will be arranged through the instrument integration and auto-launch feature. ditto - https://help.ica.illumina.com/sequencer-integration/analysis_autolaunch

The initial story card - https://trello.com/c/QcWZq6U8/1538-orcabus-v1-implement-bclconvert-workflow-manager-microservice-app

What we need:

We still need to keep tracked of this managed bcl conversion analysis run within our OrcaBus system. This helps drive all key downstream secondary analysis.

See discussion https://umccr.slack.com/archives/C03ABJTSN7J/p1710930212482959

Actions:

• Discuss and finalise with team
• Implement it

[victorskl]

Related (or, the step after or dependency on) #120

[victorskl]

@raylrui Ray has started working on the branch feature/icav2-event-translator

Also added @reisingerf for pair programming and help guide Ray when needs.

[victorskl]

Please use service namespace: orcabus.bct

[victorskl]

Renaming the namespace: orcabus.bcm

ditto #204 convo

[alexiswl]

Why can't we use full names here? bcm isn't very intuitive for anyone new coming into this ecosystem.

[victorskl]

Wouldn't end up too long to remember? Like orcabus.bclconvertmanager?

This namespace will ends up using in our event rule source filter. The longer it gets, typo could introduce by dev. Plus, another case and sep issue could arise among dev if breaking it up.

Also happy to use full name when it fits & intuitive, of course.

[alexiswl]

On the otherhand, do we really want orcabus.biqpm for the BCLConvert InteropQC Pipeline Manager? As annoying as long names are, I do think they help newbies pick up what events are coming from where faster.

It can introduce some name errors, but this is where we could use enums? Or specify this as a constant and it can be parsed into various stacks as a parameter?

[victorskl]

Okay. Got your point.

As long as this source property naming pattern represent the event source of where it come from (or the service that emitted) then it serves the purpose and requirement.

Do we follow kebab case convention in this case?

Like:

orcabus.bclconvert-manager
orcabus.bclconvert-interop-qc-pipeline-manager
orcabus.sequence-run-manager
...

So and, so forth. i.e. follow along how each service get named the directory in lib/workload/stateless/stacks/*

[victorskl]

Also just wanted to note; this is not workflow/pipeline typing yet (i.e. current Portal Workflow.type_name enum). This workflow typing (enum) will happen inside Workflow Manager entity model.

The source property from event envelope focus on the origin of service domain that emit the message into the channel.

[victorskl]

@alexiswl @reisingerf Shall we regroup discuss and, come up with a good naming strategy for this, pls? Gotta do a bit of team work here for best outcome.

[alexiswl]

Yeah happy to, don't want to delay anything here, I just think the longer name might help DX.

[victorskl]

On 16/05/2015 team discussion; to use "compact" format of CDK stack kebab case for service Namespace purpose.

(loigcal `namespace` for event rule purpose; use in event schema `source` property)
change request:
    srm -> sequencerunmanager
    wfm -> workflowmanager
    bcm -> bclconvertmanager
filemanager
metadatamanager
bclconvertinteropqcmanager
bsshicav2fastqcopymanager
cttsov2manager
pieriandxmanager

See note: https://umccr.slack.com/archives/C03ABJTSN7J/p1715906599365689

[victorskl]

OrcaBus V1: Implement bcl_convert workflow manager microservice app