[Enhancement] Add an integration connector for Strimzi

[davidradl]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

currently no Strimzi integration is present

Expected Behavior

Bring in Strimzi information from the CRD using a polling integration connector. The connector will create an event broker and associated topics, theotpics will contain the name,description, partitions and replicas information

Alternatives

the Topic integrator can be used to bring in the topic name, but not the description, partitions and replicas.

Any Further Information?

No response

Would you be prepared to be assigned this issue to work on?

• I can work on this

[planetf1]

I would like to understand more about the connector

• What kind of dependencies does it have
• How big is it
• What are the testing needs - or plan to test
• Who do we think the community of people a) using b) developing is?
• What is the release cycle - are we going to regular update or leave as is
• Who is going to be the repo owner(s) and responsible for setup & maintainance especially on an ongoing basis for dependencies, build, publish etc

We should look at

• java build
• container build
• security scans
• static code scans
• maven artifact naming
• maven publishing
• Adding to docs (ideally...)
• dependabot
• Automated test (without this hard to validate what we have works on an ongoing basis)

The actual build may need modifying to work with the above & some of the techniques we use

Despite the additional work, I am a little more inclined towards a distinct repository just for clarity & agility - though this has the downside of increasing the overall release/maint workload - especially when a change is needed.

Also I would suggest we take the opportunity to ensure the required info to do the setup is documented, for the benefit of future repos. - to the extent it makes sense, and understanding the nature of this particular connector vs the others
I think it's important we integrate the code into our broader approach if it is to sit alongside the egeria repos..

See also #5943 where we touch on some of the challenges and pros/cons - and ideally needing some more common framework/shared assets to coordinate managing the processes.

[davidradl]

@planetf1 raised some questions
What kind of dependencies does it have :

  implementation "org.slf4j:slf4j-api:${logbackVersion}"
   implementation "com.fasterxml.jackson.core:jackson-databind:${jacksonVersion}"
   implementation "com.fasterxml.jackson.core:jackson-annotations:${jacksonVersion}"
   implementation "com.fasterxml.jackson.core:jackson-core:${jacksonVersion}"
   implementation "org.junit.jupiter:junit-jupiter:${junitjupiterVersion}"
   implementation "org.odpi.egeria:open-connector-framework:${egeriaversion}"
   implementation "org.odpi.egeria:audit-log-framework:${egeriaversion}"
   implementation "org.odpi.egeria:repository-services:${egeriaversion}"
   implementation "org.odpi.egeria:repository-services-apis:${egeriaversion}"
   implementation "org.odpi.egeria:topic-integrator-api:${egeriaversion}"
   implementation "org.apache.httpcomponents:httpclient:4.5.13"
   implementation "org.apache.atlas:atlas-intg:${atlasversion}"
   implementation "org.apache.atlas:atlas-client-v2:${atlasversion}"
   implementation "org.apache.atlas:atlas-common:${atlasversion}"
   implementation "org.apache.atlas:atlas-client-common:${atlasversion}"

How big is it - pretty small 2 small classes ffdc and unit tests
What are the testing needs - or plan to test. It has extensive unit tests.it would be good to bring this into charts
Who do we think the community of people a) using b) developing is? TBA
What is the release cycle - are we going to regular update or leave as is. I suggest the release be updated
Who is going to be the repo owner(s) and responsible for setup & maintainance especially on an ongoing basis for dependencies, build, publish etc TBA
We should look at
java build
container build
security scans
static code scans
maven artifact naming
maven publishing
Adding to docs (ideally...)
dependabot

[planetf1]

Also to clarify - I fully support the suggestion we deliver a Strimzi connector. It's a very interesting Kafka deployment for Kubernetes,.

[mandy-chessell]

• What are the Atlas dependencies for?
• Why is org.odpi.egeria:repository-services:${egeriaversion} included
• How is it working without a dependency on the Data Manager API?

Could you explain the statement under alternatives - The Topic integrator can be used to bring in the topic name, but not the description, partitions and replicas. I thought this was using the topic integrator OMIS? Or is topic integrator something else?

[davidradl]

• What are the Atlas dependencies for?
• Why is org.odpi.egeria:repository-services:${egeriaversion} included
• How is it working without a dependency on the Data Manager API?

Could you explain the statement under alternatives - The Topic integrator can be used to bring in the topic name, but not the description, partitions and replicas. I thought this was using the topic integrator OMIS? Or is topic integrator something else?

• the Atlas dependancies should be removed
• I need to investigate org.odpi.egeria:repository-services:${egeriaversion} I am not sure I will try to remove it and see if it breaks.
• I assume the dependancy on the topic-integrator-api would bringi n the omas client dependancies.

I do not think I was clear enough on the alternatives. It should say It is possible to use KafkaMonitorIntegrationConnector if only the topic name is required.

[planetf1]

Adding myself as assignee to track ongoing work on creating repositories . I will create a template & new issues to manage the specifics (please leave for now). Target: w/c 14 Mar ( tues on)

[planetf1]

@davidradl The 'new repo' template is now merged. Thanks for the feedback. Could you give it a go for the strimzi integration connector first. Then can add a link to that issue here.

For feedback on the template/process - suggest we use #6298