Connecting AWS MSK using librdkafka

[priyarv1210]

Read the FAQ first: https://github.com/edenhill/librdkafka/wiki/FAQ

Do NOT create issues for questions, use the discussion forum: https://github.com/edenhill/librdkafka/discussions

Description

I am trying to connect AWS MSK using this library. We have a IAM based AWS and looks like it has to configured only through sasl jass config.

sasl jaas config is throwing "JAVA Jaas configuration is not supported".

Can you tell me how I can use sasl jaas config.

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

• librdkafka version (release number or git tag): librdkafka v1.6.2
• Apache Kafka version: KAFKA version is 2.6.2
• librdkafka client configuration:
• Operating system: RHEL
• Provide logs
  % Java JAAS configuration is not supported, see https://github.com/edenhill/libr dkafka/wiki/Using-SASL-with-librdkafka for more information.
• Provide broker log excerpts
• Critical issue

[edenhill]

You will need to contact AWS support for help with MSK.

[priyarv1210]

This is not about issue in MSK. But not able to use MSK for configs in librdkafka library. Please advise.

[mwatson-x]

You will need to contact AWS support for help with MSK.

@edenhill This seems odd because the people at AWS are saying that this should be supported by you. aws/aws-msk-iam-auth#10 (comment)

This means that we are now caught in a weird situation where nobody has responsibility for supporting this functionality

[NaodFessahaye]

Any updates on this issue? Currently facing the same problem. I would greatly appreciate it if anyone could share resources on how they solved this problem. Thank you.

[crseat]

Why is this issue closed? As far as I can see there has been no resolution.

[mensfeld]

@crseat the issue is closed because librdkafka supports IAM based auth via custom oauth callbacks with option to refresh them when needed.

[crseat]

@mensfeld is there an example of that implemented anywhere for reference?

[mensfeld]

This is for Ruby framework built on top of librdkafka: https://karafka.io/docs/Deployment/#custom-oauth-token-providers - custom oauth token providers support works and it works with MSK as any other vendor.