Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need China region support #16

Closed
guanlinz opened this issue Dec 5, 2023 · 2 comments
Closed

Need China region support #16

guanlinz opened this issue Dec 5, 2023 · 2 comments

Comments

@guanlinz
Copy link

guanlinz commented Dec 5, 2023

  • aws-msk-iam-sasl-signer-python version: v1.0.0
  • Python version: 3.7
  • Operating System: Amazon Linux 2
  • Method of installation: pip install
  • Kafka library name: [e.g. kafka-python] kafka-python
  • Kafka library version: 2.0.2
  • Provide us a sample code snippet of your producer/consumer
    Producer:
from kafka import KafkaProducer
from kafka.errors import KafkaError
import socket
import time
from aws_msk_iam_sasl_signer import MSKAuthTokenProvider

class MSKTokenProvider():
    def token(self):
        token, _ = MSKAuthTokenProvider.generate_auth_token('<my aws region>')
        return token

tp = MSKTokenProvider()

producer = KafkaProducer(
    bootstrap_servers='<my bootstrap string>',
    security_protocol='SASL_SSL',
    sasl_mechanism='OAUTHBEARER',
    sasl_oauth_token_provider=tp,
    client_id=socket.gethostname(),
)

topic = "<my-topic>"
while True:
    try:
        inp=input(">")
        producer.send(topic, inp.encode())
        producer.flush()
        print("Produced!")
    except Exception:
        print("Failed to send message:", e)

producer.close()

Description

Meet KafkaTimeout exception while connecting to MSK cluster in China region with IAM auth.

Action done

Checked the network related settings, and switch to use Java to connect to 9098 endpoint without meet any issue.

Checked the code:
In aws_msk_iam_sasl_signer.py, line 17: https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/aws_msk_iam_sasl_signer/MSKAuthTokenProvider.py#L17C1-L17C58

The ENDPOINT_URL_TEMPLATE is only for AWS Global clusters, the MSK endpoint in China has suffix: ".cn", example:
CN_NORTH_1: kafka.cn-north-1.amazonaws.com.cn
doc link for China services endpoint: https://docs.amazonaws.cn/en_us/aws/latest/userguide/endpoints-Beijing.html

Action needed

Please help to verify if the china resources can be connected, and please add China resource support if needed.
@sankalpbhatia
Copy link
Contributor

This is expected. This feature is not yet live in CN regions. So tokens even with the right endpoint won't for for now.

That being said, we have CN launch in our roadmap. Please reach out to AWS support if you wish to know more about it.

Copy link

github-actions bot commented Dec 5, 2023

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants