Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using kafka without groupID for 3 vus dropped out all memory #327

Open
AleksandrRepo opened this issue Feb 18, 2025 · 1 comment
Open

Using kafka without groupID for 3 vus dropped out all memory #327

AleksandrRepo opened this issue Feb 18, 2025 · 1 comment

Comments

@AleksandrRepo
Copy link

Hi Mostafa!

Please tell how do you think whats the potentially problem could be? I have a topic with 3 partition.
If I use only "groupID" and "groupTopics" and 3 vus (for 1 user to each partition - thats logics justify) I have to many errors "context deadline exceeded". I've already read an article about this problem in this repository but changes "maxWait" (and something like readLagInterval, readBatchTimeout, maxAttempts) didnt help me much.

For example when I read kafka without groupID with 1 or 2 vus - it works well - no error "context deadline exceeded" at all. But not all expected messages were found in this case. If I understand well - because I only 2 partition have beed read and 1 partition have not.

When I use 3 vus without groupID - I get an error like below and very soon all memory at my machine dropped out very soon (36 Gb). Probly there are some memory lack or I do something wrong? Any ideas?

ERRO[0098] Unable to read messages., OriginalError: fetching message: context deadline exceeded error="Unable to read messages., OriginalError: fetching message: context deadline exceeded"Us, 261 complete and 0 interrupted iterations
ERRO[0098] GoError: Unable to read messages., OriginalError: fetching message: context deadline exceeded
at github.com/mostafa/xk6-kafka.(*Kafka).readerClass.func1 (native)

@mostafa
Copy link
Owner

mostafa commented Feb 21, 2025

Hey @AleksandrRepo,

If you could provide me with a snippet that is causing this issue, I may be able to help you fix it. Please remove sensitive information before posting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants