docs(self-hosted): experimental external kafka #11847
Conversation
|
@aldy505 is attempting to deploy a commit to the Sentry Team on Vercel. A member of the Team first needs to authorize it. |
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
2 Skipped Deployments
|
|
This pull request has gone three weeks without activity. In another week, I will close it. But! If you comment or otherwise update it, I will reset the clock, and if you add the label "A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀 |
|
ping @BYK |
|
|
||
| Sentry uses the confluent-kafka library, which leverages the [default Kafka config from librdkafka](https://github.com/confluentinc/librdkafka/blob/master/CONFIGURATION.md). Modify your `sentry.conf.py` file like so: | ||
|
|
||
| ```python |
There was a problem hiding this comment.
Should we put these in the example config file too?
|
|
||
| ## Snuba | ||
|
|
||
| Although Snuba also uses confluent-kafka under the hood, not every configuration option is available. Modify your `docker-compose.yml` file like so: |
There was a problem hiding this comment.
Let's encourage the use of docker-compose.override.yml instead
| Although Snuba also uses confluent-kafka under the hood, not every configuration option is available. Modify your `docker-compose.yml` file like so: | |
| Although Snuba also uses confluent-kafka under the hood, not every configuration option is available. Modify your `docker-compose.override.yml` file as: |
There was a problem hiding this comment.
I personally modify the original docker compose file, create another git branch on top of it. Everytime there's a new release, I'll merge the upstream release tag onto my branch.
There was a problem hiding this comment.
More ammunition for my "this should be a patch file in the repo" suggestion below 💪🏻
|
|
||
| As of the time of writing, Vroom does not support any kind of authentication. | ||
|
|
||
| Modify your `docker-compose.yml` file like so: |
There was a problem hiding this comment.
| Modify your `docker-compose.yml` file like so: | |
| Modify your `docker-compose.override.yml` file as: |
Co-authored-by: Burak Yigit Kaya <[email protected]>
Co-authored-by: Burak Yigit Kaya <[email protected]>
Co-authored-by: Burak Yigit Kaya <[email protected]>
Co-authored-by: Burak Yigit Kaya <[email protected]>
Co-authored-by: Burak Yigit Kaya <[email protected]>
Co-authored-by: Burak Yigit Kaya <[email protected]>
Co-authored-by: Burak Yigit Kaya <[email protected]>
DESCRIBE YOUR PR
Provides guide for using external Kafka for self-hosted use cases. Obviously Sentry employees are not obliged to support or answer to any kind of questions regarding this, since this falls on the "Experimental" category.
But fun fact: Removing Kafka from the default
docker-compose.ymlwill save you around 50% of RAM usage. Kafka is that bad.IS YOUR CHANGE URGENT?
Help us prioritize incoming PRs by letting us know when the change needs to go live.
SLA
Thanks in advance for your help!
PRE-MERGE CHECKLIST
Make sure you've checked the following before merging your changes:
LEGAL BOILERPLATE
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. and is gonna need some rights from me in order to utilize my contributions in this here PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.
EXTRA RESOURCES