/capture endpoint should enforce a size limit

[guidoiaquinti]

Bug description

We currently rely on Django’s DATA_UPLOAD_MAX_MEMORY_SIZE = 20971520 # 20 MB to enforce the maximum size in bytes that a request body may be before a RequestDataTooBig exception get raised. Unfortunately, for Kafka topics and brokers, we use the default config message.max.bytes default 1 MB.

This is not a huge issue as /capture will probably return a 500 anyway if Kafka refuses to accept the message but I think we should define an event max size and enforce it in both API and Kafka (and mention it in our docs).

Expected behavior

When we send an event > limit, we should get an error describing the issue.

How to reproduce

1. Send an event of 10 MB, we don’t get a RequestDataTooBig

Environment

• PostHog Cloud

• self-hosted PostHog (ClickHouse-based), version/commit: please provide

• self-hosted PostHog (Postgres-based, legacy), version/commit: please provide

Additional context

• Internal Slack

[macobo]

Unfortunately, for Kafka topics and brokers, we use the default config message.max.bytes default 1 MB.

Note that one request can result in many kafka messages due to batching and session recording message chunking. So rather than decrease the limit, we'd need to validate each event being sent separately.

Related: #2927 #3632

[posthog-bot]

This issue hasn't seen activity in two years! If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in two weeks.

[posthog-bot]

This issue was closed due to lack of activity. Feel free to reopen if it's still relevant.