Upload static cohort csv #2932
Upload static cohort csv #2932
Conversation
There was a problem hiding this comment.
Logic overall checks out. The only bug that would happen in production is that you are producing Proto encoded data to a table that is expecting JSON. You can checkout what encoding to a json table looks like here https://github.com/PostHog/posthog/blob/master/ee/clickhouse/models/person.py#L74
This also raises a really good point that we should probably disable inserting to clickhouse for tests and use the kafka path since that is the only active path in any environment.
ee/clickhouse/models/cohort.py
Outdated
| for person_uuid in person_uuids: | ||
| pb_event = person_static_cohort_pb2.PersonStaticCohort() | ||
| pb_event.id = str(uuid.uuid4()) | ||
| pb_event.person_id = str(person_uuid) | ||
| pb_event.cohort_id = cohort_id | ||
| pb_event.team_id = team.pk | ||
|
|
||
| p = ClickhouseProducer() | ||
| p.produce_proto(sql=INSERT_PERSON_STATIC_COHORT, topic=KAFKA_PERSON_STATIC_COHORT, data=pb_event) |
There was a problem hiding this comment.
This is perfect for a proto encoded table, but the table you have defined is json encoded.
Check out producing for persons to see what producing to a json table looks like
https://github.com/PostHog/posthog/blob/master/ee/clickhouse/models/person.py#L74
|
This is ready to go. Table created on cluster in prod. I'll let you land it tomorrow so you can test it out. |
… into upload-static-cohort-csv
Changes
Allows users to upload CSV with distinct ids or email addresses to create a cohort
@fuziontech Would love a check of what I've done with the Kafka topic/protobuf etc.
Checklist