Bug with CH persons double click on retention table

[paolodamico]

Bug description

We have an issue in which the table of users for each cohort does not contain the same number of users as the table reports.

Example

(Source)

Table shows 6 users in cohort:

Persons list shows a lot more than 6 users in cohort:

Expected behavior

The persons list and the retention table should match.

How to reproduce

See description above.

Environment

Cloud, Clickhouse.

Additional context

Currently blocking internal personas analysis.

Thank you for your bug report – we love squashing them!

[paolodamico]

This seems to be happening in other graphs too. See example

[EDsCODE]

#2992 address the problem for the line graph example. Retention issue still open

[paolodamico]

I don't know if I discovered another bug or is the same bug manifesting another way. See here the table saying there's 21 users, but then the table contains way more than 21 users (~90).

In addition, worth mentioning that I'm experiencing a ton of timeouts when getting the persons list (about 8 or 9 times out of 10).

[paolodamico]

There still seems to be a bug here (unsure if it's a one-off thing), but I opened this graph today and got no users,

[yakkomajuri]

Still not really working and also essential for my work following up with onboarded users that might be churning. Right now I have no idea who the users are.

[yakkomajuri]

Are these the same?

#3018

[EDsCODE]

Dont think so. Going to revisit this today

[EDsCODE]

update on this:
#3331 and #3332 fixes the blank results and overload of results that surpassed counts.

What's happening now that's still unintuitive is that I'm filtering out people that don't exist in postgres. I think there's a sync issue between postgres and clickhouse where a user doesn't exist at all in postgres but exists in clickhouse. So the query is accounting for users in clickhouse but when I try to compile the list with person details, it can't find the user in postgres. I've opted to omit these missing rows and just return a list of the users that do exist. This will result in a count discrepancy again where sometimes the actual number of rows will be less than the number shown at the top.

I could match the returned row count with the totals at the top of the modal but then there would be a difference between the table number that's shown in the visualization and the numbers shown in the modal

[mariusandra]

Just guessing here, but could there be some issue with running the clickhouse person table ALTER queries somewhere?

One possible explanation is that these queries silently failed or were skipped for the python ingestion for a while, but worked with the plugin server ingestion. This also gives a plausible explanation to the outage as the server was just not able to handle this many newly seen alter queries.

The other option is the plugin server skipped a lot of them when ingesting, especially when clickhouse went down. It possibly wasn't fully down, but just the person queries failed at first.

What's the date range for the faulty data?

[yakkomajuri]

@mariusandra I mean the issue was reported back in December - was the plugin server at all in cloud back then?

[mariusandra]

Nope, it's only been doing something for the last 2 weeks

[Twixes]

I'll close this since it appears the original problem is not relevant anymore. A related but different problem should get its own issue.