Query events by date for properties known to be dates

[pauldambra]

Changes

First step in allowing filtering before and after a given date for the events table. When querying properties we know are dates or timestamps but are stored as strings that ClickHouse should be able to parse as DateTime

Example of querying for #6619

• Hard codes DateTime knowledge into the UI to get a release-able vertical slice
• Includes basic UI querying behind a feature flag.
• allows someone to filter any property as if it were a DateTime and returns an empty result set if it is not
• only definitely returns results for the events table
• introduces property types and formats to the API so that we can filter our $time property as a DateTime

How did you test this code?

by adding and running tests, by running the queries manually

running the UI manually with and without the feature flag, filtering on the events table and on an insight

[macobo]

Update ee/clickhouse/models/test/test_property.py#TEST_PROPERTIES

[Twixes]

I don't see "after" or "before" despite the feature flag being enabled, I suppose this may be because I have never ran the branch from #7500? If that is it, then this branch should be rebased on top of it (though #7500 is a draft, so I'm not sure what's the state of work there)

[pauldambra]

I don't see "after" or "before" despite the feature flag being enabled, I suppose this may be because I have never ran the branch from #7500? If that is it, then this branch should be rebased on top of it (though #7500 is a draft, so I'm not sure what's the state of work there)

Hmm, this doesn't depend on #7500

Current state of the branch should let you query any property that ClickHouse knows how to convert as if it was a DateTime and our $time property

[Twixes]

This will be a great addition, though as a user it's pretty confusing that there are two "greater/less than" options, the second one being "after/before". I think it would be more intuitive if is_date_after/is_date_before were just special cases of after/before UI-side, instead of entirely separate options (e.g. for properties recognized as containing dates/times, you would have a calendar icon on the right of the usual text input, which would turn the filter into before/after).

[Twixes]

Poking questions from an ignoramus:
What happens if property_expr results in a non-string? What happens if the value has leading/trailing whitespace? What happens if the datetime string has 10+ characters like 2021-04-01 18:30:00?

[pauldambra]

What happens if the datetime string has 10+ characters like 2021-04-01 18:30:00

parseDateTimeBestEffortOrNull matches a range of formats or returns null. If someone sends a datetime that ClickHouse can't parse it isn't matched in the results (because null is neither greater than nor less than any date)

What happens if property_expr results in a non-string? What happens if the value has leading/trailing whitespace?

select parseDateTimeBestEffortOrNull('   2021-12-25 00:00:01')
 -- ✅ 2021-12-25 00:00:01

select parseDateTimeBestEffortOrNull('   2021-12-25 00:00:01   ')
 -- ✅ 2021-12-25 00:00:01

select parseDateTimeBestEffortOrNull('   not a date   ')
 -- ✅ null

select toDateTimeOrNull('   2021-12-25 00:00:01')
 -- ❌ null

select toDateTimeOrNull('   2021-12-25 00:00:01   ')
 -- ❌ null

At the point there are enough examples in the code to refactor this. I'll include a call-out to the parseDateTime docs. Since this enough is only in my head right now

[pauldambra]

though as a user it's pretty confusing that there are two "greater/less than" options, the second one being "after/before". I think it would be more intuitive if is_date_after/is_date_before were just special cases of after/before UI-side,

agreed.

I think the next step is to merge this and follow up with changes to property/event definitions so the UI can be type aware. And do sensible things like this

[Twixes]

Makes sense to me! Let's get this in

[pauldambra]

Storybook is failing outside of our control. See chromaui/chromatic-cli#485

Will merge this as the changes don't impact the story book