Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement new evaluation history tracking logic #3556

Closed
dmjb opened this issue Jun 7, 2024 · 0 comments · Fixed by #3713
Closed

Implement new evaluation history tracking logic #3556

dmjb opened this issue Jun 7, 2024 · 0 comments · Fixed by #3713
Assignees
@dmjb
Labels
user-story

Comments

@dmjb
Copy link
Contributor

dmjb commented Jun 7, 2024

Please describe the enhancement

As described in design doc in epic.

Solution Proposal

As described in design doc in epic.

Describe alternatives you've considered

No response

Additional context

No response

Acceptance Criteria

No response
@dmjb dmjb added the user-story label Jun 7, 2024
@dmjb dmjb self-assigned this Jun 10, 2024
@dmjb dmjb changed the title Implement new evaluation history track logic Implement new evaluation history tracking logic Jun 11, 2024
dmjb added a commit that referenced this issue Jun 13, 2024
@dmjb
Decouple Action Engine from Rule Type Engine
9858d1f 
Relates to #3556

This allows the rule type engine to be reused across multiple profiles.
This simplifies a change in an upcoming PR to cache rule types during
profile evaluation.
@dmjb dmjb mentioned this issue Jun 13, 2024
Merged
10 tasks
dmjb added a commit that referenced this issue Jun 13, 2024
@dmjb
Decouple Action Engine from Rule Type Engine
48837c1 
Relates to #3556

This allows the rule type engine to be reused across multiple profiles.
This simplifies a change in an upcoming PR to cache rule types during
profile evaluation.
dmjb added a commit that referenced this issue Jun 14, 2024
@dmjb
Decouple Action Engine from Rule Type Engine (#3599)
5d33ee6 
Relates to #3556

This allows the rule type engine to be reused across multiple profiles.
This simplifies a change in an upcoming PR to cache rule types during
profile evaluation.
dmjb added a commit that referenced this issue Jun 19, 2024
@dmjb
Log evaluation history in new tables
1a85fc8 
This adds an interface and implementation for logging rule evaluation
statuses. The logic flow is described in the design doc. Note that this
does not wire in the logic into the engine yet, nor does it track
remediations/alerts. These will be added in future PRs.

Relates to: #3556
@dmjb dmjb mentioned this issue Jun 19, 2024
Merged
10 tasks
dmjb added a commit that referenced this issue Jun 19, 2024
@dmjb
Log evaluation history in new tables
66b296b 
This adds an interface and implementation for logging rule evaluation
statuses. The logic flow is described in the design doc. Note that this
does not wire in the logic into the engine yet, nor does it track
remediations/alerts. These will be added in future PRs.

Relates to: #3556
dmjb added a commit that referenced this issue Jun 19, 2024
@dmjb
Store most recent evaluation time in dedicated column
7f4e38d 
This will make it easier to sort the rows by date/time.

Relates to: #3556
@dmjb dmjb mentioned this issue Jun 19, 2024
Merged
10 tasks
dmjb added a commit that referenced this issue Jun 19, 2024
@dmjb
Store most recent evaluation time in dedicated column (#3661)
3292de3 
This will make it easier to sort the rows by date/time.

Relates to: #3556
dmjb added a commit that referenced this issue Jun 19, 2024
@dmjb
Log evaluation history in new tables
2ded8b3 
This adds an interface and implementation for logging rule evaluation
statuses. The logic flow is described in the design doc. Note that this
does not wire in the logic into the engine yet, nor does it track
remediations/alerts. These will be added in future PRs.

Relates to: #3556
dmjb added a commit that referenced this issue Jun 19, 2024
@dmjb
Log evaluation history in new tables
31183cf 
This adds an interface and implementation for logging rule evaluation
statuses. The logic flow is described in the design doc. Note that this
does not wire in the logic into the engine yet, nor does it track
remediations/alerts. These will be added in future PRs.

Relates to: #3556
dmjb added a commit that referenced this issue Jun 19, 2024
@dmjb
Log evaluation history in new tables (#3659)
6ff17de 
This adds an interface and implementation for logging rule evaluation
statuses. The logic flow is described in the design doc. Note that this
does not wire in the logic into the engine yet, nor does it track
remediations/alerts. These will be added in future PRs.

Relates to: #3556
dmjb added a commit that referenced this issue Jun 21, 2024
@dmjb
Define evaluation metrics
834f355 
Add in some basic metrics to track rule/entity evaluations to the
engine. Note that the metrics for alerts and remediations are not wired
in yet since they depend on some changes which will be added in the next
PR.

I have taken a different approach to handling noop metrics than is used
in the codebase so far: instead of defining an interface and separate
real vs noop implementations, I have decided to make use of the noop
metrics handling built into otel: there is now a metrics meter factory
with alternative implementation for noop vs "real" metrics. Any part of
the code which needs to create metrics can use the factory without
caring whether the metrics are actually exported or not.

Relates to: #3556
@dmjb dmjb mentioned this issue Jun 21, 2024
Merged
10 tasks
dmjb added a commit that referenced this issue Jun 24, 2024
@dmjb
Define evaluation metrics (#3688)
85f3123 
Add in some basic metrics to track rule/entity evaluations to the
engine. Note that the metrics for alerts and remediations are not wired
in yet since they depend on some changes which will be added in the next
PR.

I have taken a different approach to handling noop metrics than is used
in the codebase so far: instead of defining an interface and separate
real vs noop implementations, I have decided to make use of the noop
metrics handling built into otel: there is now a metrics meter factory
with alternative implementation for noop vs "real" metrics. Any part of
the code which needs to create metrics can use the factory without
caring whether the metrics are actually exported or not.

Relates to: #3556
dmjb added a commit that referenced this issue Jun 24, 2024
@dmjb
Start using new eval history table
ab94888 
This PR wires the eval history service into the evaluation engine. It is
gated behind a feature flag while we experiment with it. A follow on PR
will add logging for remediation and alerting.

Relates to #3556
@dmjb dmjb mentioned this issue Jun 24, 2024
Merged
10 tasks
dmjb added a commit that referenced this issue Jun 25, 2024
@dmjb
Start using new eval history table
7034f61 
This PR wires the eval history service into the evaluation engine. It is
gated behind a feature flag while we experiment with it. A follow on PR
will add logging for remediation and alerting.

Relates to #3556
dmjb added a commit that referenced this issue Jun 25, 2024
@dmjb
Start using new eval history table
83a02e8 
This PR wires the eval history service into the evaluation engine. It is
gated behind a feature flag while we experiment with it. A follow on PR
will add logging for remediation and alerting.

Relates to #3556
dmjb added a commit that referenced this issue Jun 25, 2024
@dmjb
Start using new eval history table
a21ec8a 
This PR wires the eval history service into the evaluation engine. It is
gated behind a feature flag while we experiment with it. A follow on PR
will add logging for remediation and alerting.

Relates to #3556
dmjb added a commit that referenced this issue Jun 25, 2024
@dmjb
Start using new eval history table
cbaa6d4 
This PR wires the eval history service into the evaluation engine. It is
gated behind a feature flag while we experiment with it. A follow on PR
will add logging for remediation and alerting.

Relates to #3556
dmjb added a commit that referenced this issue Jun 26, 2024
@dmjb
Start using new eval history table (#3703)
cae4b26 
This PR wires the eval history service into the evaluation engine. It is
gated behind a feature flag while we experiment with it. A follow on PR
will add logging for remediation and alerting.

Relates to #3556
dmjb added a commit that referenced this issue Jun 26, 2024
@dmjb
Store alert and remediation history
4ce9d7d 
This captures the alert and remediation statuses and stores them in the
new tables.

Fixes #3556
@dmjb dmjb mentioned this issue Jun 26, 2024
Merged
10 tasks
@dmjb dmjb closed this as completed in 37323cd Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dmjb dmjb

Labels
user-story
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Store alert and remediation history mindersec/minder
1 participant
@dmjb