Mitigated On/Before/After now use DateTimeFilter

[hblankenship]

Fixes filtering for mitigation for the on/before/after fields

[sc-7759]

[dryrunsecurity]

DryRun Security Summary

The code change enhances the Defect Dojo application's security filtering capabilities by introducing more granular filtering options for mitigation dates and EPSS metrics in the dojo/filters.py file.

Expand for full summary

Summary:

This code change appears to be a security-focused update to the dojo/filters.py file in the Defect Dojo application. The key changes include the addition of a new DateTimeFilter and updates to the ApiFindingFilter, FindingFilterHelper, and FindingFilterWithoutObjectLookups classes to introduce more granular filtering capabilities around mitigation dates and EPSS (Exploitability Prediction Score) metrics.

These changes suggest that the application is expanding its ability to filter and sort findings, which could lead to improved security posture by allowing users to more easily identify and focus on findings that may pose a higher risk or have been mitigated in a timely manner. However, it's important to ensure that these new filtering capabilities are properly secured and do not introduce any vulnerabilities, such as potential SQL injection or other injection-based attacks. The application should also validate and sanitize all user input to these filters to prevent malicious input from being used to bypass security controls or access sensitive data.

Files Changed:

• dojo/filters.py: This file contains various Django filters used for filtering and sorting data in the Defect Dojo application. The key changes in this patch include:
  • Addition of a new DateTimeFilter to the list of imported filters.
  • Updates to the ApiFindingFilter class to add new DateTimeFilter fields for mitigated_on, mitigated_before, and mitigated_after, along with corresponding filtering methods.
  • Updates to the FindingFilterHelper and FindingFilterWithoutObjectLookups classes to add similar changes for mitigated_on, mitigated_before, and mitigated_after fields, as well as new fields for epss_score, epss_score_range, epss_percentile, and epss_percentile_range to allow filtering findings based on their EPSS metrics.

Code Analysis

We ran 9 analyzers against 1 file and 0 analyzers had findings. 9 analyzers had no findings.

View PR in the DryRun Dashboard.

[Maffooch]

If a user were to be reliant on passing a date to this filter, would the API accept the absence of the timestamp? For example, if the query was mitigated_on=2025-01-02 would a validation error be thrown?

[hblankenship]

A validation error does not get thrown if you use 2025-01-02 without the timestamp. Should it?

[Maffooch]

No, a validation error should not be raised. I was asking to make sure that this change would not cause any issues for users that were currently using dates

[mtesauro]

Approved

[cneill]

This doesn't seem to be operating how I would expect... I spun up a test instance, created some findings, and marked them mitigated. The UI uses a date-picker widget that won't allow you to enter a timestamp, so I couldn't test that.

With the API, I have 3 mitigated findings with these mitigated values:

❯ curl -H "Authorization: Token ${DD_API_KEY}"  "http://localhost:8080/api/v2/findings/" | jq '.results[] | select(.mitigated != null) | .mitigated'

"2025-01-13T18:08:05.911321Z"
"2025-01-13T18:08:05.922679Z"
"2025-01-13T18:08:05.930498Z"

Testing mitigated_on with a simple date:

❯ curl -H "Authorization: Token ${DD_API_KEY}"  "http://localhost:8080/api/v2/findings/?mitigated_on=2025-01-13"
{"count":0,"next":null,"previous":null,"results":[],"prefetch":{}}%

It wants an exact timestamp:

❯ curl -H "Authorization: Token ${DD_API_KEY}"  "http://localhost:8080/api/v2/findings/?mitigated_on=2025-01-13T18:08:05.911321Z"
{"count":1,"next":null,"previous":null,"results":[{"id":1, ... }]}

The mitigated_after and mitigated_before filters sort of work as expected with simple dates, but I assume it is just plugging in a 00:00:00 for the time, so for example this returns my 3 findings that were mitigated today, 2025-01-13, where mitigated_after would only have included things after today (23:59:59) in the past.

❯ curl -H "Authorization: Token ${DD_API_KEY}"  "http://localhost:8080/api/v2/findings/?mitigated_after=2025-01-13"
{"count":3,"next":null,"previous":null,"results":[{"id":1, ... }]}

I think this needs some additional tweaking, and could really use some unit tests.

[hblankenship]

This doesn't seem to be operating how I would expect... I spun up a test instance, created some findings, and marked them mitigated. The UI uses a date-picker widget that won't allow you to enter a timestamp, so I couldn't test that.

With the API, I have 3 mitigated findings with these mitigated values:

❯ curl -H "Authorization: Token ${DD_API_KEY}"  "http://localhost:8080/api/v2/findings/" | jq '.results[] | select(.mitigated != null) | .mitigated'

"2025-01-13T18:08:05.911321Z"
"2025-01-13T18:08:05.922679Z"
"2025-01-13T18:08:05.930498Z"

Testing mitigated_on with a simple date:

❯ curl -H "Authorization: Token ${DD_API_KEY}"  "http://localhost:8080/api/v2/findings/?mitigated_on=2025-01-13"
{"count":0,"next":null,"previous":null,"results":[],"prefetch":{}}%

It wants an exact timestamp:

❯ curl -H "Authorization: Token ${DD_API_KEY}"  "http://localhost:8080/api/v2/findings/?mitigated_on=2025-01-13T18:08:05.911321Z"
{"count":1,"next":null,"previous":null,"results":[{"id":1, ... }]}

The mitigated_after and mitigated_before filters sort of work as expected with simple dates, but I assume it is just plugging in a 00:00:00 for the time, so for example this returns my 3 findings that were mitigated today, 2025-01-13, where mitigated_after would only have included things after today (23:59:59) in the past.

❯ curl -H "Authorization: Token ${DD_API_KEY}"  "http://localhost:8080/api/v2/findings/?mitigated_after=2025-01-13"
{"count":3,"next":null,"previous":null,"results":[{"id":1, ... }]}

I think this needs some additional tweaking, and could really use some unit tests.

With simple dates being given, it does indeed use 00:00:00 as the time. The request was to make it use DateTimeFilter which means it expects a full datetime. Per the Story:

The UI and API use a date filter for the mitigated field rather than a datetime filter. The datetime filter should be used instead

So when you said 'after' and gave 2025-01-13, the assumption is that you meant 2025-01-13T00:00:00. That said, I think we can alter the after to add a 23:59:59 timestamp if it is not added instead of the 00:00:00.
In a similar manner, mitigated_on will NOT work without a specific time. DateTimeFilters being what they are. Happy to entertain thoughts on this one.
Also, why do we even have on, before, and after when the mitigated filter is a range filter? I am sure it is so we do not introduce breaking changes...
Finally, should the UI change to use datetime instead of simple date?

[cneill]

This one's squirrelly... I'm making some assumptions with my comments here, so others might reasonably disagree. But I tried to capture the spirit of what users are likely to want

[cneill]

This seems like the right way to handle "after" when the user provides a simple date without a timestamp. Given that the UI currently uses a simple date picker without the ability to specify time (which seems fine), this would do what I would intuitively expect. We don't have to worry about the possibility of the user asking for "after 2025-01-01 00:00:00" here unless they're fiddling around in the Developer Console or something.

I suppose we could include microseconds here or use gte <date+1> 00:00:00 to deal with edge cases around the last second of the day...

[cneill]

Given the simple date-picker presented in the UI, I think searching for anything with a timestamp between [ <date> 00:00:00, <date+1> 00:00:00 ) would be the right approach here. Otherwise, I think this filter will only ever return Findings mitigated at exactly 00:00:00, which would be pretty uncommon.

[cneill]

This logic is a little trickier when it comes to the API, because the user could intentionally query for "after 2025-01-01 00:00:00"... But I think this approach strikes the right balance - it seems much more likely that we'll receive dates without timestamps than dates with explicit 00:00:00 timestamps, and this would mirror the UI's behavior as well. In any case, the user can always search for "after 2024-12-31" to achieve the same thing if that's truly what they want.

[cneill]

It seems very unlikely to me that a user will ever actually want to list Findings mitigated on an exact timestamp down to the microsecond, even if this would theoretically give them that option. I think searching the range [ <date> 00:00:00, <date+1> 00:00:00 ) is the right approach here too, regardless of whether the user specified a timestamp.

[cneill]

Looks good! 😄