BUGFIX: [graph] Hide filtered nodes/edges in correlation graphs

[ckane]

Right now, these are simply greyed out as "disabled". However, when we filter items from the correlation graphs, we want it to "clean up" the graph. So, hide the affected nodes+edges from view so we can explore just the correlated items we include in the filters.

Proposed changes

In buildCorrelationData in the utils/Graph.js, change the "filter" behavior from disabling filtered nodes (which simply changes their color) to remove them from the graph (and their edges). This is how the Correlation feature used to work, and the behavior regressed sometime later.

Keeping the disabled nodes + edges on the graph in correlation view significantly defeats the purpose of being able to filter these nodes out, as it makes large graphs difficult to navigate.

Checklist

• I consider the submitted work as finished
• I tested the code for its functionality
• I wrote test cases for the relevant uses case (coverage and e2e)
• I added/update the relevant documentation (either on github or on notion)
• Where necessary I refactored code to improve the overall quality

Screenshots

Example of filter behavior fixed with this PR:

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.24%. Comparing base (671eae4) to head (f1a6daa).
Report is 14 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #9236   +/-   ##
=======================================
  Coverage   65.24%   65.24%           
=======================================
  Files         624      624           
  Lines       59623    59623           
  Branches     6636     6636           
=======================================
  Hits        38899    38899           
  Misses      20724    20724           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[labo-flg]

Hi @ckane and thanks for your contribution.
We already tested different approaches in the past.
The current version (having filtered items greyed out) is actually the solution we selected in the end for good reasons.

One side effect of removing the filtered items entirely is that the 3D forces are re-applied and you lose the positioning and quickly get lost.

[ckane]

Hi @ckane and thanks for your contribution. We already tested different approaches in the past. The current version (having filtered items greyed out) is actually the solution we selected in the end for good reasons.

One side effect of removing the filtered items entirely is that the 3D forces are re-applied and you lose the positioning and quickly get lost.

I understand, especially for the primary knowledge graphs. However, in this situation, when all of the "intermediate nodes" between the current report and a linked report are "greyed out", the linked-report in that case should be "greyed out" too, but it is not, so the existing behavior doesn't solve this problem either.

In this chart, as an example, the "Regin Scanner" report at top-right is the "current report". All nodes connected to "OSINT Regin Samples" are greyed out, so that report should also get greyed out, in order to use the filters to "hide correlations based upon linked entity type".

Additionally, I have CTI analysts who say that the "greying the nodes out" in the correlation charts creates charts that are difficult to view when they're trying to explore correlations to other reporting in the system using the correlation graphs, and they similarly describe the experience of "easy to get lost".

I realize the Investigations feature is also one potential option, but this is a lot more time-consuming and requires creating new investigation entities for work that will ultimately get thrown away.

I understand the distinction is probably a matter of having differing use-cases as well as different procedures for research. Can we have the choice between either behavior as an option inside the graphing widget?