Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix and document discovery queries behavior on distributed queries and add tests #7655

Merged
merged 2 commits into from
Feb 17, 2023

Conversation

lucasmrod
Copy link
Contributor

@lucasmrod lucasmrod commented Jun 27, 2022

This PR fixes and attempts to document the discovery queries behavior in distributed queries.

The main change:
If a distributed query does not have its corresponding discovery query, then it will always be executed.
(Currently on master this is not the case, if there's at least one discovery query then all those distributed queries that do not have one defined are not executed at all.)

This PR also fixes #5260 and #7793 and adds some tests to Distributed.acceptWork.

@lucasmrod lucasmrod requested review from a team as code owners June 27, 2022 10:48
@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Jun 27, 2022

CLA Signed

The committers listed above are authorized under a signed CLA.

  • ✅ login: lucasmrod / name: Lucas Manuel Rodriguez (89a06da)

@@ -229,7 +233,7 @@ Status Distributed::acceptWork(const std::string& work) {
return Status(1, "Distributed query is not a string");
}

if (queries_to_run.empty() || queries_to_run.count(name)) {
if (!hasDiscoveryQueries || queries_to_run.count(name)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand how this fixes the bug described in #5260. How is this behavior different that prior? (I'm not enough of a c person to discern if there's something clever happening)

Are there cases hasDiscoveryQueries is false, but not !queries_to_run.empty()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi!, it's a pretty confusing feature/logic.

Currently in master:

  1. queries_to_run starts empty.
  2. First for loop iterates the received discovery queries. For each discovery query, if they return > 0 results, then the query name is added to queries_to_run.
  3. Second for loop iterates the received queries and, "stores them in the database to be run in runQueries" if queries_to_run is empty or if the corresponding discovery query returned > 0 results.

Are there cases hasDiscoveryQueries is false, but not !queries_to_run.empty()?

The bug in master is that if there are discovery queries and all of them return no results, then queries_to_run will be empty, and therefore all queries are incorrectly enqueued for execution (because it's using queries_to_run.empty() in the if).

if (!hasDiscoveryQueries || queries_to_run.count(name)) {

I believe this change fixes the above bug:

  • All queries are queued if there are no discovery queries (hasDiscoveryQueries=false).
  • If hasDiscoveryQueries=true, then the query is only enqueued if it returned results queries_to_run.count(name) == true.

Let me know if it makes sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this PR just fixes a corner-case in the current logic, it does not try to change such logic (discussed in #5260 (comment)).

Copy link
Contributor Author

@lucasmrod lucasmrod Oct 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've ended up fixing the "odd behavior" and adding documentation instead. Let me know if (by reading docs and tests) it now makes more sense.

@lucasmrod lucasmrod force-pushed the 5260-fix-discovery-queries-bug branch from 89a06da to 7387f4b Compare October 18, 2022 00:16
@lucasmrod lucasmrod changed the title Add fix for discovery query bug and add tests for distributed queries Fix and document discovery queries behavior on distributed queries and add tests Oct 18, 2022
@lucasmrod lucasmrod force-pushed the 5260-fix-discovery-queries-bug branch 2 times, most recently from ad6452a to dad93c0 Compare October 18, 2022 00:30
Copy link
Member

@zwass zwass left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks to be well-documented and tested. Thank you @lucasmrod! @directionless any further questions/concerns from you? Otherwise I'll merge later today.

@directionless directionless added this to the 5.8.0 milestone Feb 15, 2023
@zwass zwass merged commit cdadfd2 into osquery:master Feb 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Distributed queries execute regardless of discovery
4 participants