Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Second step for refactor distributed query #576

Merged
merged 11 commits into from
Jan 7, 2025

Conversation

zhuoyuan-liu
Copy link
Contributor

In this step, we would like to refactor the way of completing a distributed query and minimize the write request to the database.

Changelog:

  • Removed TrackExcution function and relevant table
  • Replaced VerifyComplete with CleanupCompletedQueries. In this case, we don't need to check the complete status whenever we receive new data from Osquery. Instead, we check it periodically with the expired function. Since we changed how to fetch the list of target queries, keeping a distributed query in an active state does not affect performance.
  • Update how to get lists of target nodes in API and osctrl-admin. We use the intersection to get a list of targeted nodes. We can easily extend this part to support more tags and unions or other operations.

TODO:

  • Remove everything related to CreateTarget function and relevant database table. This function was used by admin, API and cli. I am wondering if we can remove these duplications and make sure the Admin and CLI talk to API instead of the database directly.

@javuto javuto self-requested a review December 18, 2024 22:19
@javuto javuto added refactor Refactorization of code queries On-demand queries related issues labels Dec 18, 2024
Copy link
Collaborator

@javuto javuto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! 👏

@javuto javuto merged commit 63b2a29 into jmpsec:main Jan 7, 2025
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
queries On-demand queries related issues refactor Refactorization of code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants