Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Osquerybeat: Add install verification for osquerybeat #30388

Merged
merged 1 commit into from
Feb 15, 2022

Conversation

aleksmaus
Copy link
Member

What does this PR do?

Adds verify command to osquerybeat check_install spec step. Checks the presence of the essential files that are needed for osquerybeat to function properly. Failed verification indicates to the agent to reinstall osquerybeat.

Why is it important?

This should allow the agent with osquerybeat to recover in case if osquerybeat install became corrupted.
Addresses the final changes that allows to close #30067 (comment)

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas

Related issues

Use cases

Delete any of the essential osquerybeat binaries leaving osquerybeat install corrupted.
The essential files are osquerybeat, osqueryd, osquery-extension.ext (osquery-extension.exe on windows).
Restart the agent, observe the osquerybeat is reinstalled.

@aleksmaus aleksmaus added enhancement backport-v8.0.0 Automated backport with mergify Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team backport-v8.1.0 Automated backport with mergify labels Feb 15, 2022
@aleksmaus aleksmaus self-assigned this Feb 15, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Feb 15, 2022
@botelastic
Copy link

botelastic bot commented Feb 15, 2022

This pull request doesn't have a Team:<team> label.

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-02-15T02:19:02.881+0000

  • Duration: 115 min 31 sec

Test stats 🧪

Test Results
Failed 0
Passed 8562
Skipped 12
Total 8574

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@aleksmaus aleksmaus requested a review from lykkin February 15, 2022 11:47
Copy link
Contributor

@ph ph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ph ph merged commit 1c68693 into elastic:main Feb 15, 2022
mergify bot pushed a commit that referenced this pull request Feb 15, 2022
mergify bot pushed a commit that referenced this pull request Feb 15, 2022
@aleksmaus aleksmaus added the backport-7.17 Automated backport to the 7.17 branch with mergify label Feb 15, 2022
mergify bot pushed a commit that referenced this pull request Feb 15, 2022
(cherry picked from commit 1c68693)

# Conflicts:
#	x-pack/elastic-agent/pkg/agent/program/supported.go
#	x-pack/osquerybeat/cmd/root.go
@amolnater-qasource
Copy link

Hi @aleksmaus
We have revalidated this with Windows 10 host on latest 8.1 Snapshot and found this issue still reproducible.

Build details:
BUILD: 50363
COMMIT: 05408fe74aee0f7161fe4b1568b71e91fb030874
Artifact link: https://snapshots.elastic.co/8.1.0-732c5a10/downloads/beats/elastic-agent/elastic-agent-8.1.0-SNAPSHOT-windows-x86_64.zip

Steps followed:

  1. Installed an agent with policy having System and OS Query Manager integrations.
  2. From the installed folder at C:\Program Files\Elastic\Agent\data\elastic-agent-438761\install\osquerybeat-8.1.0-SNAPSHOT-windows-x86_64 we deleted osquery-extension.exe.
  3. We restarted agent service and observed this deleted binary isn't installed back.

Further we attempted to delete all the necessary binaries however none of them re-installed on agent restart.
12

Logs:
logs.zip

Please let us know if we are missing anything.
Thanks

@aleksmaus
Copy link
Member Author

aleksmaus commented Feb 17, 2022

Hi @aleksmaus We have revalidated this with Windows 10 host on latest 8.1 Snapshot and found this issue still reproducible.

Please let us know if we are missing anything. Thanks

The backport to 8.1 branch is still open and was not merged yet, see the github links to backports above

aleksmaus added a commit that referenced this pull request Feb 17, 2022
…uerybeat (#30405)

* Osquerybeat: Add install verification for osquerybeat (#30388)

(cherry picked from commit 1c68693)

# Conflicts:
#	x-pack/elastic-agent/pkg/agent/program/supported.go
#	x-pack/osquerybeat/cmd/root.go

* Resolve conflicts

Co-authored-by: Aleksandr Maus <[email protected]>
aleksmaus added a commit that referenced this pull request Feb 17, 2022
v1v added a commit to v1v/beats that referenced this pull request Feb 21, 2022
…into feature/use-with-kind-k8s-env

* 'feature/use-with-kind-k8s-env' of github.com:v1v/beats: (52 commits)
  ci: home is declared within withBeatsEnv
  ci: use withKindEnv step
  ci: use getBranchesFromAliases and support next-patch-8 (elastic#30400)
  Update fields.yml (elastic#29609)
  Heartbeat: fix browser metrics and trace mappings (elastic#30258)
  Apply light edits to 8.0 changelog (elastic#30351)
  packetbeat/beater: make sure Npcap installation runs before interfaces are needed (elastic#30396)
  Add a ring-buffer reporter to libbeat (elastic#28750)
  Osquerybeat: Add install verification for osquerybeat (elastic#30388)
  update windows matrix support (elastic#30373)
  Refactor of metricbeat process-gathering metrics and system/process (elastic#30076)
  adjust next changelog wording (elastic#30371)
  [Metricbeat] azure: move event report into loop validDim loop (elastic#29945)
  fix: report GitHub Check before the cache (elastic#30372)
  Add support for non-unique keys in Kafka output headers (elastic#30369)
  ci: 6 major branch reached EOL (elastic#30357)
  reduce Elastic Agent shut down time by stopping processes concurrently (elastic#29650)
  [Filebeat] Add message to register encode/decode debug logs (elastic#30271)
  [libbeat] kafka message header support (elastic#29940)
  Heartbeat: set duration to zero for syntax errors (elastic#30227)
  ...
@amolnater-qasource
Copy link

Hi @aleksmaus
We have revalidated this on 8.0.1 BC-1 build and found it working fine.

  • Installed agent with OSquerybeat.
  • Deleted osquery-extension.exe
  • Restarted agent service and observed OSquerybeat is reinstalled.

Build details:
BUILD: 49342
COMMIT: f4b44d7eb7355c9d1e38d9f2dc753b3fe10c601c
Artifact Link: https://staging.elastic.co/8.0.1-91daef6b/downloads/beats/elastic-agent/elastic-agent-8.0.1-windows-x86_64.zip

We will revalidate this on 8.1, once these merges are available.
Please let us know if anything else is required from our end.
Thanks

aleksmaus added a commit that referenced this pull request Feb 22, 2022
v1v added a commit that referenced this pull request Mar 2, 2022
…-29710

* '8.1' of github.com:elastic/beats: (51 commits)
  refactor pushDockerImages (#30414) (#30624)
  ci: add windows-2022 in the extended meta-stage (#30528) (#30630)
  Curate k8s testing versions to only keep the actively maintained (#30619) (#30625)
  [8.1](backport #30355) Add Beats upgrade docs for 8.0 (#30612)
  Remove references to gcp from the Functionbeat docs (#30579) (#30609)
  x-pack/auditbeat/module/system/socket: defend against exec with zero arguments (#30586) (#30597)
  [MySQL Enterprise] Adding default paths values to manifest.yml (#30598) (#30604)
  metricbeat - fix elasticsearch and kibana integration tests failures in 8.0 (#30566) (#30594)
  Install gawk as a replacement for mawk in Docker containers. (#30452) (#30465)
  [Filebeat] Remove RecordedFuture dataset from Threat Intel module (#30564) (#30568)
  Adjust the documentation of `backoff` options in filestream input (#30552) (#30557)
  packetbeat/beater: help the GC clean up the Npcap installer if it's not used (#30513) (#30546)
  Osquerybeat: Add install verification for osquerybeat (#30388) (#30404)
  Update docker/distribution to 2.8.0 (#30462) (#30540)
  Add `parsers` examples to `filestream` reference configuration (#30529) (#30537)
  [8.1](backport #30068) ZooKeeper module: Adapt to ZooKeeper 3.6+ `mntr` response fields' changes. (#30360)
  [8.1](backport #30512) Switch skip to use `CI` (#30525)
  Forward-port 8.0.1 changelog to 8.1 (#30517)
  packetbeat/beater: don't attempt to install npcap when already installed (#30509) (#30511)
  Add drop and explicit tests to avoid duplicate ingest of elasticsearch logs (#30440) (#30488)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.0.0 Automated backport with mergify backport-v8.1.0 Automated backport with mergify enhancement Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Elastic Agent: Child processes management issues, beats uncompleted uninstall, skipped/corrupted install.
5 participants