Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Google Cloud Storage] Add support for alternative_host for system tests #34413

Merged
merged 3 commits into from
Jan 30, 2023

Conversation

P1llus
Copy link
Member

@P1llus P1llus commented Jan 30, 2023

What does this PR do?

This adds the alternative_host option, which is a field only used for internal testing. By default the input only reaches out to the public cloud GCS API, and this allows us to override this and instead point it to our own emulator running locally for system tests.

Why is it important?

Required for adding system tests for elastic-agent integration packages.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

How to test this PR locally

  1. Start up the local GCS emulator: docker run -p 4443:4443 fsouza/fake-gcs-server -host="0.0.0.0" -public-host=localhost -port=4443 -scheme=http
  2. Download stream and run go build : https://github.com/elastic/stream
  3. Create a local file for testing, for example test.json, add a JSON object on a single line, like { "testmessage" : "success" }
  4. Populate the emulator with the testdata using stream: ./stream log --retry=30 --addr=http://localhost:4443 -p gcs --gcs-content-type="application/json" ./test.json
  5. Test filebeat with following filebeat.yml
filebeat.inputs:
- type: gcs
  project_id: testproject
  parse_json: true
  alternative_host: "http://localhost:4443/"
  buckets:
  - name: testbucket

The expected result is that a the object is retrieved and that parse_json works as expected

@P1llus P1llus requested a review from a team as a code owner January 30, 2023 10:09
@P1llus P1llus requested review from cmacknz and leehinman and removed request for a team January 30, 2023 10:09
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 30, 2023
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 30, 2023
@mergify mergify bot assigned P1llus Jan 30, 2023
@P1llus P1llus requested a review from ShourieG January 30, 2023 10:18
Copy link
Contributor

@ShourieG ShourieG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@elasticmachine
Copy link
Collaborator

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-01-30T10:11:42.402+0000

  • Duration: 71 min 45 sec

Test stats 🧪

Test Results
Failed 0
Passed 2547
Skipped 172
Total 2719

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@P1llus P1llus merged commit d1ef41c into elastic:main Jan 30, 2023
@elasticmachine
Copy link
Collaborator

❕ Build Aborted

There is a new build on-going so the previous on-going builds have been aborted.

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Start Time: 2023-01-30T10:10:40.545+0000

  • Duration: 72 min 20 sec

Test stats 🧪

Test Results
Failed 0
Passed 2547
Skipped 172
Total 2719

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

mergify bot pushed a commit that referenced this pull request Jan 30, 2023
…sts (#34413)

* [Google Cloud Storage] Add support for alternative_host to override HTTP client for system tests

* update changelog

(cherry picked from commit d1ef41c)
P1llus added a commit that referenced this pull request Jan 30, 2023
…tive_host for system tests (#34415)

* [Google Cloud Storage] Add support for alternative_host for system tests (#34413)

* [Google Cloud Storage] Add support for alternative_host to override HTTP client for system tests

* update changelog

(cherry picked from commit d1ef41c)

* Update CHANGELOG.next.asciidoc

---------

Co-authored-by: Marius Iversen <[email protected]>
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
…sts (#34413)

* [Google Cloud Storage] Add support for alternative_host to override HTTP client for system tests

* update changelog
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v8.6.0 Automated backport with mergify enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants