Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add enableGzipHttpCompression and enableBatchLogs options to SplunkIO #24197

Merged
merged 1 commit into from
Nov 17, 2022
Merged

Add enableGzipHttpCompression and enableBatchLogs options to SplunkIO #24197

merged 1 commit into from
Nov 17, 2022

Conversation

andreigurau
Copy link
Contributor

Transferring over some SplunkIO options (enableGzipHttpCompression and enableBatchLogs) from the SplunkIO implementation in DataflowTemplates here, so that DataflowTemplates can use Beam's SplunkIO implementation (there is an inconsistency in that classic templates use the DataflowTemplates implementation while flex templates use Beam's implementation, and Beam's implementation seems to be missing some options).

The rootCaCertificatePath option will be done in a separate PR, since adding that option will require more changes than the above two options.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Choose reviewer(s) and mention them in a comment (R: @username).
  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI.

johnjcasey
johnjcasey reviewed Nov 16, 2022
View reviewed changes
Copy link
Contributor

@johnjcasey johnjcasey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than some cleanup

@github-actions
Copy link
Contributor

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @apilloud for label java.
R: @johnjcasey for label io.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@andreigurau
Add enableGzipHttpCompression option to SplunkIO
5708d5e 
add enableBatchLogs as SplunkIO option

spotless

fix issue with setEnableBatchLogs
@johnjcasey johnjcasey merged commit 8cd94c5 into apache:master Nov 17, 2022
@andreigurau andreigurau deleted the splunk-options branch November 17, 2022 15:37
andreigurau added a commit to andreigurau/beam that referenced this pull request Nov 17, 2022
@andreigurau
Add enableGzipHttpCompression option to SplunkIO (apache#24197)
7ddb5b9 
add enableBatchLogs as SplunkIO option

spotless

fix issue with setEnableBatchLogs
andreigurau added a commit to andreigurau/beam that referenced this pull request Nov 17, 2022
@andreigurau
add rootCaCertificate option to SplunkIO
ef37e9b 
fix test

Add error reporting for BatchConverter match failure (apache#24022)

* add error reporting for BatchConverters

* Test pytorch

* Finish up torch tests

* yapf

* yapf

* Remove else

Update automation to use Go 1.19 (apache#24175)

Co-authored-by: lostluck <[email protected]>

Fix broken json for notebook (apache#24183)

Using Teardown context instead of deprecated finalize (apache#24180)

* Using Teardown context instead of deprecated finalize

* making function public

Co-authored-by: Scott Strong <[email protected]>

[Python]Support pipe operator as Union (PEP -604) (apache#24106)

Fixes apache#21972

Add custom inference function support to the PyTorch model handler (apache#24062)

* Initial type def and function signature

* [Draft] Add custom inference fn support to Pytorch Model Handler

* Formatting

* Split out default

* Remove Keyed version for testing

* Move device optimization

* Make default available for import, add to test classes

* Remove incorrect default from keyed test

* Keyed impl

* Fix device arg

* custom inference test

* formatting

* Add helpers to define custom inference functions using model methods

* Trailing whitespace

* Unit tests

* Fix incorrect getattr syntax

* Type typo

* Fix docstring

* Fix keyed helper, add basic generate route

* Modify generate() to be different than forward()

* formatting

* Remove extra generate() def

Strip FGAC database role from changestreams metadata requests (apache#24177)

Co-authored-by: Doug Judd <[email protected]>

Updated README of Interactive Beam

Removed deprecated cache_dir runner param in favor of the cache_root global option.

Minor update

Fix arguments to checkState in BatchViewOverrides

Re-use serializable pipeline options when already available (apache#24192)

Fix Python PostCommit Example CustomPTransformIT on portable (apache#24159)

* Fix Python PostCommit Examples on portable

* Fix custom_ptransform pipeline options gets modified

* Specify flinkConfDir

revert upgrade to go 1.19 for action unit tests (apache#24189)

Use only ValueProviders in SpannerConfig (apache#24156)

[Tour of Beam] [Frontend] Content tree URLs (apache#23776)

* Content tree navigation (apache#23593)

Unit content navigation (apache#23593)

Update URL on node click (apache#23593)

Active unit color (apache#23593)

removeListener in unit (apache#23593)

First unit is opened on group title click (apache#23593)

WIP by Alexey Inkin (apache#23593)

selectedUnitColor (apache#23593)

Unit borderRadius (apache#23593)

RegExp todo (apache#23593)

added referenced collection package to remove warning (apache#23593)

small refinement (apache#23593)

expand on group tap, padding, openNode (apache#23593)

group expansion bug fix (apache#23593)

selected & unselected progress indicators (apache#23593)

* AnimatedBuilders instead of StatefulWidgets in unit & group (apache#23593)

* fixed _getNodeAncestors (apache#23593)

* get sdkId (apache#23593)

* addressing comments (apache#23593)

* sdkId getter & StatelessExpansionTile (apache#23593)

* expand & collapse group (apache#23593)

* StatelessExpansionTile (apache#23593)

* license (apache#23593)

* ValueChanged and ValueKey in StatelessExpansionTile (apache#23593)

Co-authored-by: darkhan.nausharipov <[email protected]>
Co-authored-by: Alexey Inkin <[email protected]>

refs: issue-24196, fix broken hyperlink

Add a reference to Java RunInference example

Python TextIO Performance Test (apache#23951)

* Python TextIO Performance Test

* Add filebasedio_perf_test module for unified test framework for
  Python file-based IOs

* Fix MetricsReader publishes metrics duplicately if more than one
  load test declared. This is because MetricsReader.publishers was
  static class variable

* Fix pylint

* Distribute Python performance tests random time at a day instead of all at 3PM

* Add information about length conversion

Fix PythonLint (apache#24219)

Bump loader-utils from 1.4.1 to 1.4.2 in /sdks/python/apache_beam/runners/interactive/extensions/apache-beam-jupyterlab-sidepanel (apache#24191)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Temporary update Python RC validation job

updates

updates

Uses _all to follow alias/datastreams when estimating index size

Fixes apache#24117

Adds test for following aliases when estimating index size

Bump github.com/aws/aws-sdk-go-v2/config from 1.18.0 to 1.18.1 in /sdks (apache#24222)

Bumps [github.com/aws/aws-sdk-go-v2/config](https://github.com/aws/aws-sdk-go-v2) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/aws/aws-sdk-go-v2/releases)
- [Changelog](https://github.com/aws/aws-sdk-go-v2/blob/main/CHANGELOG.md)
- [Commits](aws/aws-sdk-go-v2@config/v1.18.0...config/v1.18.1)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go-v2/config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Add enableGzipHttpCompression option to SplunkIO (apache#24197)

add enableBatchLogs as SplunkIO option

spotless

fix issue with setEnableBatchLogs
prodriguezdefino pushed a commit to prodriguezdefino/beam-pabs that referenced this pull request Nov 18, 2022
@andreigurau @prodriguezdefino
Add enableGzipHttpCompression option to SplunkIO (apache#24197)
1033975 
add enableBatchLogs as SplunkIO option

spotless

fix issue with setEnableBatchLogs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@johnjcasey johnjcasey johnjcasey approved these changes

Assignees
No one assigned
Labels
io java Next Action: Reviewers
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@andreigurau @johnjcasey