Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[filebeat][streaming] - Added retry functionality to websocket connections #40601

Merged
merged 12 commits into from
Aug 28, 2024
Merged

[filebeat][streaming] - Added retry functionality to websocket connections #40601

merged 12 commits into from
Aug 28, 2024

Conversation

ShourieG
Copy link
Contributor

Type of change

Please label this PR with one of the following labels, depending on the scope of your change:

  • Enhancement
  • Docs

Proposed commit message

Added retry functionality to websocket connections under the streaming input. Docs & tests have been updated accordingly to support the new functionality.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Disruptive User Impact

None

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@ShourieG ShourieG requested a review from a team as a code owner August 23, 2024 12:48
@ShourieG ShourieG requested review from rdner and mauri870 August 23, 2024 12:48
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 23, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Aug 23, 2024

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @ShourieG? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@ShourieG ShourieG added the Team:Security-Service Integrations Security Service Integrations Team label Aug 23, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 23, 2024
@ShourieG ShourieG added needs_team Indicates that the issue/PR needs a Team:* label 8.16-candidate labels Aug 23, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 23, 2024
@ShourieG ShourieG added enhancement needs_team Indicates that the issue/PR needs a Team:* label labels Aug 23, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 23, 2024
@ShourieG ShourieG added needs_team Indicates that the issue/PR needs a Team:* label input:streaming labels Aug 23, 2024
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 23, 2024
@ShourieG ShourieG requested review from a team and efd6 August 23, 2024 13:07
@ShourieG
Copy link
Contributor Author

@rdner, can I get a CODEOWNER review on this.

@pierrehilbert
Copy link
Collaborator

@mauri870 is part of the code owner so his approval would work.
I'm adding someone else as a reviewer as @rdner is off this week

@pierrehilbert pierrehilbert requested a review from belimawr August 26, 2024 08:46
@mauri870
Copy link
Member

Thanks for the ping. I've reviewed the PR, and it looks good.

belimawr
belimawr requested changes Aug 26, 2024
View reviewed changes
x-pack/filebeat/docs/inputs/input-streaming.asciidoc Outdated Show resolved Hide resolved
x-pack/filebeat/input/streaming/config.go Outdated Show resolved Hide resolved
x-pack/filebeat/input/streaming/websocket.go Outdated Show resolved Hide resolved
x-pack/filebeat/input/streaming/websocket.go Outdated Show resolved Hide resolved
x-pack/filebeat/input/streaming/websocket.go Outdated Show resolved Hide resolved
@pierrehilbert pierrehilbert added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Aug 27, 2024
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@ShourieG
Copy link
Contributor Author

@belimawr, Addressed the PR comments.

mauri870
mauri870 reviewed Aug 27, 2024
View reviewed changes
x-pack/filebeat/input/streaming/websocket.go
var closeErr *websocket.CloseError
if errors.As(err, &closeErr) {
switch closeErr.Code {
case websocket.CloseGoingAway,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to use https://pkg.go.dev/github.com/gorilla/websocket#IsCloseError here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IsCloseError will require us to pass all the possible codes in the func param which can seem a bit messy when we are dealing with so many types of codes. It can be used but the signature is messy compared to the switch statement.

@ShourieG ShourieG merged commit 0c3c9c6 into elastic:main Aug 28, 2024
20 checks passed
@ShourieG ShourieG deleted the websocket/add_retry branch August 28, 2024 09:04
@ShourieG ShourieG added the backport-8.15 Automated backport to the 8.15 branch with mergify label Dec 12, 2024
mergify bot pushed a commit that referenced this pull request Dec 12, 2024
@ShourieG @mergify
[filebeat][streaming] - Added retry functionality to websocket connec…
e95361d 
…tions (#40601)

* added websocket retry logic, added input & config tests and updated docs

* updated changelog

* fixed function name spelling error

* added a retryable error check

* addressed PR comments

* passed metrics to handleConnectionResponse to track errors

* addressed PR suggestions

* updated retry dial signature

(cherry picked from commit 0c3c9c6)

# Conflicts:
#	x-pack/filebeat/input/streaming/websocket.go
@mergify mergify bot mentioned this pull request Dec 12, 2024
Closed
5 tasks
@ShourieG ShourieG removed the backport-8.15 Automated backport to the 8.15 branch with mergify label Dec 12, 2024
This was referenced Dec 13, 2024
Merged
Merged
This was referenced Dec 16, 2024
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mauri870 mauri870 mauri870 approved these changes

@belimawr belimawr belimawr approved these changes

@rdner rdner Awaiting requested review from rdner rdner was automatically assigned from elastic/elastic-agent-data-plane

@efd6 efd6 Awaiting requested review from efd6

Assignees

@ShourieG ShourieG

Labels
8.16-candidate enhancement input:streaming Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team Team:Security-Service Integrations Security Service Integrations Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[filebeat][websocket] - Add retry mechanism to socket connections
6 participants
@ShourieG @elasticmachine @pierrehilbert @mauri870 @belimawr @efd6