Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http_endpoint: 7.14.0 breaks backwards compatibility by encoding numbers as strings #27382

Closed
adriansr opened this issue Aug 14, 2021 · 2 comments · Fixed by #27480
Closed

http_endpoint: 7.14.0 breaks backwards compatibility by encoding numbers as strings #27382

adriansr opened this issue Aug 14, 2021 · 2 comments · Fixed by #27480

Comments

@adriansr
Copy link
Contributor

For confirmed bugs, please report:

  • Version: 7.14.0
  • Operating System: -
  • Discuss Forum URL: -
  • Steps to Reproduce: -

Run filebeat with the following conf:

# filebeat.yml snippet
filebeat.inputs:
- type: http_endpoint
  listen_address: localhost
  listen_port: 9999
$ ./filebeat run -e -N -d '*'

Ingest a document containing a number:

$ curl -XPOST 'http://localhost:9999/' -H 'Content-Type: application/json' --data '{"a":42}'

The following document is produced:

2021-08-14T11:20:13.886+0200	DEBUG	[processors]	processing/processors.go:203	Publish event: {
  "@timestamp": "2021-08-14T09:20:13.886Z",
  "@metadata": {
    "beat": "filebeat",
    "type": "_doc",
    "version": "8.0.0"
  },
  "json": {
    "a": "42"
  },
  "input": {
    "type": "http_endpoint"
  },
  [...]
}

This can break existing ingest pipelines that are expecting numbers instead of strings.

This is caused by #26279 changing the input to use Decoder.UseNumber flag.

@elasticmachine
Copy link
Collaborator

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@andrewkroh
Copy link
Member

I guess it's missing a call to

func TransformNumbers(dict common.MapStr) {
.

adriansr added a commit to adriansr/beats that referenced this issue Aug 18, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes elastic#27382
adriansr added a commit that referenced this issue Aug 19, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes #27382
mergify bot pushed a commit that referenced this issue Aug 19, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes #27382

(cherry picked from commit 704007d)
mergify bot pushed a commit that referenced this issue Aug 19, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes #27382

(cherry picked from commit 704007d)
mergify bot pushed a commit that referenced this issue Aug 19, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes #27382

(cherry picked from commit 704007d)
adriansr added a commit that referenced this issue Aug 19, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes #27382

(cherry picked from commit 704007d)

Co-authored-by: Adrian Serrano <[email protected]>
adriansr added a commit that referenced this issue Aug 19, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes #27382

(cherry picked from commit 704007d)

Co-authored-by: Adrian Serrano <[email protected]>
adriansr added a commit that referenced this issue Aug 19, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes #27382

(cherry picked from commit 704007d)

Co-authored-by: Adrian Serrano <[email protected]>
Icedroid pushed a commit to Icedroid/beats that referenced this issue Nov 1, 2021
This adds a necessary transform to the output of http_endpoint
so that numeric values are encoded as such instead of as a
string.

Fixes elastic#27382
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants