Incorrect data types in Fleet managed ingest pipelines

[andrewkroh]

Kibana version:

{
  "version": {
    "number": "7.11.0",
    "build_hash": "1a6d7308e58787bf0448f65c039739b40d4a6e3d",
    "build_number": 37074,
    "build_snapshot": true
  }
}

Elasticsearch version:

{
  "version": {
    "number": "7.11.0-SNAPSHOT",
    "build_flavor": "default",
    "build_type": "docker",
    "build_hash": "279bf21e59fcbac1135922b4828f82578f266234",
    "build_date": "2020-12-08T15:51:24.143094Z"
  }
}

Server OS version: Docker

Browser version: Firefox 83.0

Browser OS version: MacOS

Original install method (e.g. download page, yum, from source, etc.): Docker

Describe the bug:

Steps to reproduce:

1. Setup Fleet.
2. Install Palo Alto integration.
3. View the ingest node pipeline with
   GET _ingest/pipeline/logs-panw.panos-*
4. Find the script processor with several params.

5. Compare those `params` values to the definition contained in the package definition. Note the JSON contains strings, but the pipeline source contains numbers. You can download the package from the package registry `curl -L -O https://epr.elastic.co/epr/panw/panw-0.3.2.zip`. The pipeline is in default.yml.

Expected behavior:
The ingest node pipeline in Elasticsearch should contain numeric values for params. My assumption is that there's an issue with the YAML to JSON conversion, but I'm not sure. What I expect to see is this conversion output (where JSON has numbers).

Any additional context:

In this instance the incorrect data types cause problems within the Painless script. But this could cause problems elsewhere too. The error we see from this pipeline is

Cannot convert [java.lang.String] to an integral value.

[elasticmachine]

Pinging @elastic/ingest-management (Feature:Fleet)

[ruflin]

@skh Could you take a look at this?

[skh]

I can reproduce the problem, but I also see it when I add the ingest pipeline manually with

curl -u elastic:changeme -X PUT "localhost:9200/_ingest/pipeline/my-pipeline-id?pretty" -H 'Content-Type: application/yaml' --data-binary @/home/skh/tmp/panw-0.3.2/data_stream/panos/elasticsearch/ingest_pipeline/default.yml

Can you check if adding the pipeline with curl works for you? Am I missing something?

[andrewkroh]

I can produce the same issue with the Elasticsearch API directly. @skh, so Fleet is directly passing YAML content to the _ingest/pipeline API?

[skh]

Fleet is directly passing YAML content to the _ingest/pipeline API?

Mostly, yes. We do insert references to other pipelines, if necessary, but other than that we just pass it through.

[skh]

Closing in favor of elastic/elasticsearch#66555 , please reopen if necessary.