breaking change in 7.6.0 - boolean custom headers that were set as non-string values must now be set as string or Kibana refuses to start

[geekpete]

Kibana version: 7.6.0,7.6.1,7.6.2

• Appears to have arrived in 7.6.0 and later.
• Not reproducible on 7.5.2.

Original install method (e.g. download page, yum, from source, etc.):

Describe the bug:

A breaking change has occurred in a minor release where custom header settings that are set as boolean values must now be only defined as strings.
This will probably affect all non-string values in the same way.

Steps to reproduce:

1. Configure Kibana yaml config file with a custom header that's a boolean:

server.customResponseHeaders: {
    my-custom-boolean-header: true
}

2. Start Kibana

Kibana will log errors about expected value of type [string] but got [boolean]

Expected behavior:

If there's going to be a change to only accept string values for all custom headers then deprecate the ability to use non-string values in this major version and throw deprecation warnings then make the mandatory change in the next major version.
Avoid breaking change in a minor version release.

Screenshots (if relevant):

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

  log   [02:34:22.333] [fatal][root] { Error: [config validation of [server].customResponseHeaders.my-custom-boolean-header]: expected value of type [string] but got [boolean]
    at ObjectType.validate (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/@kbn/config-schema/target/out/types/type.js:59:19)
    at ConfigService.validateAtPath (/home/peter/Apps/kibana-7.6.0-linux-x86_64/src/core/server/config/config_service.js:188:19)
    at MapSubscriber.getDistinctConfig.pipe.config [as project] (/home/peter/Apps/kibana-7.6.0-linux-x86_64/src/core/server/config/config_service.js:196:81)
    at MapSubscriber._next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/operators/map.js:49:35)
    at MapSubscriber.Subscriber.next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Subscriber.js:66:18)
    at DistinctUntilChangedSubscriber._next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/operators/distinctUntilChanged.js:69:30)
    at DistinctUntilChangedSubscriber.Subscriber.next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Subscriber.js:66:18)
    at MapSubscriber._next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/operators/map.js:55:26)
    at MapSubscriber.Subscriber.next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Subscriber.js:66:18)
    at ReplaySubject._subscribe (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/ReplaySubject.js:76:28)
    at ReplaySubject.Observable._trySubscribe (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Observable.js:44:25)
    at ReplaySubject.Subject._trySubscribe (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Subject.js:102:51)
    at ReplaySubject.Observable.subscribe (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Observable.js:30:22)
    at MapSubscriber.shareReplayOperation (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/operators/shareReplay.js:44:32)
    at Observable.subscribe (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Observable.js:25:31)
    at MapOperator.call (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/operators/map.js:32:23)
  cause:
   { Error: expected value of type [string] but got [boolean]
       at RecordOfType.handleError (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/@kbn/config-schema/target/out/types/record_type.js:44:23)
       at RecordOfType.onError (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/@kbn/config-schema/target/out/types/type.js:77:40)
       at type.Type.schema.error (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/@kbn/config-schema/target/out/types/type.js:49:53)
       at finish (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/joi/lib/types/any/index.js:529:50)
       at type._validate (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/joi/lib/types/any/index.js:656:28)
       at type._base (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/joi/lib/types/object/index.js:212:45)
       at type._validate (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/joi/lib/types/any/index.js:614:37)
       at type._validateWithOptions (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/joi/lib/types/any/index.js:674:29)
       at module.exports.internals.Any.root.validate (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/joi/lib/index.js:146:23)
       at ObjectType.validate (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/@kbn/config-schema/target/out/types/type.js:54:72)
       at ConfigService.validateAtPath (/home/peter/Apps/kibana-7.6.0-linux-x86_64/src/core/server/config/config_service.js:188:19)
       at MapSubscriber.getDistinctConfig.pipe.config [as project] (/home/peter/Apps/kibana-7.6.0-linux-x86_64/src/core/server/config/config_service.js:196:81)
       at MapSubscriber._next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/operators/map.js:49:35)
       at MapSubscriber.Subscriber.next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Subscriber.js:66:18)
       at DistinctUntilChangedSubscriber._next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/operators/distinctUntilChanged.js:69:30)
       at DistinctUntilChangedSubscriber.Subscriber.next (/home/peter/Apps/kibana-7.6.0-linux-x86_64/node_modules/rxjs/internal/Subscriber.js:66:18)
     cause: undefined,
     path: [ 'customResponseHeaders', 'my-custom-boolean-header' ] } }

 FATAL  Error: [config validation of [server].customResponseHeaders.my-custom-boolean-header]: expected value of type [string] but got [boolean]

Any additional context:

[elasticmachine]

Pinging @elastic/kibana-platform (Team:Platform)

[pgayvallet]

I confirm that the change was introduced in 7.6

Legacy schema was accepting anything:

      customResponseHeaders: Joi.object()
        .unknown(true)
        .default({}),

When migrating to the Kibana platform, we changed that to only accept strings

      customResponseHeaders: schema.recordOf(schema.string(), schema.string(), {
        defaultValue: {},
      }),

For BWC, we could change this back to schema.recordOf(schema.string(), schema.any()) and stringify the values in the HttpConfig constructor.

[joshdover]

Let's fix this in 7.7.1+ and add a known issue + workaround in the 7.6.0, 7.6.1, 7.6.2, and 7.7.0 release notes. Example: https://www.elastic.co/guide/en/kibana/current/release-notes-7.6.0.html

[geekpete]

Tested as affecting 7.7.0 as well.
Also seems to be triggered with any non-string value:

server.customResponseHeaders: {
    my-custom-boolean-header: 200
}

log [23:42:53.200] [fatal][root] { Error: [config validation of [server].customResponseHeaders.my-custom-boolean-header]: expected value of type [string] but got [number]

server.customResponseHeaders: {
    my-custom-boolean-header: {"test":"test"}
}

log [23:44:18.621] [fatal][root] { Error: [config validation of [server].customResponseHeaders.my-custom-boolean-header]: expected value of type [string] but got [Object]

[pgayvallet]

Release notes:
7.6.0, 7.6.1, 7.6.2: #66690
7.7.0: #66693

Fix:
master: #66689
7.x: #66807
7.8: #66808
7.7.1: #66809