[Connectors] Check connector's responses #115797

cnasikas · 2021-10-20T15:16:39Z

Summary

This PR adds validation to the responses of the connectors. Specifically, it checks a) if the response is a JSON and b) if certain fields are defined in the response.

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

kibanamachine · 2021-10-20T15:50:34Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: 6fd2cdc
Storybooks Preview
Pipeline Steps (look for red circles / failed steps)
Interpreting CI Failures

Failed CI Steps

Test Failures

Kibana Pipeline / general / X-Pack Alerting API Integration Tests.x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/jira·ts.alerting api integration security and spaces enabled Actions Jira Jira - Executor Execution should handle creating an incident without comments

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://dryrun

[00:00:00]     │
[00:00:00]       └-: alerting api integration security and spaces enabled
[00:00:00]         └-> "before all" hook in "alerting api integration security and spaces enabled"
[00:00:00]         └-: Actions
[00:00:00]           └-> "before all" hook in "Actions"
[00:00:00]           └-> "before all" hook in "Actions"
[00:00:00]             │ debg creating space
[00:00:01]             │ debg created space
[00:00:01]             │ debg creating space
[00:00:02]             │ debg created space
[00:00:02]             │ debg creating space
[00:00:03]             │ debg created space
[00:00:03]             │ debg creating user no_kibana_privileges
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [no_kibana_privileges]
[00:00:03]             │ debg created user no_kibana_privileges
[00:00:03]             │ debg creating role no_kibana_privileges
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [no_kibana_privileges]
[00:00:03]             │ debg creating user superuser
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [superuser]
[00:00:03]             │ debg created user superuser
[00:00:03]             │ debg creating user global_read
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [global_read]
[00:00:03]             │ debg created user global_read
[00:00:03]             │ debg creating role global_read_role
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [global_read_role]
[00:00:03]             │ debg creating user space_1_all
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all]
[00:00:03]             │ debg created user space_1_all
[00:00:03]             │ debg creating role space_1_all_role
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_role]
[00:00:03]             │ debg creating user space_1_all_with_restricted_fixture
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all_with_restricted_fixture]
[00:00:03]             │ debg created user space_1_all_with_restricted_fixture
[00:00:03]             │ debg creating role space_1_all_with_restricted_fixture_role
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_with_restricted_fixture_role]
[00:00:03]             │ debg creating user space_1_all_alerts_none_actions
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all_alerts_none_actions]
[00:00:03]             │ debg created user space_1_all_alerts_none_actions
[00:00:03]             │ debg creating role space_1_all_alerts_none_actions_role
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_alerts_none_actions_role]
[00:00:34]           └-: Jira
[00:00:34]             └-> "before all" hook in "Jira"
[00:00:34]             └-> "before all" hook in "Jira"
[00:00:35]             └-: Jira - Executor
[00:00:35]               └-> "before all" hook in "Jira - Executor"
[00:00:35]               └-> "before all" hook in "Jira - Executor"
[00:00:36]               └-: Execution
[00:00:36]                 └-> "before all" hook for "should handle creating an incident without comments"
[00:00:36]                 └-> should handle creating an incident without comments
[00:00:36]                   └-> "before each" hook: global before each for "should handle creating an incident without comments"
[00:00:36]                   │ proc [kibana] [2021-10-20T15:50:29.307+00:00][INFO ][plugins.eventLog] event logged: {"@timestamp":"2021-10-20T15:50:29.306Z","event":{"provider":"actions","action":"execute-start","start":"2021-10-20T15:50:29.305Z"},"kibana":{"saved_objects":[{"rel":"primary","type":"action","id":"7286f570-31bd-11ec-94e5-fb98c1ab9f29","type_id":".jira"}],"server_uuid":"5b2de169-2785-441b-ae8c-186a1936b17d","version":"8.0.0"},"message":"action started: .jira:7286f570-31bd-11ec-94e5-fb98c1ab9f29: A jira simulator","ecs":{"version":"1.8.0"}}
[00:00:36]                   │ proc [kibana] [2021-10-20T15:50:29.329+00:00][WARN ][plugins.actions] action execution failure: .jira:7286f570-31bd-11ec-94e5-fb98c1ab9f29: A jira simulator: an error occurred while running the action executor: [Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading 'created') Reason: unknown. Reason: unknown
[00:00:36]                   │ proc [kibana] [2021-10-20T15:50:29.330+00:00][INFO ][plugins.eventLog] event logged: {"@timestamp":"2021-10-20T15:50:29.329Z","event":{"provider":"actions","action":"execute","start":"2021-10-20T15:50:29.305Z","end":"2021-10-20T15:50:29.329Z","duration":24000000,"outcome":"failure"},"kibana":{"saved_objects":[{"rel":"primary","type":"action","id":"7286f570-31bd-11ec-94e5-fb98c1ab9f29","type_id":".jira"}],"server_uuid":"5b2de169-2785-441b-ae8c-186a1936b17d","version":"8.0.0"},"message":"action execution failure: .jira:7286f570-31bd-11ec-94e5-fb98c1ab9f29: A jira simulator","error":{"message":"an error occurred while running the action executor: [Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading 'created') Reason: unknown. Reason: unknown"},"ecs":{"version":"1.8.0"}}
[00:00:36]                   └- ✖ fail: alerting api integration security and spaces enabled Actions Jira Jira - Executor Execution should handle creating an incident without comments
[00:00:36]                   │       Error: expected { status: 'error',
[00:00:36]                   │   message: 'an error occurred while running the action executor',
[00:00:36]                   │   retry: false,
[00:00:36]                   │   connector_id: '7286f570-31bd-11ec-94e5-fb98c1ab9f29',
[00:00:36]                   │   service_message: '[Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading \'created\') Reason: unknown. Reason: unknown' } to sort of equal { status: 'ok',
[00:00:36]                   │   connector_id: '7286f570-31bd-11ec-94e5-fb98c1ab9f29',
[00:00:36]                   │   data: 
[00:00:36]                   │    { id: '123',
[00:00:36]                   │      title: 'CK-1',
[00:00:36]                   │      pushedDate: '2020-04-27T14:17:45.490Z',
[00:00:36]                   │      url: 'http://elastic:changeme@localhost:61121/api/_actions-FTS-external-service-simulators/jira/browse/CK-1' } }
[00:00:36]                   │       + expected - actual
[00:00:36]                   │ 
[00:00:36]                   │        {
[00:00:36]                   │          "connector_id": "7286f570-31bd-11ec-94e5-fb98c1ab9f29"
[00:00:36]                   │       -  "message": "an error occurred while running the action executor"
[00:00:36]                   │       -  "retry": false
[00:00:36]                   │       -  "service_message": "[Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading 'created') Reason: unknown. Reason: unknown"
[00:00:36]                   │       -  "status": "error"
[00:00:36]                   │       +  "data": {
[00:00:36]                   │       +    "id": "123"
[00:00:36]                   │       +    "pushedDate": "2020-04-27T14:17:45.490Z"
[00:00:36]                   │       +    "title": "CK-1"
[00:00:36]                   │       +    "url": "http://elastic:changeme@localhost:61121/api/_actions-FTS-external-service-simulators/jira/browse/CK-1"
[00:00:36]                   │       +  }
[00:00:36]                   │       +  "status": "ok"
[00:00:36]                   │        }
[00:00:36]                   │       
[00:00:36]                   │       at Assertion.assert (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:00:36]                   │       at Assertion.eql (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:244:8)
[00:00:36]                   │       at Context.<anonymous> (test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/jira.ts:403:27)
[00:00:36]                   │       at runMicrotasks (<anonymous>)
[00:00:36]                   │       at processTicksAndRejections (node:internal/process/task_queues:96:5)
[00:00:36]                   │       at Object.apply (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:00:36]                   │ 
[00:00:36]                   │

Stack Trace

Error: expected { status: 'error',
  message: 'an error occurred while running the action executor',
  retry: false,
  connector_id: '7286f570-31bd-11ec-94e5-fb98c1ab9f29',
  service_message: '[Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading \'created\') Reason: unknown. Reason: unknown' } to sort of equal { status: 'ok',
  connector_id: '7286f570-31bd-11ec-94e5-fb98c1ab9f29',
  data: 
   { id: '123',
     title: 'CK-1',
     pushedDate: '2020-04-27T14:17:45.490Z',
     url: 'http://elastic:changeme@localhost:61121/api/_actions-FTS-external-service-simulators/jira/browse/CK-1' } }
    at Assertion.assert (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:244:8)
    at Context.<anonymous> (test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/jira.ts:403:27)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.apply (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16) {
  actual: '{\n' +
    '  "connector_id": "7286f570-31bd-11ec-94e5-fb98c1ab9f29"\n' +
    '  "message": "an error occurred while running the action executor"\n' +
    '  "retry": false\n' +
    `  "service_message": "[Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading 'created') Reason: unknown. Reason: unknown"\n` +
    '  "status": "error"\n' +
    '}',
  expected: '{\n' +
    '  "connector_id": "7286f570-31bd-11ec-94e5-fb98c1ab9f29"\n' +
    '  "data": {\n' +
    '    "id": "123"\n' +
    '    "pushedDate": "2020-04-27T14:17:45.490Z"\n' +
    '    "title": "CK-1"\n' +
    '    "url": "http://elastic:changeme@localhost:61121/api/_actions-FTS-external-service-simulators/jira/browse/CK-1"\n' +
    '  }\n' +
    '  "status": "ok"\n' +
    '}',
  showDiff: true
}

Kibana Pipeline / general / X-Pack Alerting API Integration Tests.x-pack/test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/jira·ts.alerting api integration security and spaces enabled Actions Jira Jira - Executor Execution should handle creating an incident without comments

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]     │
[00:00:00]       └-: alerting api integration security and spaces enabled
[00:00:00]         └-> "before all" hook in "alerting api integration security and spaces enabled"
[00:00:00]         └-: Actions
[00:00:00]           └-> "before all" hook in "Actions"
[00:00:00]           └-> "before all" hook in "Actions"
[00:00:00]             │ debg creating space
[00:00:00]             │ debg created space
[00:00:00]             │ debg creating space
[00:00:01]             │ debg created space
[00:00:01]             │ debg creating space
[00:00:02]             │ debg created space
[00:00:02]             │ debg creating user no_kibana_privileges
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [no_kibana_privileges]
[00:00:03]             │ debg created user no_kibana_privileges
[00:00:03]             │ debg creating role no_kibana_privileges
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [no_kibana_privileges]
[00:00:03]             │ debg creating user superuser
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [superuser]
[00:00:03]             │ debg created user superuser
[00:00:03]             │ debg creating user global_read
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [global_read]
[00:00:03]             │ debg created user global_read
[00:00:03]             │ debg creating role global_read_role
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [global_read_role]
[00:00:03]             │ debg creating user space_1_all
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all]
[00:00:03]             │ debg created user space_1_all
[00:00:03]             │ debg creating role space_1_all_role
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_role]
[00:00:03]             │ debg creating user space_1_all_with_restricted_fixture
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all_with_restricted_fixture]
[00:00:03]             │ debg created user space_1_all_with_restricted_fixture
[00:00:03]             │ debg creating role space_1_all_with_restricted_fixture_role
[00:00:03]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_with_restricted_fixture_role]
[00:00:03]             │ debg creating user space_1_all_alerts_none_actions
[00:00:03]             │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all_alerts_none_actions]
[00:00:03]             │ debg created user space_1_all_alerts_none_actions
[00:00:03]             │ debg creating role space_1_all_alerts_none_actions_role
[00:00:04]             │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_alerts_none_actions_role]
[00:00:32]           └-: Jira
[00:00:32]             └-> "before all" hook in "Jira"
[00:00:32]             └-> "before all" hook in "Jira"
[00:00:33]             └-: Jira - Executor
[00:00:33]               └-> "before all" hook in "Jira - Executor"
[00:00:33]               └-> "before all" hook in "Jira - Executor"
[00:00:34]               └-: Execution
[00:00:34]                 └-> "before all" hook for "should handle creating an incident without comments"
[00:00:34]                 └-> should handle creating an incident without comments
[00:00:34]                   └-> "before each" hook: global before each for "should handle creating an incident without comments"
[00:00:34]                   │ proc [kibana] [2021-10-20T15:48:10.901+00:00][INFO ][plugins.eventLog] event logged: {"@timestamp":"2021-10-20T15:48:10.899Z","event":{"provider":"actions","action":"execute-start","start":"2021-10-20T15:48:10.899Z"},"kibana":{"saved_objects":[{"rel":"primary","type":"action","id":"20010340-31bd-11ec-b243-cbdd5c712aea","type_id":".jira"}],"server_uuid":"5b2de169-2785-441b-ae8c-186a1936b17d","version":"8.0.0"},"message":"action started: .jira:20010340-31bd-11ec-b243-cbdd5c712aea: A jira simulator","ecs":{"version":"1.8.0"}}
[00:00:34]                   │ proc [kibana] [2021-10-20T15:48:10.915+00:00][WARN ][plugins.actions] action execution failure: .jira:20010340-31bd-11ec-b243-cbdd5c712aea: A jira simulator: an error occurred while running the action executor: [Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading 'created') Reason: unknown. Reason: unknown
[00:00:34]                   │ proc [kibana] [2021-10-20T15:48:10.916+00:00][INFO ][plugins.eventLog] event logged: {"@timestamp":"2021-10-20T15:48:10.915Z","event":{"provider":"actions","action":"execute","start":"2021-10-20T15:48:10.899Z","end":"2021-10-20T15:48:10.915Z","duration":16000000,"outcome":"failure"},"kibana":{"saved_objects":[{"rel":"primary","type":"action","id":"20010340-31bd-11ec-b243-cbdd5c712aea","type_id":".jira"}],"server_uuid":"5b2de169-2785-441b-ae8c-186a1936b17d","version":"8.0.0"},"message":"action execution failure: .jira:20010340-31bd-11ec-b243-cbdd5c712aea: A jira simulator","error":{"message":"an error occurred while running the action executor: [Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading 'created') Reason: unknown. Reason: unknown"},"ecs":{"version":"1.8.0"}}
[00:00:34]                   └- ✖ fail: alerting api integration security and spaces enabled Actions Jira Jira - Executor Execution should handle creating an incident without comments
[00:00:34]                   │       Error: expected { status: 'error',
[00:00:34]                   │   message: 'an error occurred while running the action executor',
[00:00:34]                   │   retry: false,
[00:00:34]                   │   connector_id: '20010340-31bd-11ec-b243-cbdd5c712aea',
[00:00:34]                   │   service_message: '[Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading \'created\') Reason: unknown. Reason: unknown' } to sort of equal { status: 'ok',
[00:00:34]                   │   connector_id: '20010340-31bd-11ec-b243-cbdd5c712aea',
[00:00:34]                   │   data: 
[00:00:34]                   │    { id: '123',
[00:00:34]                   │      title: 'CK-1',
[00:00:34]                   │      pushedDate: '2020-04-27T14:17:45.490Z',
[00:00:34]                   │      url: 'http://elastic:changeme@localhost:61121/api/_actions-FTS-external-service-simulators/jira/browse/CK-1' } }
[00:00:34]                   │       + expected - actual
[00:00:34]                   │ 
[00:00:34]                   │        {
[00:00:34]                   │          "connector_id": "20010340-31bd-11ec-b243-cbdd5c712aea"
[00:00:34]                   │       -  "message": "an error occurred while running the action executor"
[00:00:34]                   │       -  "retry": false
[00:00:34]                   │       -  "service_message": "[Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading 'created') Reason: unknown. Reason: unknown"
[00:00:34]                   │       -  "status": "error"
[00:00:34]                   │       +  "data": {
[00:00:34]                   │       +    "id": "123"
[00:00:34]                   │       +    "pushedDate": "2020-04-27T14:17:45.490Z"
[00:00:34]                   │       +    "title": "CK-1"
[00:00:34]                   │       +    "url": "http://elastic:changeme@localhost:61121/api/_actions-FTS-external-service-simulators/jira/browse/CK-1"
[00:00:34]                   │       +  }
[00:00:34]                   │       +  "status": "ok"
[00:00:34]                   │        }
[00:00:34]                   │       
[00:00:34]                   │       at Assertion.assert (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:00:34]                   │       at Assertion.eql (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:244:8)
[00:00:34]                   │       at Context.<anonymous> (test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/jira.ts:403:27)
[00:00:34]                   │       at runMicrotasks (<anonymous>)
[00:00:34]                   │       at processTicksAndRejections (node:internal/process/task_queues:96:5)
[00:00:34]                   │       at Object.apply (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:00:34]                   │ 
[00:00:34]                   │

Stack Trace

Error: expected { status: 'error',
  message: 'an error occurred while running the action executor',
  retry: false,
  connector_id: '20010340-31bd-11ec-b243-cbdd5c712aea',
  service_message: '[Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading \'created\') Reason: unknown. Reason: unknown' } to sort of equal { status: 'ok',
  connector_id: '20010340-31bd-11ec-b243-cbdd5c712aea',
  data: 
   { id: '123',
     title: 'CK-1',
     pushedDate: '2020-04-27T14:17:45.490Z',
     url: 'http://elastic:changeme@localhost:61121/api/_actions-FTS-external-service-simulators/jira/browse/CK-1' } }
    at Assertion.assert (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:244:8)
    at Context.<anonymous> (test/alerting_api_integration/security_and_spaces/tests/actions/builtin_action_types/jira.ts:403:27)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.apply (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16) {
  actual: '{\n' +
    '  "connector_id": "20010340-31bd-11ec-b243-cbdd5c712aea"\n' +
    '  "message": "an error occurred while running the action executor"\n' +
    '  "retry": false\n' +
    `  "service_message": "[Action][Jira]: Unable to create incident. Error: [Action][Jira]: Unable to get incident with id 123. Error: Cannot read properties of undefined (reading 'created') Reason: unknown. Reason: unknown"\n` +
    '  "status": "error"\n' +
    '}',
  expected: '{\n' +
    '  "connector_id": "20010340-31bd-11ec-b243-cbdd5c712aea"\n' +
    '  "data": {\n' +
    '    "id": "123"\n' +
    '    "pushedDate": "2020-04-27T14:17:45.490Z"\n' +
    '    "title": "CK-1"\n' +
    '    "url": "http://elastic:changeme@localhost:61121/api/_actions-FTS-external-service-simulators/jira/browse/CK-1"\n' +
    '  }\n' +
    '  "status": "ok"\n' +
    '}',
  showDiff: true
}

Kibana Pipeline / general / X-Pack API Integration Tests.x-pack/test/api_integration/apis/security_solution/network_dns·ts.apis SecuritySolution Endpoints Network DNS With packetbeat Make sure that we get Dns data and sorting by uniqueDomains descending

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://github.com/elastic/kibana/issues/87793

[00:00:00]     │
[00:00:00]       └-: apis
[00:00:00]         └-> "before all" hook in "apis"
[00:07:59]         └-: SecuritySolution Endpoints
[00:07:59]           └-> "before all" hook in "SecuritySolution Endpoints"
[00:08:30]           └-: Network DNS
[00:08:30]             └-> "before all" hook in "Network DNS"
[00:08:30]             └-: With packetbeat
[00:08:30]               └-> "before all" hook for "Make sure that we get Dns data and sorting by uniqueDomains ascending"
[00:08:30]               └-> "before all" hook for "Make sure that we get Dns data and sorting by uniqueDomains ascending"
[00:08:30]                 │ info [x-pack/test/functional/es_archives/packetbeat/dns] Loading "mappings.json"
[00:08:30]                 │ info [x-pack/test/functional/es_archives/packetbeat/dns] Loading "data.json.gz"
[00:08:30]                 │ info [o.e.c.m.MetadataCreateIndexService] [node-01] [packetbeat-7.0.0-iot-2019.11] creating index, cause [api], templates [], shards [3]/[1]
[00:08:30]                 │ info [x-pack/test/functional/es_archives/packetbeat/dns] Created index "packetbeat-7.0.0-iot-2019.11"
[00:08:30]                 │ debg [x-pack/test/functional/es_archives/packetbeat/dns] "packetbeat-7.0.0-iot-2019.11" settings {"index":{"lifecycle":{"name":"packetbeat-7.0.0","rollover_alias":"packetbeat-7.0.0"},"mapping":{"total_fields":{"limit":"10000"}},"number_of_replicas":"1","number_of_shards":"3","query":{"default_field":["tags","message","agent.version","agent.name","agent.type","agent.id","agent.ephemeral_id","client.address","client.mac","client.domain","client.geo.continent_name","client.geo.country_name","client.geo.region_name","client.geo.city_name","client.geo.country_iso_code","client.geo.region_iso_code","client.geo.name","cloud.provider","cloud.availability_zone","cloud.region","cloud.instance.id","cloud.instance.name","cloud.machine.type","cloud.account.id","container.runtime","container.id","container.image.name","container.image.tag","container.name","destination.address","destination.mac","destination.domain","destination.geo.continent_name","destination.geo.country_name","destination.geo.region_name","destination.geo.city_name","destination.geo.country_iso_code","destination.geo.region_iso_code","destination.geo.name","ecs.version","error.id","error.message","error.code","event.id","event.kind","event.category","event.action","event.outcome","event.type","event.module","event.dataset","event.hash","event.timezone","file.path","file.target_path","file.extension","file.type","file.device","file.inode","file.uid","file.owner","file.gid","file.group","file.mode","group.id","group.name","host.hostname","host.name","host.id","host.mac","host.type","host.architecture","host.os.platform","host.os.name","host.os.full","host.os.family","host.os.version","host.os.kernel","host.geo.continent_name","host.geo.country_name","host.geo.region_name","host.geo.city_name","host.geo.country_iso_code","host.geo.region_iso_code","host.geo.name","http.request.method","http.request.body.content","http.request.referrer","http.response.body.content","http.version","log.level","network.name","network.type","network.iana_number","network.transport","network.application","network.protocol","network.direction","network.community_id","observer.mac","observer.hostname","observer.vendor","observer.version","observer.serial_number","observer.type","observer.os.platform","observer.os.name","observer.os.full","observer.os.family","observer.os.version","observer.os.kernel","observer.geo.continent_name","observer.geo.country_name","observer.geo.region_name","observer.geo.city_name","observer.geo.country_iso_code","observer.geo.region_iso_code","observer.geo.name","organization.name","organization.id","os.platform","os.name","os.full","os.family","os.version","os.kernel","process.name","process.args","process.executable","process.title","process.working_directory","server.address","server.mac","server.domain","server.geo.continent_name","server.geo.country_name","server.geo.region_name","server.geo.city_name","server.geo.country_iso_code","server.geo.region_iso_code","server.geo.name","service.id","service.name","service.type","service.state","service.version","service.ephemeral_id","source.address","source.mac","source.domain","source.geo.continent_name","source.geo.country_name","source.geo.region_name","source.geo.city_name","source.geo.country_iso_code","source.geo.region_iso_code","source.geo.name","url.original","url.full","url.scheme","url.domain","url.path","url.query","url.fragment","url.username","url.password","user.id","user.name","user.full_name","user.email","user.hash","user.group.id","user.group.name","user_agent.original","user_agent.name","user_agent.version","user_agent.device.name","user_agent.os.platform","user_agent.os.name","user_agent.os.full","user_agent.os.family","user_agent.os.version","user_agent.os.kernel","agent.hostname","error.type","cloud.project.id","kubernetes.pod.name","kubernetes.pod.uid","kubernetes.namespace","kubernetes.node.name","kubernetes.container.name","kubernetes.container.image","type","server.process.name","server.process.args","server.process.executable","server.process.working_directory","server.process.start","client.process.name","client.process.args","client.process.executable","client.process.working_directory","client.process.start","flow.id","status","method","resource","path","query","params","request","response","amqp.reply-text","amqp.exchange","amqp.exchange-type","amqp.consumer-tag","amqp.routing-key","amqp.queue","amqp.content-type","amqp.content-encoding","amqp.delivery-mode","amqp.correlation-id","amqp.reply-to","amqp.expiration","amqp.message-id","amqp.timestamp","amqp.type","amqp.user-id","amqp.app-id","cassandra.request.headers.flags","cassandra.request.headers.stream","cassandra.request.headers.op","cassandra.request.query","cassandra.response.headers.flags","cassandra.response.headers.stream","cassandra.response.headers.op","cassandra.response.result.type","cassandra.response.result.rows.meta.keyspace","cassandra.response.result.rows.meta.table","cassandra.response.result.rows.meta.flags","cassandra.response.result.rows.meta.paging_state","cassandra.response.result.keyspace","cassandra.response.result.schema_change.change","cassandra.response.result.schema_change.keyspace","cassandra.response.result.schema_change.table","cassandra.response.result.schema_change.object","cassandra.response.result.schema_change.target","cassandra.response.result.schema_change.name","cassandra.response.result.schema_change.args","cassandra.response.result.prepared.prepared_id","cassandra.response.result.prepared.req_meta.keyspace","cassandra.response.result.prepared.req_meta.table","cassandra.response.result.prepared.req_meta.flags","cassandra.response.result.prepared.req_meta.paging_state","cassandra.response.result.prepared.resp_meta.keyspace","cassandra.response.result.prepared.resp_meta.table","cassandra.response.result.prepared.resp_meta.flags","cassandra.response.result.prepared.resp_meta.paging_state","cassandra.response.authentication.class","cassandra.response.warnings","cassandra.response.event.type","cassandra.response.event.change","cassandra.response.event.host","cassandra.response.event.schema_change.change","cassandra.response.event.schema_change.keyspace","cassandra.response.event.schema_change.table","cassandra.response.event.schema_change.object","cassandra.response.event.schema_change.target","cassandra.response.event.schema_change.name","cassandra.response.event.schema_change.args","cassandra.response.error.msg","cassandra.response.error.type","cassandra.response.error.details.read_consistency","cassandra.response.error.details.write_type","cassandra.response.error.details.keyspace","cassandra.response.error.details.table","cassandra.response.error.details.stmt_id","cassandra.response.error.details.num_failures","cassandra.response.error.details.function","cassandra.response.error.details.arg_types","dhcpv4.transaction_id","dhcpv4.flags","dhcpv4.client_mac","dhcpv4.server_name","dhcpv4.op_code","dhcpv4.hardware_type","dhcpv4.option.message_type","dhcpv4.option.parameter_request_list","dhcpv4.option.class_identifier","dhcpv4.option.domain_name","dhcpv4.option.hostname","dhcpv4.option.message","dhcpv4.option.boot_file_name","dns.op_code","dns.response_code","dns.question.name","dns.question.type","dns.question.class","dns.question.registered_domain","dns.answers.name","dns.answers.type","dns.answers.class","dns.answers.data","dns.authorities.name","dns.authorities.type","dns.authorities.class","dns.additionals.name","dns.additionals.type","dns.additionals.class","dns.additionals.data","dns.opt.version","dns.opt.ext_rcode","http.response.status_phrase","icmp.version","icmp.request.message","icmp.response.message","memcache.protocol_type","memcache.request.line","memcache.request.command","memcache.response.command","memcache.request.type","memcache.response.type","memcache.response.error_msg","memcache.request.opcode","memcache.response.opcode","memcache.response.status","memcache.request.raw_args","memcache.request.automove","memcache.response.version","mongodb.error","mongodb.fullCollectionName","mongodb.startingFrom","mongodb.query","mongodb.returnFieldsSelector","mongodb.selector","mongodb.update","mongodb.cursorId","mysql.insert_id","mysql.num_fields","mysql.num_rows","mysql.query","mysql.error_message","nfs.tag","nfs.opcode","nfs.status","rpc.xid","rpc.status","rpc.auth_flavor","rpc.cred.gids","rpc.cred.machinename","pgsql.error_message","pgsql.error_severity","pgsql.num_fields","pgsql.num_rows","redis.return_value","redis.error","thrift.params","thrift.service","thrift.return_value","thrift.exceptions","tls.version","tls.resumption_method","tls.client_hello.version","tls.client_hello.extensions.server_name_indication","tls.client_hello.extensions.application_layer_protocol_negotiation","tls.client_hello.extensions.session_ticket","tls.client_hello.extensions.supported_versions","tls.client_hello.extensions.supported_groups","tls.client_hello.extensions.signature_algorithms","tls.client_hello.extensions.ec_points_formats","tls.client_hello.extensions._unparsed_","tls.server_hello.version","tls.server_hello.selected_cipher","tls.server_hello.selected_compression_method","tls.server_hello.session_id","tls.server_hello.extensions.session_ticket","tls.server_hello.extensions.supported_versions","tls.server_hello.extensions.ec_points_formats","tls.server_hello.extensions._unparsed_","tls.client_certificate.serial_number","tls.client_certificate.public_key_algorithm","tls.client_certificate.signature_algorithm","tls.client_certificate.raw","tls.client_certificate.subject.country","tls.client_certificate.subject.organization","tls.client_certificate.subject.organizational_unit","tls.client_certificate.subject.province","tls.client_certificate.subject.common_name","tls.client_certificate.issuer.country","tls.client_certificate.issuer.organization","tls.client_certificate.issuer.organizational_unit","tls.client_certificate.issuer.province","tls.client_certificate.issuer.common_name","tls.client_certificate.fingerprint.md5","tls.client_certificate.fingerprint.sha1","tls.client_certificate.fingerprint.sha256","tls.server_certificate.serial_number","tls.server_certificate.public_key_algorithm","tls.server_certificate.signature_algorithm","tls.server_certificate.raw","tls.server_certificate.subject.country","tls.server_certificate.subject.organization","tls.server_certificate.subject.organizational_unit","tls.server_certificate.subject.province","tls.server_certificate.subject.common_name","tls.server_certificate.issuer.country","tls.server_certificate.issuer.organization","tls.server_certificate.issuer.organizational_unit","tls.server_certificate.issuer.province","tls.server_certificate.issuer.common_name","tls.server_certificate.fingerprint.md5","tls.server_certificate.fingerprint.sha1","tls.server_certificate.fingerprint.sha256","tls.alert_types","tls.fingerprints.ja3.hash","tls.fingerprints.ja3.str","fields.*"]},"refresh_interval":"5s"}}
[00:08:30]                 │ info [o.e.c.m.MetadataMappingService] [node-01] [packetbeat-7.0.0-iot-2019.11/oMJ4-sHKRruMdwJex2yNRA] update_mapping [_doc]
[00:08:32]                 │ info [x-pack/test/functional/es_archives/packetbeat/dns] Indexed 5700 docs into "packetbeat-7.0.0-iot-2019.11"
[00:08:32]               └-> Make sure that we get Dns data and sorting by uniqueDomains ascending
[00:08:32]                 └-> "before each" hook: global before each for "Make sure that we get Dns data and sorting by uniqueDomains ascending"
[00:08:32]                 └- ✓ pass  (40ms)
[00:08:32]               └-> Make sure that we get Dns data and sorting by uniqueDomains descending
[00:08:32]                 └-> "before each" hook: global before each for "Make sure that we get Dns data and sorting by uniqueDomains descending"
[00:08:32]                 └- ✖ fail: apis SecuritySolution Endpoints Network DNS With packetbeat Make sure that we get Dns data and sorting by uniqueDomains descending
[00:08:32]                 │      Error: expected 0 to equal 10
[00:08:32]                 │       at Assertion.assert (/dev/shm/workspace/parallel/6/kibana/node_modules/@kbn/expect/expect.js:100:11)
[00:08:32]                 │       at Assertion.be.Assertion.equal (/dev/shm/workspace/parallel/6/kibana/node_modules/@kbn/expect/expect.js:227:8)
[00:08:32]                 │       at Assertion.be (/dev/shm/workspace/parallel/6/kibana/node_modules/@kbn/expect/expect.js:69:22)
[00:08:32]                 │       at Context.<anonymous> (test/api_integration/apis/security_solution/network_dns.ts:87:44)
[00:08:32]                 │       at runMicrotasks (<anonymous>)
[00:08:32]                 │       at processTicksAndRejections (node:internal/process/task_queues:96:5)
[00:08:32]                 │       at Object.apply (/dev/shm/workspace/parallel/6/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
[00:08:32]                 │ 
[00:08:32]                 │

Stack Trace

Error: expected 0 to equal 10
    at Assertion.assert (/dev/shm/workspace/parallel/6/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (/dev/shm/workspace/parallel/6/kibana/node_modules/@kbn/expect/expect.js:227:8)
    at Assertion.be (/dev/shm/workspace/parallel/6/kibana/node_modules/@kbn/expect/expect.js:69:22)
    at Context.<anonymous> (test/api_integration/apis/security_solution/network_dns.ts:87:44)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.apply (/dev/shm/workspace/parallel/6/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

and 52 more failures, only showing the first 3.

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @cnasikas

elasticmachine · 2021-10-20T17:48:50Z

Pinging @elastic/security-threat-hunting-cases (Team:Threat Hunting:Cases)

pmuellr

I made some comments on the unique-ness of the messages being thrown. I'm a little sensitive to this, as many of the existing connectors like email/webhook don't have very good messages on failures :-). I made the comments on the Jira code, but I think it's probably applicable to the other connectors modified here.

x-pack/plugins/actions/server/builtin_action_types/jira/service.ts

x-pack/plugins/actions/server/builtin_action_types/lib/axios_utils.ts

cnasikas · 2021-10-21T08:26:33Z

Thank you @pmuellr! I totally agree with your feedback. We need to be careful with our messages as it will help a lot to potential SDHs. Fixed 🙂

cnasikas · 2021-10-21T13:41:59Z

@elasticmachine merge upstream

pmuellr

LGTM, thanks for the changes Christos!

YulNaumenko

LGTM

cnasikas · 2021-10-22T00:51:45Z

@elasticmachine merge upstream

jportner

Nice job on this, looks really comprehensive. A few nits and one question in the comments below 👍

x-pack/plugins/actions/server/builtin_action_types/lib/axios_utils.test.ts

x-pack/plugins/actions/server/builtin_action_types/lib/axios_utils.ts

x-pack/plugins/actions/server/builtin_action_types/resilient/service.test.ts

x-pack/plugins/actions/server/builtin_action_types/lib/axios_utils.ts

jportner

LGTM!

cnasikas · 2021-10-24T21:06:17Z

@elasticmachine merge upstream

kibanamachine · 2021-10-24T23:02:32Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💔 Build #1221 failed fa8a208
💔 Build #1161 failed b4b9316
💔 Build #942 failed 44599cc
💔 Build #877 failed 4613a9f
💔 Build #814 failed 4b4a344

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @cnasikas

Co-authored-by: Kibana Machine <[email protected]>

kibanamachine · 2021-10-25T07:37:43Z

💚 Backport successful

Status	Branch	Result
✅	7.16

This backport PR will be merged automatically after passing CI.

Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Kibana Machine <[email protected]>

Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Christos Nasikas <[email protected]>

Co-authored-by: Kibana Machine <[email protected]>

cnasikas added 5 commits October 20, 2021 15:46

Create throwIfRequestIsNotValid util function

f8de93b

Improve createErrorMessage

715bf28

Check responses in Jira

d0f1ab0

Check response in IBM resilient

ce7df44

Check response in Swimlane

6fd2cdc

cnasikas self-assigned this Oct 20, 2021

cnasikas added Team:Threat Hunting:Cases bug Fixes for quality problems that affect the customer experience v7.16.0 v8.0.0 release_note:skip Skip the PR/issue when compiling release notes labels Oct 20, 2021

Check response in Jira

a03912d

cnasikas marked this pull request as ready for review October 20, 2021 17:48

cnasikas requested a review from a team as a code owner October 20, 2021 17:48

cnasikas requested a review from jportner October 20, 2021 17:49

YulNaumenko self-requested a review October 20, 2021 18:45

pmuellr requested changes Oct 20, 2021

View reviewed changes

x-pack/plugins/actions/server/builtin_action_types/jira/service.ts Outdated Show resolved Hide resolved

x-pack/plugins/actions/server/builtin_action_types/lib/axios_utils.ts Outdated Show resolved Hide resolved

Improve messages

4b4a344

cnasikas force-pushed the connector_res branch from c89d041 to 4b4a344 Compare October 21, 2021 08:25

cnasikas requested a review from pmuellr October 21, 2021 08:26

Fix integration tests

4613a9f

Merge branch 'master' into connector_res

44599cc

pmuellr approved these changes Oct 21, 2021

View reviewed changes

YulNaumenko approved these changes Oct 22, 2021

View reviewed changes

Merge branch 'master' into connector_res

b4b9316

jportner reviewed Oct 22, 2021

View reviewed changes

cnasikas added 2 commits October 23, 2021 13:34

Merge branch 'master' into connector_res

270218e

PR feedback

fa8a208

jportner approved these changes Oct 24, 2021

View reviewed changes

Merge branch 'master' into connector_res

f763d2c

cnasikas added the auto-backport Deprecated - use backport:version if exact versions are needed label Oct 25, 2021

cnasikas merged commit 4d36440 into elastic:master Oct 25, 2021

cnasikas deleted the connector_res branch October 25, 2021 07:34

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Oct 25, 2021

[Connectors] Check connector's responses (elastic#115797)

2abb24f

Co-authored-by: Kibana Machine <[email protected]>

kibanamachine mentioned this pull request Oct 25, 2021

[7.16] [Connectors] Check connector's responses (#115797) #116085

Merged

cnasikas added the v7.15.2 label Oct 25, 2021

cnasikas mentioned this pull request Oct 25, 2021

[7.15] [Connectors] Check connector's responses (#115797) #116086

Merged

cnasikas added a commit that referenced this pull request Oct 25, 2021

[Connectors] Check connector's responses (#115797) (#116086)

85567f2

Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Kibana Machine <[email protected]>

kibanamachine added a commit that referenced this pull request Oct 25, 2021

[Connectors] Check connector's responses (#115797) (#116085)

296406b

Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Christos Nasikas <[email protected]>

shivindera pushed a commit to shivindera/kibana that referenced this pull request Oct 25, 2021

[Connectors] Check connector's responses (elastic#115797)

8341195

Co-authored-by: Kibana Machine <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Connectors] Check connector's responses #115797

[Connectors] Check connector's responses #115797

cnasikas commented Oct 20, 2021 •

edited

Loading

kibanamachine commented Oct 20, 2021 •

edited

Loading

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

elasticmachine commented Oct 20, 2021

pmuellr left a comment

cnasikas commented Oct 21, 2021

cnasikas commented Oct 21, 2021

pmuellr left a comment

YulNaumenko left a comment

cnasikas commented Oct 22, 2021

jportner left a comment

jportner left a comment

cnasikas commented Oct 24, 2021

kibanamachine commented Oct 24, 2021

kibanamachine commented Oct 25, 2021

[Connectors] Check connector's responses #115797

[Connectors] Check connector's responses #115797

Conversation

cnasikas commented Oct 20, 2021 • edited Loading

Summary

Checklist

For maintainers

kibanamachine commented Oct 20, 2021 • edited Loading

💔 Build Failed

Failed CI Steps

Test Failures

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

Metrics [docs]

elasticmachine commented Oct 20, 2021

pmuellr left a comment

Choose a reason for hiding this comment

cnasikas commented Oct 21, 2021

cnasikas commented Oct 21, 2021

pmuellr left a comment

Choose a reason for hiding this comment

YulNaumenko left a comment

Choose a reason for hiding this comment

cnasikas commented Oct 22, 2021

jportner left a comment

Choose a reason for hiding this comment

jportner left a comment

Choose a reason for hiding this comment

cnasikas commented Oct 24, 2021

kibanamachine commented Oct 24, 2021

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Oct 25, 2021

💚 Backport successful

cnasikas commented Oct 20, 2021 •

edited

Loading

kibanamachine commented Oct 20, 2021 •

edited

Loading