[TaskManager] fetch more tasks to run than available capacity

[pmuellr]

resolves #65552

Summary

Currently TaskManager attempts to claim exactly the number of tasks that
it has capacity for.

As an optimization, we're going to change to have it request more tasks than
it has capacity for. This should improve latency, as when tasks complete,
there may be some of these excess tasks that can be started, as they are
already claimed (they still need to be marked running).

All the plumbing already handles getting more tasks than we asked for, we
were just never asking for more than we needed previously.

Summarize your PR. If it involves visual changes include a screenshot or gif.

Checklist

Delete any items that are not applicable to this PR.

• Unit or functional tests were updated or added to match the most common scenarios

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[kibanamachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request
• Commit: e116e26
• Pipeline Steps (look for red circles / failed steps)
• Interpreting CI Failures

Failed CI Steps

• Execute xpack-kibana-ciGroup2
• Execute xpack-kibana-ciGroup2

---

Test Failures

Plugin Functional Tests.x-pack/test/plugin_api_integration/test_suites/task_manager/task_manager_integration·js.task_manager scheduling and running tasks should prioritize tasks which are called using runNow

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: task_manager
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: scheduling and running tasks
[00:00:00]             └-> "before all" hook
[00:00:00]             └-> should support middleware
[00:00:00]               └-> "before each" hook: global before each
[00:00:00]               └-> "before each" hook
[00:00:00]               └-> "before each" hook
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] [.kibana_task_manager_test_result] creating index, cause [api], templates [], shards [1]/[1]
[00:00:00]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] [ilm-history-2-000001] creating index, cause [api], templates [ilm-history], shards [1]/[0]
[00:00:00]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] moving index [ilm-history-2-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [ilm-history-ilm-policy]
[00:00:00]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] moving index [ilm-history-2-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"wait-for-indexing-complete"}] in policy [ilm-history-ilm-policy]
[00:00:00]               │ debg Task created: 791cada0-e2fc-11ea-933d-77f639be8bc6
[00:00:00]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:00]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] moving index [ilm-history-2-000001] from [{"phase":"hot","action":"unfollow","name":"wait-for-indexing-complete"}] to [{"phase":"hot","action":"unfollow","name":"wait-for-follow-shard-tasks"}] in policy [ilm-history-ilm-policy]
[00:00:01]               │ debg --- retry.try failed again with the same message...
[00:00:01]               │ debg --- retry.try failed again with the same message...
[00:00:02]               │ debg --- retry.try failed again with the same message...
[00:00:02]               │ debg --- retry.try failed again with the same message...
[00:00:03]               │ debg --- retry.try failed again with the same message...
[00:00:03]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] [.kibana_task_manager_test_result/tBBD_MiASRuNuT215FjvLw] update_mapping [_doc]
[00:00:03]               │ debg Task found: 791cada0-e2fc-11ea-933d-77f639be8bc6
[00:00:03]               │ debg Task status: claiming
[00:00:03]               │ debg Task state: {}
[00:00:03]               │ debg Task params: {
[00:00:03]               │        "originalParams": {
[00:00:03]               │          "historyItem": 58
[00:00:03]               │        },
[00:00:03]               │        "superFly": "My middleware param!"
[00:00:03]               │      }
[00:00:03]               │ debg --- retry.try error: expected undefined to sort of equal 1
[00:00:04]               │ debg Task found: 791cada0-e2fc-11ea-933d-77f639be8bc6
[00:00:04]               │ debg Task status: idle
[00:00:04]               │ debg Task state: {
[00:00:04]               │        "count": 1
[00:00:04]               │      }
[00:00:04]               │ debg Task params: {
[00:00:04]               │        "originalParams": {
[00:00:04]               │          "historyItem": 58
[00:00:04]               │        },
[00:00:04]               │        "superFly": "My middleware param!"
[00:00:04]               │      }
[00:00:04]               └- ✓ pass  (3.8s) "task_manager scheduling and running tasks should support middleware"
[00:00:04]             └-> should remove non-recurring tasks after they complete
[00:00:04]               └-> "before each" hook: global before each
[00:00:04]               └-> "before each" hook
[00:00:04]               └-> "before each" hook
[00:00:05]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:05]               │ debg --- retry.try failed again with the same message...
[00:00:06]               │ debg --- retry.try failed again with the same message...
[00:00:06]               │ debg --- retry.try error: expected [ { taskType: 'sampleTask',
[00:00:06]               │          retryAt: '2020-08-20T15:48:13.765Z',
[00:00:06]               │          runAt: '2020-08-20T15:47:42.654Z',
[00:00:06]               │          scope: [ 'testing' ],
[00:00:06]               │          startedAt: null,
[00:00:06]               │          state: {},
[00:00:06]               │          params: 
[00:00:06]               │           { originalParams: {},
[00:00:06]               │             superFly: 'My middleware param!' },
[00:00:06]               │          ownerId: 'kibana:5b2de169-2785-441b-ae8c-186a1936b17d',
[00:00:06]               │          scheduledAt: '2020-08-20T15:47:42.654Z',
[00:00:06]               │          attempts: 0,
[00:00:06]               │          status: 'claiming',
[00:00:06]               │          id: '7be654f0-e2fc-11ea-933d-77f639be8bc6' } ] to sort of equal []
[00:00:07]               └- ✓ pass  (2.1s) "task_manager scheduling and running tasks should remove non-recurring tasks after they complete"
[00:00:07]             └-> should use a given ID as the task document ID
[00:00:07]               └-> "before each" hook: global before each
[00:00:07]               └-> "before each" hook
[00:00:07]               └-> "before each" hook
[00:00:07]               └- ✓ pass  (372ms) "task_manager scheduling and running tasks should use a given ID as the task document ID"
[00:00:07]             └-> should allow a task with a given ID to be scheduled multiple times
[00:00:07]               └-> "before each" hook: global before each
[00:00:07]               └-> "before each" hook
[00:00:07]               └-> "before each" hook
[00:00:07]               │ proc [kibana]   log   [15:47:45.243] [error][data][elasticsearch] [version_conflict_engine_exception]: [task:test-task-to-reschedule-in-task-manager]: version conflict, document already exists (current version [1])
[00:00:07]               └- ✓ pass  (44ms) "task_manager scheduling and running tasks should allow a task with a given ID to be scheduled multiple times"
[00:00:07]             └-> should reschedule if task errors
[00:00:07]               └-> "before each" hook: global before each
[00:00:07]               └-> "before each" hook
[00:00:07]               └-> "before each" hook
[00:00:07]               │ debg --- retry.try error: Cannot read property 'id' of undefined
[00:00:08]               │ debg --- retry.try error: expected 'test-task-for-sample-task-plugin-to-test-task-manager' to sort of equal '7d795100-e2fc-11ea-933d-77f639be8bc6'
[00:00:08]               │ debg --- retry.try failed again with the same message...
[00:00:09]               │ debg --- retry.try failed again with the same message...
[00:00:09]               │ proc [kibana]   log   [15:47:46.901] [error][plugins][taskManager][taskManager] Task sampleTask "7d795100-e2fc-11ea-933d-77f639be8bc6" failed: Error: Dangit!!!!!
[00:00:09]               │ debg --- retry.try failed again with the same message...
[00:00:10]               └- ✓ pass  (2.7s) "task_manager scheduling and running tasks should reschedule if task errors"
[00:00:10]             └-> should reschedule if task returns runAt
[00:00:10]               └-> "before each" hook: global before each
[00:00:10]               └-> "before each" hook
[00:00:11]               └-> "before each" hook
[00:00:11]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:11]               │ debg --- retry.try failed again with the same message...
[00:00:12]               │ debg --- retry.try failed again with the same message...
[00:00:12]               │ debg --- retry.try error: expected 14 to sort of equal 15
[00:00:13]               └- ✓ pass  (2.1s) "task_manager scheduling and running tasks should reschedule if task returns runAt"
[00:00:13]             └-> should reschedule if task has an interval
[00:00:13]               └-> "before each" hook: global before each
[00:00:13]               └-> "before each" hook
[00:00:14]               └-> "before each" hook
[00:00:14]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:14]               │ debg --- retry.try failed again with the same message...
[00:00:15]               │ debg --- retry.try failed again with the same message...
[00:00:15]               │ debg --- retry.try error: expected undefined to sort of equal 1
[00:00:16]               └- ✓ pass  (2.1s) "task_manager scheduling and running tasks should reschedule if task has an interval"
[00:00:16]             └-> should support the deprecated interval field
[00:00:16]               └-> "before each" hook: global before each
[00:00:16]               └-> "before each" hook
[00:00:17]               └-> "before each" hook
[00:00:17]               │ proc [kibana]   log   [15:47:54.826] [warning][plugins][taskManager][taskManager] Task of type "sampleTask" has been scheduled with the deprecated 'interval' field which is due to be removed in a future release
[00:00:17]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:17]               │ debg --- retry.try failed again with the same message...
[00:00:18]               │ debg --- retry.try failed again with the same message...
[00:00:18]               │ debg --- retry.try failed again with the same message...
[00:00:19]               │ debg --- retry.try failed again with the same message...
[00:00:19]               │ debg --- retry.try failed again with the same message...
[00:00:20]               │ debg --- retry.try failed again with the same message...
[00:00:20]               │ debg --- retry.try failed again with the same message...
[00:00:21]               │ debg --- retry.try failed again with the same message...
[00:00:21]               │ debg --- retry.try error: expected undefined to sort of equal 1
[00:00:22]               └- ✓ pass  (5.2s) "task_manager scheduling and running tasks should support the deprecated interval field"
[00:00:22]             └-> should return a task run result when asked to run a task now
[00:00:22]               └-> "before each" hook: global before each
[00:00:22]               └-> "before each" hook
[00:00:23]               └-> "before each" hook
[00:00:23]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:23]               │ debg --- retry.try failed again with the same message...
[00:00:24]               │ debg --- retry.try failed again with the same message...
[00:00:24]               │ debg --- retry.try failed again with the same message...
[00:00:25]               │ debg --- retry.try failed again with the same message...
[00:00:25]               │ debg --- retry.try failed again with the same message...
[00:00:26]               │ debg --- retry.try failed again with the same message...
[00:00:26]               │ debg --- retry.try failed again with the same message...
[00:00:27]               │ debg --- retry.try error: expected undefined to sort of equal 1
[00:00:27]               │ debg --- retry.try failed again with the same message...
[00:00:28]               │ debg --- retry.try error: expected 1 to sort of equal 2
[00:00:29]               │ debg --- retry.try failed again with the same message...
[00:00:29]               └- ✓ pass  (6.5s) "task_manager scheduling and running tasks should return a task run result when asked to run a task now"
[00:00:29]             └-> should prioritize tasks which are called using runNow
[00:00:29]               └-> "before each" hook: global before each
[00:00:29]               └-> "before each" hook
[00:00:30]               └-> "before each" hook
[00:00:30]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:30]               │ debg --- retry.try failed again with the same message...
[00:00:31]               │ debg --- retry.try failed again with the same message...
[00:00:31]               │ debg --- retry.try failed again with the same message...
[00:00:32]               │ debg --- retry.try failed again with the same message...
[00:00:32]               │ debg --- retry.try failed again with the same message...
[00:00:42]               └- ✖ fail: task_manager scheduling and running tasks should prioritize tasks which are called using runNow
[00:00:42]               │       Error: expected { id: '8af94790-e2fc-11ea-933d-77f639be8bc6',
[00:00:42]               │   error: 'Error: Failed to run task "8af94790-e2fc-11ea-933d-77f639be8bc6" as it is currently running' } to sort of equal { id: '8af94790-e2fc-11ea-933d-77f639be8bc6' }
[00:00:42]               │       + expected - actual
[00:00:42]               │ 
[00:00:42]               │        {
[00:00:42]               │       -  "error": "Error: Failed to run task \"8af94790-e2fc-11ea-933d-77f639be8bc6\" as it is currently running"
[00:00:42]               │          "id": "8af94790-e2fc-11ea-933d-77f639be8bc6"
[00:00:42]               │        }
[00:00:42]               │       
[00:00:42]               │       at Assertion.assert (/dev/shm/workspace/parallel/14/kibana/packages/kbn-expect/expect.js:100:11)
[00:00:42]               │       at Assertion.eql (/dev/shm/workspace/parallel/14/kibana/packages/kbn-expect/expect.js:244:8)
[00:00:42]               │       at Context.it (test/plugin_api_integration/test_suites/task_manager/task_manager_integration.js:400:37)
[00:00:42]               │ 
[00:00:42]               │ 

Stack Trace

{ Error: expected { id: '8af94790-e2fc-11ea-933d-77f639be8bc6',
  error: 'Error: Failed to run task "8af94790-e2fc-11ea-933d-77f639be8bc6" as it is currently running' } to sort of equal { id: '8af94790-e2fc-11ea-933d-77f639be8bc6' }
    at Assertion.assert (/dev/shm/workspace/parallel/14/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/14/kibana/packages/kbn-expect/expect.js:244:8)
    at Context.it (test/plugin_api_integration/test_suites/task_manager/task_manager_integration.js:400:37)
  actual:
   '{\n  "error": "Error: Failed to run task \\"8af94790-e2fc-11ea-933d-77f639be8bc6\\" as it is currently running"\n  "id": "8af94790-e2fc-11ea-933d-77f639be8bc6"\n}',
  expected: '{\n  "id": "8af94790-e2fc-11ea-933d-77f639be8bc6"\n}',
  showDiff: true }

---

Plugin Functional Tests.x-pack/test/plugin_api_integration/test_suites/task_manager/task_manager_integration·js.task_manager scheduling and running tasks should prioritize tasks which are called using runNow

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://dryrun

[00:00:00]       │
[00:00:00]         └-: task_manager
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: scheduling and running tasks
[00:00:00]             └-> "before all" hook
[00:00:00]             └-> should support middleware
[00:00:00]               └-> "before each" hook: global before each
[00:00:00]               └-> "before each" hook
[00:00:00]               └-> "before each" hook
[00:00:00]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] [.kibana_task_manager_test_result] creating index, cause [api], templates [], shards [1]/[1]
[00:00:00]               │ debg Task created: e24e98f0-e2f8-11ea-b654-4d8baeac624a
[00:00:00]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:00]               │ debg --- retry.try failed again with the same message...
[00:00:01]               │ debg --- retry.try failed again with the same message...
[00:00:01]               │ debg --- retry.try failed again with the same message...
[00:00:02]               │ debg --- retry.try failed again with the same message...
[00:00:02]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] [ilm-history-2-000001] creating index, cause [api], templates [ilm-history], shards [1]/[0]
[00:00:02]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] moving index [ilm-history-2-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [ilm-history-ilm-policy]
[00:00:02]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] moving index [ilm-history-2-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"wait-for-indexing-complete"}] in policy [ilm-history-ilm-policy]
[00:00:02]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] moving index [ilm-history-2-000001] from [{"phase":"hot","action":"unfollow","name":"wait-for-indexing-complete"}] to [{"phase":"hot","action":"unfollow","name":"wait-for-follow-shard-tasks"}] in policy [ilm-history-ilm-policy]
[00:00:02]               │ debg --- retry.try failed again with the same message...
[00:00:03]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xxl-1597933453544464597] [.kibana_task_manager_test_result/ObttRfylTH295pVKteLqKw] update_mapping [_doc]
[00:00:03]               │ debg Task found: e24e98f0-e2f8-11ea-b654-4d8baeac624a
[00:00:03]               │ debg Task status: claiming
[00:00:03]               │ debg Task state: {}
[00:00:03]               │ debg Task params: {
[00:00:03]               │        "originalParams": {
[00:00:03]               │          "historyItem": 51
[00:00:03]               │        },
[00:00:03]               │        "superFly": "My middleware param!"
[00:00:03]               │      }
[00:00:03]               │ debg --- retry.try error: expected undefined to sort of equal 1
[00:00:03]               │ debg Task found: e24e98f0-e2f8-11ea-b654-4d8baeac624a
[00:00:03]               │ debg Task status: idle
[00:00:03]               │ debg Task state: {
[00:00:03]               │        "count": 1
[00:00:03]               │      }
[00:00:03]               │ debg Task params: {
[00:00:03]               │        "originalParams": {
[00:00:03]               │          "historyItem": 51
[00:00:03]               │        },
[00:00:03]               │        "superFly": "My middleware param!"
[00:00:03]               │      }
[00:00:03]               └- ✓ pass  (3.7s) "task_manager scheduling and running tasks should support middleware"
[00:00:03]             └-> should remove non-recurring tasks after they complete
[00:00:03]               └-> "before each" hook: global before each
[00:00:03]               └-> "before each" hook
[00:00:04]               └-> "before each" hook
[00:00:04]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:05]               │ debg --- retry.try failed again with the same message...
[00:00:05]               │ debg --- retry.try failed again with the same message...
[00:00:06]               │ debg --- retry.try error: expected [ { taskType: 'sampleTask',
[00:00:06]               │          retryAt: '2020-08-20T15:22:32.158Z',
[00:00:06]               │          runAt: '2020-08-20T15:22:01.013Z',
[00:00:06]               │          scope: [ 'testing' ],
[00:00:06]               │          startedAt: null,
[00:00:06]               │          state: {},
[00:00:06]               │          params: 
[00:00:06]               │           { originalParams: {},
[00:00:06]               │             superFly: 'My middleware param!' },
[00:00:06]               │          ownerId: 'kibana:5b2de169-2785-441b-ae8c-186a1936b17d',
[00:00:06]               │          scheduledAt: '2020-08-20T15:22:01.013Z',
[00:00:06]               │          attempts: 0,
[00:00:06]               │          status: 'claiming',
[00:00:06]               │          id: 'e5026e50-e2f8-11ea-b654-4d8baeac624a' } ] to sort of equal []
[00:00:07]               └- ✓ pass  (2.5s) "task_manager scheduling and running tasks should remove non-recurring tasks after they complete"
[00:00:07]             └-> should use a given ID as the task document ID
[00:00:07]               └-> "before each" hook: global before each
[00:00:07]               └-> "before each" hook
[00:00:07]               └-> "before each" hook
[00:00:07]               └- ✓ pass  (19ms) "task_manager scheduling and running tasks should use a given ID as the task document ID"
[00:00:07]             └-> should allow a task with a given ID to be scheduled multiple times
[00:00:07]               └-> "before each" hook: global before each
[00:00:07]               └-> "before each" hook
[00:00:07]               └-> "before each" hook
[00:00:07]               │ proc [kibana]   log   [15:22:03.694] [error][data][elasticsearch] [version_conflict_engine_exception]: [task:test-task-to-reschedule-in-task-manager]: version conflict, document already exists (current version [1])
[00:00:07]               └- ✓ pass  (39ms) "task_manager scheduling and running tasks should allow a task with a given ID to be scheduled multiple times"
[00:00:07]             └-> should reschedule if task errors
[00:00:07]               └-> "before each" hook: global before each
[00:00:07]               └-> "before each" hook
[00:00:07]               └-> "before each" hook
[00:00:07]               │ debg --- retry.try error: Cannot read property 'id' of undefined
[00:00:08]               │ debg --- retry.try error: expected 'test-task-for-sample-task-plugin-to-test-task-manager' to sort of equal 'e6a23ba0-e2f8-11ea-b654-4d8baeac624a'
[00:00:08]               │ debg --- retry.try failed again with the same message...
[00:00:09]               │ proc [kibana]   log   [15:22:05.299] [error][plugins][taskManager][taskManager] Task sampleTask "e6a23ba0-e2f8-11ea-b654-4d8baeac624a" failed: Error: Dangit!!!!!
[00:00:09]               │ debg --- retry.try failed again with the same message...
[00:00:09]               │ debg --- retry.try failed again with the same message...
[00:00:10]               └- ✓ pass  (2.6s) "task_manager scheduling and running tasks should reschedule if task errors"
[00:00:10]             └-> should reschedule if task returns runAt
[00:00:10]               └-> "before each" hook: global before each
[00:00:10]               └-> "before each" hook
[00:00:10]               └-> "before each" hook
[00:00:10]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:11]               │ debg --- retry.try failed again with the same message...
[00:00:11]               │ debg --- retry.try failed again with the same message...
[00:00:12]               │ debg --- retry.try error: expected 8 to sort of equal 9
[00:00:13]               └- ✓ pass  (2.1s) "task_manager scheduling and running tasks should reschedule if task returns runAt"
[00:00:13]             └-> should reschedule if task has an interval
[00:00:13]               └-> "before each" hook: global before each
[00:00:13]               └-> "before each" hook
[00:00:13]               └-> "before each" hook
[00:00:14]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:14]               │ debg --- retry.try failed again with the same message...
[00:00:15]               │ debg --- retry.try failed again with the same message...
[00:00:15]               │ debg --- retry.try error: expected undefined to sort of equal 1
[00:00:16]               └- ✓ pass  (2.1s) "task_manager scheduling and running tasks should reschedule if task has an interval"
[00:00:16]             └-> should support the deprecated interval field
[00:00:16]               └-> "before each" hook: global before each
[00:00:16]               └-> "before each" hook
[00:00:17]               └-> "before each" hook
[00:00:17]               │ proc [kibana]   log   [15:22:13.236] [warning][plugins][taskManager][taskManager] Task of type "sampleTask" has been scheduled with the deprecated 'interval' field which is due to be removed in a future release
[00:00:17]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:17]               │ debg --- retry.try failed again with the same message...
[00:00:18]               │ debg --- retry.try failed again with the same message...
[00:00:18]               │ debg --- retry.try failed again with the same message...
[00:00:19]               │ debg --- retry.try failed again with the same message...
[00:00:19]               │ debg --- retry.try failed again with the same message...
[00:00:20]               │ debg --- retry.try failed again with the same message...
[00:00:20]               │ debg --- retry.try failed again with the same message...
[00:00:21]               │ debg --- retry.try failed again with the same message...
[00:00:21]               │ debg --- retry.try error: expected undefined to sort of equal 1
[00:00:22]               └- ✓ pass  (5.2s) "task_manager scheduling and running tasks should support the deprecated interval field"
[00:00:22]             └-> should return a task run result when asked to run a task now
[00:00:22]               └-> "before each" hook: global before each
[00:00:22]               └-> "before each" hook
[00:00:23]               └-> "before each" hook
[00:00:23]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:23]               │ debg --- retry.try failed again with the same message...
[00:00:24]               │ debg --- retry.try failed again with the same message...
[00:00:24]               │ debg --- retry.try failed again with the same message...
[00:00:25]               │ debg --- retry.try failed again with the same message...
[00:00:25]               │ debg --- retry.try failed again with the same message...
[00:00:26]               │ debg --- retry.try failed again with the same message...
[00:00:26]               │ debg --- retry.try failed again with the same message...
[00:00:27]               │ debg --- retry.try error: expected undefined to sort of equal 1
[00:00:27]               │ debg --- retry.try failed again with the same message...
[00:00:28]               │ debg --- retry.try error: expected 1 to sort of equal 2
[00:00:29]               │ debg --- retry.try failed again with the same message...
[00:00:29]               └- ✓ pass  (6.5s) "task_manager scheduling and running tasks should return a task run result when asked to run a task now"
[00:00:29]             └-> should prioritize tasks which are called using runNow
[00:00:29]               └-> "before each" hook: global before each
[00:00:29]               └-> "before each" hook
[00:00:30]               └-> "before each" hook
[00:00:30]               │ debg --- retry.try error: expected 0 to sort of equal 1
[00:00:30]               │ debg --- retry.try failed again with the same message...
[00:00:31]               │ debg --- retry.try failed again with the same message...
[00:00:31]               │ debg --- retry.try failed again with the same message...
[00:00:32]               │ debg --- retry.try failed again with the same message...
[00:00:32]               │ debg --- retry.try failed again with the same message...
[00:00:42]               └- ✖ fail: task_manager scheduling and running tasks should prioritize tasks which are called using runNow
[00:00:42]               │       Error: expected { id: 'f42b0bd0-e2f8-11ea-b654-4d8baeac624a',
[00:00:42]               │   error: 'Error: Failed to run task "f42b0bd0-e2f8-11ea-b654-4d8baeac624a" as it is currently running' } to sort of equal { id: 'f42b0bd0-e2f8-11ea-b654-4d8baeac624a' }
[00:00:42]               │       + expected - actual
[00:00:42]               │ 
[00:00:42]               │        {
[00:00:42]               │       -  "error": "Error: Failed to run task \"f42b0bd0-e2f8-11ea-b654-4d8baeac624a\" as it is currently running"
[00:00:42]               │          "id": "f42b0bd0-e2f8-11ea-b654-4d8baeac624a"
[00:00:42]               │        }
[00:00:42]               │       
[00:00:42]               │       at Assertion.assert (/dev/shm/workspace/parallel/14/kibana/packages/kbn-expect/expect.js:100:11)
[00:00:42]               │       at Assertion.eql (/dev/shm/workspace/parallel/14/kibana/packages/kbn-expect/expect.js:244:8)
[00:00:42]               │       at Context.it (test/plugin_api_integration/test_suites/task_manager/task_manager_integration.js:400:37)
[00:00:42]               │ 
[00:00:42]               │ 

Stack Trace

{ Error: expected { id: 'f42b0bd0-e2f8-11ea-b654-4d8baeac624a',
  error: 'Error: Failed to run task "f42b0bd0-e2f8-11ea-b654-4d8baeac624a" as it is currently running' } to sort of equal { id: 'f42b0bd0-e2f8-11ea-b654-4d8baeac624a' }
    at Assertion.assert (/dev/shm/workspace/parallel/14/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/14/kibana/packages/kbn-expect/expect.js:244:8)
    at Context.it (test/plugin_api_integration/test_suites/task_manager/task_manager_integration.js:400:37)
  actual:
   '{\n  "error": "Error: Failed to run task \\"f42b0bd0-e2f8-11ea-b654-4d8baeac624a\\" as it is currently running"\n  "id": "f42b0bd0-e2f8-11ea-b654-4d8baeac624a"\n}',
  expected: '{\n  "id": "f42b0bd0-e2f8-11ea-b654-4d8baeac624a"\n}',
  showDiff: true }

---

Build metrics

✅ unchanged

History

• 💔 Build #69194 failed 82e30a95ca3ebb42de505c5bc9a94cafec451b88
• 💔 Build #69075 failed d23f3e43f0d0d622aa710592f4ee07d676ae3348

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[pmuellr]

no longer needed, not much of use here anyway