Expressions debug mode #57841

streamich · 2020-02-18T13:34:52Z

Summary

Dev Docs

Add ability to execute expression in a "debug" mode, which would collect execution information about each function, which would be used in Expression Explorer (developed by Canvas) as well as in Canvas app when expression editor is open.

Checklist

Delete any items that are not applicable to this PR.

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios

For maintainers

This was checked for breaking API changes and was labeled appropriately

elasticmachine · 2020-02-18T16:20:10Z

Pinging @elastic/kibana-canvas (Team:Canvas)

elasticmachine · 2020-02-18T16:20:12Z

Pinging @elastic/kibana-app-arch (Team:AppArch)

Dosant

Code lgtm

src/plugins/expressions/common/execution/execution.ts

src/plugins/expressions/common/execution/execution.test.ts

src/plugins/expressions/common/execution/execution.ts

…-mode

src/plugins/expressions/common/execution/execution.ts

ppisljar

code LGTM

* feat: 🎸 add ability to execute expression in debug mode * feat: 🎸 store resolved arguments in debug information * feat: 🎸 track function execution time and set "success" flag * feat: 🎸 store debug information for functions that throw * feat: 🎸 use performance.now, safe "fn" reference, fix typo

kibanamachine · 2020-04-10T17:36:36Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: 410bdad

Test Failures

Kibana Pipeline / kibana-oss-agent / Chrome UI Functional Tests.test/functional/apps/discover/_errors·js.discover app errors invalid scripted field error is rendered

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:01:49]         └-: discover app
[00:01:49]           └-> "before all" hook
[00:01:49]           └-> "before all" hook
[00:09:43]           └-: errors
[00:09:43]             └-> "before all" hook
[00:09:43]             └-> "before all" hook
[00:09:43]               │ info [logstash_functional] Loading "mappings.json"
[00:09:43]               │ info [logstash_functional] Loading "data.json.gz"
[00:09:43]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.22"
[00:09:43]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.20"
[00:09:43]               │ info [logstash_functional] Skipped restore for existing index "logstash-2015.09.21"
[00:09:44]               │ info [invalid_scripted_field] Loading "mappings.json"
[00:09:44]               │ info [invalid_scripted_field] Loading "data.json.gz"
[00:09:44]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana_2/VRg9WK31TomV7R-Ao2Bhtg] deleting index
[00:09:44]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana_1/0Z0gV9GURHKusoeQ-GRQYw] deleting index
[00:09:44]               │ info [invalid_scripted_field] Deleted existing index [".kibana_2",".kibana_1"]
[00:09:44]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] applying create index request using v1 templates []
[00:09:44]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:09:44]               │ info [invalid_scripted_field] Created index ".kibana"
[00:09:44]               │ debg [invalid_scripted_field] ".kibana" settings {"index":{"number_of_replicas":"0","number_of_shards":"1"}}
[00:09:44]               │ info [invalid_scripted_field] Indexed 2 docs into ".kibana"
[00:09:45]               │ info Creating index .kibana_2.
[00:09:45]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] applying create index request using v1 templates []
[00:09:45]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:09:45]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] updating number_of_replicas to [0] for indices [.kibana_2]
[00:09:45]               │ info Reindexing .kibana to .kibana_1
[00:09:45]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] applying create index request using v1 templates []
[00:09:45]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:09:45]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] updating number_of_replicas to [0] for indices [.kibana_1]
[00:09:45]               │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] 8292 finished with response BulkByScrollResponse[took=24.3ms,timed_out=false,sliceId=null,updated=0,created=2,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:09:45]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana/o4Z_KNQNQzuF_eoh-9Gyxg] deleting index
[00:09:45]               │ info Migrating .kibana_1 saved objects to .kibana_2
[00:09:45]               │ debg Migrating saved objects config:7.0.0-alpha1, index-pattern:18ea0c30-2d77-11e8-93f2-6f8a391f33ef
[00:09:45]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana_2/SKhYXyNQQ8SedgVaHpp5TQ] update_mapping [_doc]
[00:09:45]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana_2/SKhYXyNQQ8SedgVaHpp5TQ] update_mapping [_doc]
[00:09:45]               │ info Pointing alias .kibana to .kibana_2.
[00:09:45]               │ info Finished in 470ms.
[00:09:45]               │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:09:46]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-oraclelinux-tests-xl-1586536333950054352] [.kibana_2/SKhYXyNQQ8SedgVaHpp5TQ] update_mapping [_doc]
[00:09:47]               │ debg navigating to discover url: http://localhost:6161/app/kibana#/discover
[00:09:47]               │ debg Navigate to: http://localhost:6161/app/kibana#/discover
[00:09:47]               │ debg ... sleep(700) start
[00:09:47]               │ debg browser[INFO] http://localhost:6161/app/kibana?_t=1586538857595#/discover 350 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:09:47]               │
[00:09:47]               │ debg browser[INFO] http://localhost:6161/bundles/app/kibana/bootstrap.js 9:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:09:47]               │ debg ... sleep(700) end
[00:09:47]               │ debg returned from get, calling refresh
[00:09:48]               │ debg browser[INFO] http://localhost:6161/app/kibana?_t=1586538857595#/discover 350 Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'self'". Either the 'unsafe-inline' keyword, a hash ('sha256-P5polb1UreUSOe5V/Pv7tc+yeZuJXiOi/3fqhGsU7BE='), or a nonce ('nonce-...') is required to enable inline execution.
[00:09:48]               │
[00:09:48]               │ debg browser[INFO] http://localhost:6161/bundles/app/kibana/bootstrap.js 9:19 "^ A single error about an inline script not firing due to content security policy is expected!"
[00:09:48]               │ debg currentUrl = http://localhost:6161/app/kibana#/discover
[00:09:48]               │          appUrl = http://localhost:6161/app/kibana#/discover
[00:09:48]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:09:51]               │ debg TestSubjects.find(kibanaChrome)
[00:09:51]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=10000
[00:09:51]               │ debg browser[INFO] http://localhost:6161/bundles/plugin/data/data.plugin.js 62:139970 "INFO: 2020-04-10T17:14:20Z
[00:09:51]               │        Adding connection to http://localhost:6161/elasticsearch
[00:09:51]               │
[00:09:51]               │      "
[00:09:51]               │ debg ... sleep(501) start
[00:09:51]               │ debg ... sleep(501) end
[00:09:51]               │ debg in navigateTo url = http://localhost:6161/app/kibana#/discover?_g=()&_a=(columns:!(_source),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:51]               │ debg --- retry.try error: URL changed, waiting for it to settle
[00:09:52]               │ debg ... sleep(501) start
[00:09:52]               │ debg ... sleep(501) end
[00:09:52]               │ debg in navigateTo url = http://localhost:6161/app/kibana#/discover?_g=()&_a=(columns:!(_source),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:52]               │ debg TestSubjects.exists(statusPageContainer)
[00:09:52]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:09:55]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:09:55]             └-: invalid scripted field error
[00:09:55]               └-> "before all" hook
[00:09:55]               └-> is rendered
[00:09:55]                 └-> "before each" hook: global before each
[00:09:55]                 │ debg TestSubjects.exists(discoverFetchError)
[00:09:55]                 │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="discoverFetchError"]') with timeout=2500
[00:09:58]                 │ debg --- retry.tryForTime error: [data-test-subj="discoverFetchError"] is not displayed
[00:09:58]                 │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/discover app errors invalid scripted field error is rendered.png"
[00:09:58]                 │ info Current URL is: http://localhost:6161/app/kibana#/discover?_g=()&_a=(columns:!(_source),index:%2718ea0c30-2d77-11e8-93f2-6f8a391f33ef%27,interval:auto,query:(language:kuery,query:%27%27),sort:!())
[00:09:58]                 │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/discover app errors invalid scripted field error is rendered.html
[00:09:58]                 └- ✖ fail: "discover app errors invalid scripted field error is rendered"
[00:09:58]                 │

Stack Trace

Error: expected false to equal true
    at Assertion.assert (packages/kbn-expect/expect.js:100:11)
    at Assertion.be.Assertion.equal (packages/kbn-expect/expect.js:227:8)
    at Assertion.be (packages/kbn-expect/expect.js:69:22)
    at Context.it (test/functional/apps/discover/_errors.js:41:40)

Kibana Pipeline / kibana-intake-agent / Jest Tests.packages/kbn-plugin-generator/sao_template.plugin generator sao integration skips files when answering no

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 16 times on tracked branches: https://github.com/elastic/kibana/issues/58250

Stack Trace

Error: expect(received).toBe(expected) // Object.is equality

Expected: "/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/packages/kbn-optimizer/target/.bootstrap-cache"
Received: "/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/packages/kbn-plugin-generator/sao_template/template/common/index.ts"
    at jest.requireMock.readFile.mockImplementation (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/packages/kbn-optimizer/src/optimizer/cache_keys.test.ts:60:20)
    at finalReturnValue (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-mock/build/index.js:480:41)
    at /var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-mock/build/index.js:489:13
    at mockConstructor (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-mock/build/index.js:182:19)
    at go$readFile (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/graceful-fs/graceful-fs.js:110:14)
    at Object.readFile (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/graceful-fs/graceful-fs.js:107:12)
    at Promise (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/universalify/index.js:13:12)
    at new Promise (<anonymous>)
    at Object.readFile (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/universalify/index.js:7:14)
    at Promise.all.paths.map.relative (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/majo/dist/majo.common.js:47:17)
    at Array.map (<anonymous>)
    at Majo.<anonymous> (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/majo/dist/majo.common.js:45:29)
    at Generator.next (<anonymous>)
    at c (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/majo/dist/majo.common.js:10:99)

Kibana Pipeline / kibana-intake-agent / Jest Tests.packages/kbn-plugin-generator/sao_template.plugin generator sao integration includes app when answering yes

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 16 times on tracked branches: https://github.com/elastic/kibana/issues/58251

Stack Trace

Error: expect(received).toBe(expected) // Object.is equality

Expected: "/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/packages/kbn-optimizer/target/.bootstrap-cache"
Received: "/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/packages/kbn-plugin-generator/sao_template/template/common/index.ts"
    at jest.requireMock.readFile.mockImplementation (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/packages/kbn-optimizer/src/optimizer/cache_keys.test.ts:60:20)
    at finalReturnValue (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-mock/build/index.js:480:41)
    at /var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-mock/build/index.js:489:13
    at mockConstructor (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/jest-mock/build/index.js:182:19)
    at go$readFile (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/graceful-fs/graceful-fs.js:110:14)
    at Object.readFile (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/graceful-fs/graceful-fs.js:107:12)
    at Promise (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/universalify/index.js:13:12)
    at new Promise (<anonymous>)
    at Object.readFile (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/universalify/index.js:7:14)
    at Promise.all.paths.map.relative (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/majo/dist/majo.common.js:47:17)
    at Array.map (<anonymous>)
    at Majo.<anonymous> (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/majo/dist/majo.common.js:45:29)
    at Generator.next (<anonymous>)
    at c (/var/lib/jenkins/workspace/elastic+kibana+pipeline-pull-request/kibana/node_modules/majo/dist/majo.common.js:10:99)

and 9 more failures, only showing the first 3.

History

💚 Build #28371 succeeded 410bdad
💚 Build #28365 succeeded 6aef516
💚 Build #27301 succeeded 9e35351

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

streamich added 4 commits February 18, 2020 13:45

feat: 🎸 add ability to execute expression in debug mode

b48cd00

feat: 🎸 store resolved arguments in debug information

be83bba

feat: 🎸 track function execution time and set "success" flag

ce8d2be

feat: 🎸 store debug information for functions that throw

9e35351

streamich requested a review from a team as a code owner February 18, 2020 13:34

streamich added release_note:plugin_api_changes Contains a Plugin API changes section for the breaking plugin API changes section. review Team:AppArch Team:Presentation Presentation Team for Dashboard, Input Controls, and Canvas v7.7.0 v8.0.0 labels Feb 18, 2020

streamich added the Feature:ExpressionLanguage Interpreter expression language (aka canvas pipeline) label Feb 18, 2020

streamich requested review from Dosant and crob611 February 20, 2020 08:02

Dosant approved these changes Feb 20, 2020

View reviewed changes

src/plugins/expressions/common/execution/execution.ts Outdated Show resolved Hide resolved

streamich commented Feb 21, 2020

View reviewed changes

src/plugins/expressions/common/execution/execution.test.ts Outdated Show resolved Hide resolved

src/plugins/expressions/common/execution/execution.ts Show resolved Hide resolved

streamich added 2 commits February 24, 2020 08:47

feat: 🎸 use performance.now, safe "fn" reference, fix typo

6aef516

Merge remote-tracking branch 'upstream/master' into expressions-debug…

410bdad

…-mode

ppisljar reviewed Feb 24, 2020

View reviewed changes

src/plugins/expressions/common/execution/execution.ts Show resolved Hide resolved

src/plugins/expressions/common/execution/execution.ts Show resolved Hide resolved

ppisljar approved these changes Feb 24, 2020

View reviewed changes

streamich merged commit 581f1bd into elastic:master Feb 24, 2020

streamich mentioned this pull request Feb 24, 2020

[7.x] Expressions debug mode (#57841) #58354

Merged

wylieconlon mentioned this pull request Sep 29, 2020

[Expressions] Finish connecting debug mode on expressions #78874

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expressions debug mode #57841

Expressions debug mode #57841

streamich commented Feb 18, 2020 •

edited by jinmu03

Loading

elasticmachine commented Feb 18, 2020

elasticmachine commented Feb 18, 2020

Dosant left a comment

ppisljar left a comment

kibanamachine commented Apr 10, 2020

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

Expressions debug mode #57841

Expressions debug mode #57841

Conversation

streamich commented Feb 18, 2020 • edited by jinmu03 Loading

Summary

Dev Docs

Checklist

For maintainers

elasticmachine commented Feb 18, 2020

elasticmachine commented Feb 18, 2020

Dosant left a comment

Choose a reason for hiding this comment

ppisljar left a comment

Choose a reason for hiding this comment

kibanamachine commented Apr 10, 2020

💔 Build Failed

Test Failures

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

History

streamich commented Feb 18, 2020 •

edited by jinmu03

Loading