Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] LangPainlessClientYamlTestSuiteIT test {yaml=painless/20_scriptfield/Scripted Field with error accessing an unsupported field via the script fields api} failing #79986

Closed
ywangd opened this issue Oct 28, 2021 · 2 comments · Fixed by #80026
Assignees
Labels
:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache Team:Core/Infra Meta label for core/infra team >test-failure Triaged test failures from CI

Comments

@ywangd
Copy link
Member

ywangd commented Oct 28, 2021

This failed on my PR and is reproducible

Build scan:
https://gradle-enterprise.elastic.co/s/co5sy2mcxu2xg/tests/:modules:lang-painless:yamlRestTest/org.elasticsearch.painless.LangPainlessClientYamlTestSuiteIT/test%20%7Byaml=painless%2F20_scriptfield%2FScripted%20Field%20with%20error%20accessing%20an%20unsupported%20field%20via%20the%20script%20fields%20api%7D

Reproduction line:
./gradlew ':modules:lang-painless:yamlRestTest' --tests "org.elasticsearch.painless.LangPainlessClientYamlTestSuiteIT.test {yaml=painless/20_scriptfield/Scripted Field with error accessing an unsupported field via the script fields api}" -Dtests.seed=18C365613D7AD5B6 -Dtests.locale=de-DE -Dtests.timezone=SystemV/CST6 -Druntime.java=17

Applicable branches:
master, 8.0

Reproduces locally?:
Yes

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.painless.LangPainlessClientYamlTestSuiteIT&tests.test=test%20%7Byaml%3Dpainless/20_scriptfield/Scripted%20Field%20with%20error%20accessing%20an%20unsupported%20field%20via%20the%20script%20fields%20api%7D

Failure excerpt:

java.lang.AssertionError: Failure at [painless/20_scriptfield:160]: expected [400] status code but api [search] returned [200 OK] [---
took: 12
timed_out: false
_shards:
  total: 2
  successful: 1
  skipped: 0
  failed: 1
  failures:
  - shard: 0
    index: "test"
    node: "8soJAuywRHKzkmoG2YSV7g"
    reason:
      type: "script_exception"
      reason: "runtime error"
      script_stack:
      - "org.elasticsearch.script.field.DelegateDocValuesField.getValue(DelegateDocValuesField.java:56)"
      - "field('foo').getValue('')"
      - "            ^---- HERE"
      script: "field('foo').getValue('')"
      lang: "painless"
      position:
        offset: 12
        start: 0
        end: 25
      caused_by:
        type: "unsupported_operation_exception"
        reason: "field [foo] is not supported through the fields api, use [doc] instead"
hits:
  total: 1
  max_score: 1.0
  hits: []
]

  at __randomizedtesting.SeedInfo.seed([18C365613D7AD5B6:90975ABB9386B84E]:0)
  at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:489)
  at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.test(ESClientYamlSuiteTestCase.java:462)
  at jdk.internal.reflect.GeneratedMethodAccessor12.invoke(null:-1)
  at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:568)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:375)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:824)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:475)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:375)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:831)
  at java.lang.Thread.run(Thread.java:833)

  Caused by: java.lang.AssertionError: expected [400] status code but api [search] returned [200 OK] [---
  took: 12
  timed_out: false
  _shards:
    total: 2
    successful: 1
    skipped: 0
    failed: 1
    failures:
    - shard: 0
      index: "test"
      node: "8soJAuywRHKzkmoG2YSV7g"
      reason:
        type: "script_exception"
        reason: "runtime error"
        script_stack:
        - "org.elasticsearch.script.field.DelegateDocValuesField.getValue(DelegateDocValuesField.java:56)"
        - "field('foo').getValue('')"
        - "            ^---- HERE"
        script: "field('foo').getValue('')"
        lang: "painless"
        position:
          offset: 12
          start: 0
          end: 25
        caused_by:
          type: "unsupported_operation_exception"
          reason: "field [foo] is not supported through the fields api, use [doc] instead"
  hits:
    total: 1
    max_score: 1.0
    hits: []
  ]

    at org.junit.Assert.fail(Assert.java:88)
    at org.elasticsearch.test.rest.yaml.section.DoSection.execute(DoSection.java:365)
    at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:478)
    at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.test(ESClientYamlSuiteTestCase.java:462)
    at jdk.internal.reflect.GeneratedMethodAccessor12.invoke(null:-1)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:568)
    at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
    at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
    at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
    at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
    at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:375)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:824)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:475)
    at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
    at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
    at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
    at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
    at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
    at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
    at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
    at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:375)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:831)
    at java.lang.Thread.run(Thread.java:833)

@ywangd ywangd added :Core/Infra/Scripting Scripting abstractions, Painless, and Mustache >test-failure Triaged test failures from CI labels Oct 28, 2021
@elasticmachine elasticmachine added the Team:Core/Infra Meta label for core/infra team label Oct 28, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@stu-elastic
Copy link
Contributor

It looks like one shard is empty and so not all shards have failures, leading to a success response.

stu-elastic added a commit to stu-elastic/elasticsearch that referenced this issue Oct 28, 2021
The 20_scriptfield:"Scripted Field with error accessing an
unsupported field via the script fields api" test expects to fail because
the field is not yet supported in the fields API.

However, if the number of shards in the target index is randomly more
than one, the empty shard will succeed leading to a 200 response rather
than the expected 400 response.

This change forces the number of shards to 1 to avoid this scenario.

Fixes: elastic#79986
stu-elastic added a commit that referenced this issue Oct 28, 2021
The 20_scriptfield:"Scripted Field with error accessing an
unsupported field via the script fields api" test expects to fail because
the field is not yet supported in the fields API.

However, if the number of shards in the target index is randomly more
than one, the empty shard will succeed leading to a 200 response rather
than the expected 400 response.

This change forces the number of shards to 1 to avoid this scenario.

Fixes: #79986
stu-elastic added a commit to stu-elastic/elasticsearch that referenced this issue Oct 28, 2021
The 20_scriptfield:"Scripted Field with error accessing an
unsupported field via the script fields api" test expects to fail because
the field is not yet supported in the fields API.

However, if the number of shards in the target index is randomly more
than one, the empty shard will succeed leading to a 200 response rather
than the expected 400 response.

This change forces the number of shards to 1 to avoid this scenario.

Fixes: elastic#79986
Backport: 9511c54
stu-elastic added a commit that referenced this issue Oct 28, 2021
The 20_scriptfield:"Scripted Field with error accessing an
unsupported field via the script fields api" test expects to fail because
the field is not yet supported in the fields API.

However, if the number of shards in the target index is randomly more
than one, the empty shard will succeed leading to a 200 response rather
than the expected 400 response.

This change forces the number of shards to 1 to avoid this scenario.

Fixes: #79986
Backport: 9511c54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Core/Infra/Scripting Scripting abstractions, Painless, and Mustache Team:Core/Infra Meta label for core/infra team >test-failure Triaged test failures from CI
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants