Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

XContentParser with excludes filters emits invalid json in common cases #80142

Open
weizijun opened this issue Nov 1, 2021 · 6 comments
Open
Labels
>bug :Core/Infra/Core Core issues without another label Team:Core/Infra Meta label for core/infra team

Comments

@weizijun
Copy link
Contributor

weizijun commented Nov 1, 2021

Elasticsearch version (bin/elasticsearch --version): master

Plugins installed: [] no

JVM version (java -version):

OS version (uname -a if on a Unix-like system):

Description of the problem including expected versus actual behavior:

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) index creation, mappings, settings, query etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

  1. add a test in AbstractXContentFilteringTestCase
    public void testExcludesFailed() throws IOException {
        final Builder sample = builder -> builder.startObject().field("bar", "test1").field("foo", "test2").endObject();
        Builder expected = builder -> builder.startObject().field("bar", "test1").field("foo", "test2").endObject();
        testFilter(expected, sample, emptySet(), singleton("bar.xxx"));
    }
  1. throw the exception:
com.fasterxml.jackson.core.JsonGenerationException: Can not write a field name, expecting a value

	at __randomizedtesting.SeedInfo.seed([D5C7EEE02AA53137:388E1895E77A618D]:0)
	at com.fasterxml.jackson.core.JsonGenerator._reportError(JsonGenerator.java:2080)
	at com.fasterxml.jackson.core.json.UTF8JsonGenerator.writeFieldName(UTF8JsonGenerator.java:220)
	at com.fasterxml.jackson.core.JsonGenerator._copyCurrentContents(JsonGenerator.java:1938)
	at com.fasterxml.jackson.core.JsonGenerator.copyCurrentStructure(JsonGenerator.java:1914)
	at org.elasticsearch.xcontent.json.JsonXContentGenerator.copyCurrentStructure(JsonXContentGenerator.java:396)
	at org.elasticsearch.xcontent.XContentBuilder.copyCurrentStructure(XContentBuilder.java:1161)
	at org.elasticsearch.xcontent.support.filtering.AbstractXContentFilteringTestCase.filterOnParser(AbstractXContentFilteringTestCase.java:87)
	at org.elasticsearch.xcontent.support.filtering.AbstractXContentFilteringTestCase.testExcludesFailed(AbstractXContentFilteringTestCase.java:94)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)

I think the right action is passed the test.
It appears in excludes mode.
when excludes pattern contain than longer than input. like:

{
  "bar" : "test1",
  "foo" : "test2"
}

and the pattern is : bar.xxx.

@weizijun weizijun added >bug needs:triage Requires assignment of a team area label labels Nov 1, 2021
@nik9000 nik9000 self-assigned this Nov 1, 2021
@nik9000 nik9000 added :Core/Infra/Core Core issues without another label and removed needs:triage Requires assignment of a team area label labels Nov 1, 2021
@elasticmachine elasticmachine added the Team:Core/Infra Meta label for core/infra team label Nov 1, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@nik9000
Copy link
Member

nik9000 commented Nov 1, 2021

Oh yeah, that's fixed in modern jackson. Let's see about updating!

nik9000 added a commit to nik9000/elasticsearch that referenced this issue Nov 1, 2021
This grabs a new version of jackson and snakeyaml to get some bug fixes.
It also updates azure to stay compatible with the new jackson.

Closes elastic#66555
Closes elastic#67214
Closes elastic#80142
nik9000 added a commit to nik9000/elasticsearch that referenced this issue Nov 1, 2021
This grabs a new version of jackson and snakeyaml to get some bug fixes.
It also updates azure to stay compatible with the new jackson.

Closes elastic#66555
Closes elastic#67214
Closes elastic#80142
@weizijun
Copy link
Contributor Author

weizijun commented Nov 2, 2021

in the new jackson version, filterOnBuilder is ok, but filterOnParser is also bad

@nik9000 nik9000 changed the title xcontent excludes filter badcase XContentParser with excludes filters emits invalid json in common cases Nov 3, 2021
@nik9000
Copy link
Member

nik9000 commented Nov 3, 2021

in the new jackson version, filterOnBuilder is ok, but filterOnParser is also bad

The upgrade to jackson in #80160 did seem to fix it for me. That's just stuck....

@weizijun
Copy link
Contributor Author

weizijun commented Nov 3, 2021

in the new jackson version, filterOnBuilder is ok, but filterOnParser is also bad

The upgrade to jackson in #80160 did seem to fix it for me. That's just stuck....

yeah, It really fixed. Sorry, I misled you. I test the branch of #80160, it really work!

@nik9000
Copy link
Member

nik9000 commented Nov 3, 2021

yeah, It really fixed. Sorry, I misled you. I test the branch of #80160, it really work!

It happens to me a lot too. I'm glad it's fixed though.

@nik9000 nik9000 removed their assignment Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Core/Infra/Core Core issues without another label Team:Core/Infra Meta label for core/infra team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants