Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move ambiguous object field name detection into DotExpandingXContentParser #82359

Merged
merged 4 commits into from
Jan 10, 2022

Conversation

romseygeek
Copy link
Contributor

Detecting when a field name contains double dots, or starts with a dot, is currently
done by the splitAndValidatePath method on DocumentParser. However it makes
more sense to do this as part of the DotExpandingXContentParser which actually
does the work of converting field names containing dots to XContent objects.

@romseygeek romseygeek added :Search Foundations/Mapping Index mappings, including merging and defining field types >refactoring v8.1.0 labels Jan 10, 2022
@romseygeek romseygeek requested review from nik9000 and javanna January 10, 2022 11:51
@romseygeek romseygeek self-assigned this Jan 10, 2022
@elasticmachine elasticmachine added the Team:Search Meta label for search team label Jan 10, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@romseygeek
Copy link
Contributor Author

This leaves splitAndValidatePaths in DocumentMapper for now as it is still used by the dynamic mapper construction code, but we can then remove it as part of #81449 .

Note that this is just a refactor and doesn't try and fix #28948

@@ -461,7 +461,6 @@ private static void innerParseObject(DocumentParserContext context, ObjectMapper
while (token != XContentParser.Token.END_OBJECT) {
if (token == XContentParser.Token.FIELD_NAME) {
currentFieldName = context.parser().currentName();
splitAndValidatePath(currentFieldName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it ok to omit this call now? What was it doing and where is that method called now? Seems like the new method incorporates some of its logic?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new method is pretty much a straight copy. It's called in the DotExpandingXContentParser, which InternalDocumentParserContext creates to wrap the source document's xcontent parser. The existing call sites were places where we previously split up field names, but that all happens in the expanding parser now so it makes more sense to handle things there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok then I guess I am unclear on why the original method stays around, are there still places where we call it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep it's still called from createDynamicUpdate(). This gets completely refactored in #81449 so it will go away entirely there.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, thanks! Can we in the meantime share the code between the two then, or are there subtle differences?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are slightly different because the x-content lib doesn't have access to Strings, but in implementation they are the same. Given that they're private static though, and that the copy in DocumentParser will be removed immediately in a follow up, I'd prefer to keep two versions for the moment?

@romseygeek
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/docs

@romseygeek
Copy link
Contributor Author

@elasticmachine update branch

@romseygeek romseygeek merged commit 0ca3db6 into elastic:master Jan 10, 2022
@romseygeek romseygeek deleted the mapper/field-name-validation branch January 10, 2022 15:29
for (String part : parts) {
// check if the field name contains only whitespace
if (part.isEmpty()) {
throw new IllegalArgumentException("object field cannot contain only whitespace: ['" + fullFieldPath + "']");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@romseygeek is the mention of "object" in this error and the next one accurate? Don't we split any path regardless of the token we are reading from the parser?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think that's a leftover, and it was probably inaccurate even before this change. It should be 'field name'

@@ -48,7 +48,7 @@ public Token nextToken() throws IOException {

private void expandDots() throws IOException {
String field = delegate().currentName();
String[] subpaths = field.split("\\.");
String[] subpaths = splitAndValidatePath(field);
if (subpaths.length == 0) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@romseygeek this check is redundant now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is, good catch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>refactoring :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Search Meta label for search team v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants