Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Manual backport: Fix suggestion classes for Term, Phrase, and Completion. (#477) #529

Merged
merged 1 commit into from
Jun 14, 2023
Merged

Manual backport: Fix suggestion classes for Term, Phrase, and Completion. (#477) #529

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
### Removed

### Fixed

- Fixed Suggesters for Completion, Term, and Phrase and refactored the Suggestion class ([#477](https://github.com/opensearch-project/opensearch-java/pull/477))
### Security

## [2.5.0] - 06/02/2023
Expand Down
212 changes: 212 additions & 0 deletions USER_GUIDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,11 @@
- [Search for the documents](#search-for-the-documents)
- [Get raw JSON results](#get-raw-json-results)
- [Search documents using a match query](#search-documents-using-a-match-query)
- [Search documents using suggesters](#search-documents-using-suggesters)
- [App Data class](#app-data-class)
- [Using completion suggester](#using-completion-suggester)
- [Using term suggester](#using-term-suggester)
- [Using phrase suggester](#using-phrase-suggester)
- [Bulk requests](#bulk-requests)
- [Aggregations](#aggregations)
- [Delete the document](#delete-the-document)
Expand Down Expand Up @@ -175,6 +180,213 @@ for (int i = 0; i < searchResponse.hits().hits().size(); i++) {
}
```

## Search documents using suggesters

### App Data class

```java
public static class AppData {

private int intValue;
private String msg;

public int getIntValue() {
return intValue;
}

public void setIntValue(int intValue) {
this.intValue = intValue;
}

public String getMsg() {
return msg;
}

public void setMsg(String msg) {
this.msg = msg;
}
}
```

### Using completion suggester

```java
String index = "completion-suggester";

Property intValueProp = new Property.Builder()
.long_(v -> v)
.build();
Property msgCompletionProp = new Property.Builder()
.completion(c -> c)
.build();
client.indices().create(c -> c
.index(index)
.mappings(m -> m
.properties("intValue", intValueProp)
.properties("msg", msgCompletionProp)));

AppData appData = new AppData();
appData.setIntValue(1337);
appData.setMsg("foo");

client.index(b -> b
.index(index)
.id("1")
.document(appData)
.refresh(Refresh.True));

appData.setIntValue(1338);
appData.setMsg("foobar");

client.index(b -> b
.index(index)
.id("2")
.document(appData)
.refresh(Refresh.True));

String suggesterName = "msgSuggester";

CompletionSuggester completionSuggester = FieldSuggesterBuilders.completion()
.field("msg")
.size(1)
.build();
FieldSuggester fieldSuggester = new FieldSuggester.Builder().prefix("foo")
.completion(completionSuggester)
.build();
Suggester suggester = new Suggester.Builder()
.suggesters(Collections.singletonMap(suggesterName, fieldSuggester))
.build();
SearchRequest searchRequest = new SearchRequest.Builder()
.index(index)
.suggest(suggester)
.build();

SearchResponse<AppData> response = client.search(searchRequest, AppData.class);
```

### Using term suggester

```java
String index = "term-suggester";

// term suggester does not require a special mapping
client.indices().create(c -> c
.index(index));

AppData appData = new AppData();
appData.setIntValue(1337);
appData.setMsg("foo");

client.index(b -> b
.index(index)
.id("1")
.document(appData)
.refresh(Refresh.True));

appData.setIntValue(1338);
appData.setMsg("foobar");

client.index(b -> b
.index(index)
.id("2")
.document(appData)
.refresh(Refresh.True));

String suggesterName = "msgSuggester";

TermSuggester termSuggester = FieldSuggesterBuilders.term()
.field("msg")
.size(1)
.build();
FieldSuggester fieldSuggester = new FieldSuggester.Builder().text("fool")
.term(termSuggester)
.build();
Suggester suggester = new Suggester.Builder()
.suggesters(Collections.singletonMap(suggesterName, fieldSuggester))
.build();
SearchRequest searchRequest = new SearchRequest.Builder()
.index(index)
.suggest(suggester)
.build();

SearchResponse<AppData> response = client.search(searchRequest, AppData.class);
```

### Using phrase suggester

```java
String index = "test-phrase-suggester";

ShingleTokenFilter shingleTokenFilter = new ShingleTokenFilter.Builder().minShingleSize("2")
.maxShingleSize("3")
.build();

Analyzer analyzer = new Analyzer.Builder()
.custom(new CustomAnalyzer.Builder().tokenizer("standard")
.filter(Arrays.asList("lowercase", "shingle")).build())
.build();

TokenFilter tokenFilter = new TokenFilter.Builder()
.definition(new TokenFilterDefinition.Builder()
.shingle(shingleTokenFilter).build())
.build();

IndexSettingsAnalysis indexSettingsAnalysis = new IndexSettingsAnalysis.Builder()
.analyzer("trigram", analyzer)
.filter("shingle", tokenFilter)
.build();

IndexSettings settings = new IndexSettings.Builder().analysis(indexSettingsAnalysis).build();

TypeMapping mapping = new TypeMapping.Builder().properties("msg", new Property.Builder()
.text(new TextProperty.Builder().fields("trigram", new Property.Builder()
.text(new TextProperty.Builder().analyzer("trigram").build())
.build()).build())
.build()).build();

client.indices().create(c -> c
.index(index)
.settings(settings)
.mappings(mapping));

AppData appData = new AppData();
appData.setIntValue(1337);
appData.setMsg("Design Patterns");

client.index(b -> b
.index(index)
.id("1")
.document(appData)
.refresh(Refresh.True));

appData.setIntValue(1338);
appData.setMsg("Software Architecture Patterns Explained");

client.index(b -> b
.index(index)
.id("2")
.document(appData)
.refresh(Refresh.True));

String suggesterName = "msgSuggester";

PhraseSuggester phraseSuggester = FieldSuggesterBuilders.phrase()
.field("msg.trigram")
.build();
FieldSuggester fieldSuggester = new FieldSuggester.Builder().text("design paterns")
.phrase(phraseSuggester)
.build();
Suggester suggester = new Suggester.Builder()
.suggesters(Collections.singletonMap(suggesterName, fieldSuggester))
.build();
SearchRequest searchRequest = new SearchRequest.Builder()
.index(index)
.suggest(suggester)
.build();

SearchResponse<AppData> response = client.search(searchRequest, AppData.class);
```

## Bulk requests

```java
Expand Down
85 changes: 68 additions & 17 deletions java-client/src/main/java/org/opensearch/client/json/ExternallyTaggedUnion.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,21 @@
package org.opensearch.client.json;

import org.opensearch.client.util.TaggedUnion;


import jakarta.json.stream.JsonGenerator;
import jakarta.json.stream.JsonParser;
import jakarta.json.stream.JsonParsingException;

import java.util.ArrayList;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.function.BiFunction;
import java.util.function.Function;

import javax.annotation.Nullable;

import static jakarta.json.stream.JsonParser.Event;

Expand All @@ -50,31 +57,43 @@
* encodes a name+type in a single JSON property.
*
*/
public interface ExternallyTaggedUnion {
public class ExternallyTaggedUnion {

private ExternallyTaggedUnion() {}

/**
* A deserializer for externally-tagged unions. Since the union variant discriminant is provided externally, this cannot be a
* regular {@link JsonpDeserializer} as the caller has to provide the discriminant value.
*/
class Deserializer<Union extends TaggedUnion<?, Member>, Member> {
public static class Deserializer<Union extends TaggedUnion<?, ?>, Member> {
private final Map<String, JsonpDeserializer<? extends Member>> deserializers;
private final BiFunction<String, Member, Union> unionCtor;
private final Function<Member, Union> unionCtor;
@Nullable
private final BiFunction<String, JsonData, Union> unKnownUnionCtor;

public Deserializer(Map<String, JsonpDeserializer<? extends Member>> deserializers, BiFunction<String, Member, Union> unionCtor) {
public Deserializer(Map<String, JsonpDeserializer<? extends Member>> deserializers,
Function<Member, Union> unionCtor) {
this.deserializers = deserializers;
this.unionCtor = unionCtor;
this.unKnownUnionCtor = null;
}

public Deserializer(Map<String, JsonpDeserializer<? extends Member>> deserializers,
Function<Member, Union> unionCtor, BiFunction<String, JsonData, Union> unKnownUnionCtor) {
this.deserializers = deserializers;
this.unionCtor = unionCtor;
this.unKnownUnionCtor = unKnownUnionCtor;
}

/**
* Deserialize a union value, given its type.
*/
public Union deserialize(String type, JsonParser parser, JsonpMapper mapper) {
public Union deserialize(String type, JsonParser parser, JsonpMapper mapper, Event event) {
JsonpDeserializer<? extends Member> deserializer = deserializers.get(type);
if (deserializer == null) {
throw new JsonParsingException("Unknown variant type '" + type + "'", parser.getLocation());
if (unKnownUnionCtor != null) {
return unKnownUnionCtor.apply(type, JsonData._DESERIALIZER.deserialize(parser, mapper, event));
}
}

return unionCtor.apply(type, deserializer.deserialize(parser, mapper));
return unionCtor.apply(deserializer.deserialize(parser, mapper, event));
}

/**
Expand All @@ -86,8 +105,9 @@ public TypedKeysDeserializer<Union> typedKeys() {
}
}

class TypedKeysDeserializer<Union extends TaggedUnion<?, ?>> extends JsonpDeserializerBase<Map<String, Union>> {
public static class TypedKeysDeserializer<Union extends TaggedUnion<?, ?>> extends JsonpDeserializerBase<Map<String, Union>> {
Deserializer<Union, ?> deserializer;

protected TypedKeysDeserializer(Deserializer<Union, ?> deser) {
super(EnumSet.of(Event.START_OBJECT));
this.deserializer = deser;
Expand All @@ -107,22 +127,53 @@ public void deserializeEntry(String key, JsonParser parser, JsonpMapper mapper,
int hashPos = key.indexOf('#');
if (hashPos == -1) {
throw new JsonParsingException(
"Property name '" + key + "' is not in the 'type#name' format. Make sure the request has 'typed_keys' set.",
parser.getLocation()
);
"Property name '" + key
+ "' is not in the 'type#name' format. Make sure the request has 'typed_keys' set.",
parser.getLocation());
}

String type = key.substring(0, hashPos);
String name = key.substring(hashPos + 1);

targetMap.put(name, deserializer.deserialize(type, parser, mapper));
targetMap.put(name, deserializer.deserialize(type, parser, mapper, parser.next()));
}
}

public static <T extends TaggedUnion<?, ?>> JsonpDeserializer<Map<String, List<T>>> arrayDeserializer(
TypedKeysDeserializer<T> deserializer) {
return JsonpDeserializer.of(
EnumSet.of(Event.START_OBJECT),
(parser, mapper, event) -> {
Map<String, List<T>> result = new HashMap<>();
String key = null;
while ((event = parser.next()) != Event.END_OBJECT) {
JsonpUtils.expectEvent(parser, event, Event.KEY_NAME);
// Split key and type
key = parser.getString();
int hashPos = key.indexOf('#');

String type = key.substring(0, hashPos);
String name = key.substring(hashPos + 1);

List<T> list = new ArrayList<>();
JsonpUtils.expectNextEvent(parser, Event.START_ARRAY);
try {
while ((event = parser.next()) != Event.END_ARRAY) {
list.add(deserializer.deserializer.deserialize(type, parser, mapper, event));
}
} catch (Exception e) {
throw e;
}
result.put(name, list);
}
return result;
});
}

/**
* Serialize an externally tagged union using the typed keys encoding.
*/
static <T extends JsonpSerializable & TaggedUnion<? extends JsonEnum, ?>> void serializeTypedKeys(
public static <T extends JsonpSerializable & TaggedUnion<? extends JsonEnum, ?>> void serializeTypedKeys(
Map<String, T> map, JsonGenerator generator, JsonpMapper mapper
) {
generator.writeStartObject();
Expand All @@ -133,7 +184,7 @@ public void deserializeEntry(String key, JsonParser parser, JsonpMapper mapper,
/**
* Serialize an externally tagged union using the typed keys encoding, without the enclosing start/end object.
*/
static <T extends JsonpSerializable & TaggedUnion<? extends JsonEnum, ?>> void serializeTypedKeysInner(
public static <T extends JsonpSerializable & TaggedUnion<? extends JsonEnum, ?>> void serializeTypedKeysInner(
Map<String, T> map, JsonGenerator generator, JsonpMapper mapper
) {
for (Map.Entry<String, T> entry: map.entrySet()) {
Expand Down
Loading