Skip to content

Commit

Permalink
Address Julie's feedback
Browse files Browse the repository at this point in the history
-decodeVectorMagnitude go back to Buffer
-correct documentation to use doc access methods for
	vector values and magnitude instead of get functions
- rename getVectorMagnitude to getMagnitude
- remove unnecessary yml tests
  • Loading branch information
mayya-sharipova committed Apr 8, 2021
1 parent d9dfdb1 commit 388b1f8
Show file tree
Hide file tree
Showing 6 changed files with 22 additions and 78 deletions.
16 changes: 8 additions & 8 deletions docs/reference/vectors/vector-functions.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ linearly scanned. Thus, expect the query time grow linearly
with the number of matched documents. For this reason, we recommend
to limit the number of matched documents with a `query` parameter.

This is the list of available vector functions:
This is the list of available vector functions and vector access methods:

1. `cosineSimilarity` – calculates cosine similarity
2. `dotProduct` – calculates dot product
3. `l1norm` – calculates L^1^ distance
4. `l2norm` - calculates L^2^ distance
5. `getVectorValue` – returns a vector's value as an array of floats
6. `getVectorMagnitude` – returns a vector's magnitude
5. `doc[<field>].vectorValue` – returns a vector's value as an array of floats
6. `doc[<field>].magnitude` – returns a vector's magnitude


Let's create an index with a `dense_vector` mapping and index a couple
Expand Down Expand Up @@ -210,10 +210,10 @@ The recommended way to access dense vectors is through `cosineSimilarity`,
`dotProduct`, `l1norm` or `l2norm` functions. But for custom use cases,
you can access dense vectors's values directly through the following functions:

- `float[] getVectorValue()` – returns a vector's value as an array of floats
- `doc[<field>].vectorValue` – returns a vector's value as an array of floats

- `float getVectorMagnitude()` – returns a vector's magnitude (available for
vectors created in the version 7.5 or later).
- `doc[<field>].magnitude` – returns a vector's magnitude as a float
(available for vectors created in the version 7.5 or later).

For example, the script below implements a cosine similarity using these
two functions:
Expand All @@ -235,8 +235,8 @@ GET my-index-000001/_search
},
"script": {
"source": """
float[] v = doc['my_dense_vector'].getVectorValue();
float vm = doc['my_dense_vector'].getVectorMagnitude();
float[] v = doc['my_dense_vector'].vectorValue;
float vm = doc['my_dense_vector'].magnitude;
float dotProduct = 0;
for (int i = 0; i < v.length; i++) {
dotProduct += v[i] * params.queryVector[i];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -189,55 +189,3 @@ setup:
- match: {hits.hits.0._id: "1"}
- match: {hits.hits.1._id: "2"}
- match: {hits.hits.1._score: 0.0}

---
"No sort, no aggs, no docvalue_fields are allowed":
- do:
index:
refresh: true
index: test-index
id: 1
body:
my_dense_vector: [10, 10, 10]

# sorting on dense_vector field is not supported
- do:
catch: bad_request
search:
index: test-index
body:
query:
match_all: {}
sort:
my_dense_vector

- match: { status: 400 }
- match: { error.root_cause.0.reason: "Field [my_dense_vector] of type [dense_vector] doesn't support sort" }

# aggs on dense_vector field are not supported
- do:
catch: bad_request
search:
index: test-index
body:
aggs:
my_agg:
terms:
field: my_dense_vector

- match: { status: 400 }
- match: { error.root_cause.0.reason: "Field [my_dense_vector] of type [dense_vector] doesn't support docvalue_fields or aggregations" }


# docvalue_fields of dense_vector field are not supported
- do:
catch: bad_request
search:
index: test-index
body:
query:
match_all: {}
docvalue_fields: ["my_dense_vector"]

- match: { status: 400 }
- match: { error.root_cause.0.reason: "Field [my_dense_vector] of type [dense_vector] doesn't support docvalue_fields or aggregations" }
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
script:
source: |
float s = 0;
for (def el : doc['v'].getVectorValue()) {
for (def el : doc['v'].vectorValue) {
s += el;
}
s;
Expand All @@ -51,7 +51,7 @@
- match: { hits.hits.2._score: 3 }


# check getVectorMagnitude() API
# check getMagnitude() API
- do:
headers:
Content-Type: application/json
Expand All @@ -61,7 +61,7 @@
script_score:
query: { "exists" : { "field" : "v" } }
script:
source: "doc['v'].getVectorMagnitude()"
source: "doc['v'].magnitude"

- match: { hits.hits.0._id: "3" }
- gte: {hits.hits.0._score: 3.3166}
Expand All @@ -82,7 +82,7 @@
script_score:
query: { match_all: { } }
script:
source: "doc['v'].getVectorValue()[0]"
source: "doc['v'].vectorValue[0]"

- match: { status: 400 }
- match: { error.root_cause.0.type: "script_exception" }
Expand All @@ -96,7 +96,7 @@
script_score:
query: { match_all: { } }
script:
source: "doc['v'].getVectorMagnitude()"
source: "doc['v'].magnitude"

- match: { status: 400 }
- match: { error.root_cause.0.type: "script_exception" }
Expand All @@ -113,8 +113,8 @@
query: { "exists": { "field": "v" } }
script:
source: |
float[] v = doc['v'].getVectorValue();
float vm = doc['v'].getVectorMagnitude();
float[] v = doc['v'].vectorValue;
float vm = doc['v'].magnitude;
int closestPv = 0;
float maxCosSim = -1;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,11 @@ public static int denseVectorLength(Version indexVersion, BytesRef vectorBR) {
*/
public static float decodeVectorMagnitude(Version indexVersion, BytesRef vectorBR) {
assert indexVersion.onOrAfter(Version.V_7_5_0);
int offset = vectorBR.offset + vectorBR.length - INT_BYTES;
int intValue = ((vectorBR.bytes[offset] & 0xFF) << 24) |
((vectorBR.bytes[offset+1] & 0xFF) << 16) |
((vectorBR.bytes[offset+2] & 0xFF) << 8) |
(vectorBR.bytes[offset+3] & 0xFF);
return Float.intBitsToFloat(intValue);
ByteBuffer byteBuffer = ByteBuffer.wrap(vectorBR.bytes, vectorBR.offset, vectorBR.length);
return byteBuffer.getFloat(vectorBR.offset + vectorBR.length - 4);
}

public static float getVectorMagnitude(Version indexVersion, BytesRef vectorBR) {
public static float getMagnitude(Version indexVersion, BytesRef vectorBR) {
if (vectorBR == null) {
throw new IllegalArgumentException("A document doesn't have a value for a vector field!");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ BytesRef getEncodedValue() {
@Override
public BytesRef get(int index) {
throw new UnsupportedOperationException("accessing a vector field's value through 'get' or 'value' is not supported!" +
"Use 'getVectorValue' or 'getVectorMagnitude' instead!'");
"Use 'vectorValue' or 'magnitude' instead!'");
}

/**
Expand All @@ -60,8 +60,8 @@ public float[] getVectorValue() {
/**
* Get dense vector's magnitude
*/
public float getVectorMagnitude() {
return VectorEncoderDecoder.getVectorMagnitude(indexVersion, value);
public float getMagnitude() {
return VectorEncoderDecoder.getMagnitude(indexVersion, value);
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
#
class org.elasticsearch.xpack.vectors.query.DenseVectorScriptDocValues {
float[] getVectorValue()
float getVectorMagnitude()
float getMagnitude()
}
class org.elasticsearch.script.ScoreScript @no_import {
}
Expand Down

0 comments on commit 388b1f8

Please sign in to comment.