Expose dense vector iterator in painless

[stu-elastic]

Painless does not have access to dense_vector at the moment except as an opaque value, users can pass it around but not access it's contents.

We can expose the vector as an iterator, which will give access to the data without revealing the internal representation.

cc: @mayya-sharipova @jtibshirani

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Scripting)

[mayya-sharipova]

I would suggest NOT to expose dense vector iterator.
There are two reasons for this:

1. I have not encountered a need to iterate over values of a dense_vector field, except probably the issue Bug report: Indexing a list of floats in a painless script seems to access the elements in sorted order #49695, which a user solved through a custom plugin. We provide predefined vector functions, and would like users to access vectors only through these functions.
2. Complex vector representation. Each vector is represented as a binary value that encodes not only an original vector values but also computed vector magnitude. In the future, we may add more data to this encoding. So even if a user gets this binary value, it will be challenging to decode it.

@stu-elastic I am wondering if you have any specific need to expose dense vector iterator?

[stu-elastic]

@mayya-sharipova this is based on a request from @honzakral. After chatting we thought an iterator was pretty lightweight.

I'm trying to understand point 2, if we iterated over a vector and provided the values, what's the issue there? We can simply ignore the magnitude at the end.

[mayya-sharipova]

@honzakral I am interested to learn about a use case to access vector values directly.

@stu-elastic

We can simply ignore the magnitude at the end.

For now depending on index version it is just magnitude at the end. But later we may add more metadata.

Were you planning to iterate over Binary DocValues as it would be tricky for a user to decode these docvalues?
Alternatively I can see how we can expose an iterator over float[] -- original vectors' values by first decoding them, something we do in our vector functions

[stu-elastic]

Alternatively I can see how we can expose an iterator over float[] -- original vectors' values by first decoding them, something we do in our vector functions

That's what we were thinking, allow users to get their data back.

[honzakral]

The use case we encountered with is the ability to access data in order, just as #49695. The idea was to store historical records (price at a point of time) where we are then interested in deltas between two arbitrary points (price yesterday compared to price a year ago).

Another use case that came up was a user implementing custom vector function in painless.

[mayya-sharipova]

@honzakral thanks, looks to be valid use cases to me.

That's what we were thinking, allow users to get their data back.

@stu-elastic thanks, makes sense. We need to think how to implement it as DenseVectorScriptDocValues needs to know an index version to decode vectors in a right way.

[mayya-sharipova]

Relevant request from another user of exposing vector functions in other painless contexts besides ScoreScript.CONTEXT.

[jdconrad]

@mayya-sharipova Just wanted to check to see if you would still like this to be exposed.

[coreation]

@jdconrad @mayya-sharipova You'll have to excuse me if this isn't on topic, but to my understanding the following example could be another use case to have access to the vector fields. I'm currently experiencing something weird in the following:

def aha =[]; def xt = params.filterVectors; for (int i=0; i< xt.size();i++) { aha.add(cosineSimilarity(xt[i], doc['media.full_body_dense_vector'].asList()));} throw new Exception(aha.toString());

The result that is added to the array, is always the cosineSimilarity outcome of the first iteration, however if you log the xt[i], it's definitely looping through the vectors we pass as parameters. This is my best hunch so far: is this because the script is compiled, where the function is compiled without having the variable values injected? I just can't wrap my head around how I should be using these functions within a for/while loop :/

If I'd have access to the dense vector field, I could write my cosSim. function which might get rid of that problem...but I'm not sure about that one. Any pointers on this would be great!

[mayya-sharipova]

@coreation I see the problem. Thank you for reporting it. I can reproduce your problem with this example:

PUT test_index
{
  "mappings": {
    "properties": {
      "v": {
        "type": "dense_vector",
        "dims" : 3
      }
    }
  }
}


POST test_index/_bulk
{ "index" : { "_id" : "1" } }
{"v" : [10, 10, 10]}
{ "index" : { "_id" : "2" } }
{"v" : [10, 20, 30]}

POST test_index/_explain/1
{
  "query": {
    "script_score" : {
      "query" : {"match_all" : {}},
      "script": {
        "source": """
            def aha =[];
            def xt = params.filterVectors; 
            for (int i=0; i< xt.size(); i++) 
            { 
            	aha.add(cosineSimilarity(xt[i], 'v'));
            } 
            Debug.explain(aha); 
        """, 
        "params": {
          "filterVectors": [[10, 10, 10], [10, 20, 30] ] 
        }
      }
    }
  }
}

Debug.explain(aha) returns "[0.999999889879181, 0.999999889879181]" instead of the expected "[0.999999889879181, 0.9258200697395147]

@jdconrad @stu-elastic The problem here is that the result of cosineSimilarity(xt[i], 'v') seems to be calculated only once in the loop and then cached and reused for all other loops iterations. What can be done in this case to ensure we calculate cosineSimilarity(xt[i], 'v') for each iteration?

[stu-elastic]

@coreation and @mayya-sharipova let's move this discussion to a new issue.

You'll want to look at the implementation of the CosineSimilarity whitelisted method.

[coreation]

@stu-elastic @mayya-sharipova thanks for following up! I've already made a separate issue for this and it has been triaged :) The ticket is #70437