Why is there a small difference between the vector of a single sentence and the vector of a batch of sentences?

[wencan]

model = SentenceTransformer('LaBSE')
zh_vec = model.encode('可以给你认识的人打个电话。')
vecs = model.encode(['contacts who may know about a job?', '可以给你认识的人打个电话。'])

print(zh_vec.mean(), vecs1[1].mean())

got:
-0.011027637 -0.011027641

[ir2718]

@wencan

This is a known issue stemming from the underlying libraries (probably even torch)
#2312
huggingface/transformers#2401

Although, on second thought, when you instantiate a model this way, you're using it in training mode. It might be because of the way dropout behaves in training mode. Try adding model.eval() before calculating the embeddings.

[tomaarsen]

Hello!

@ir2718 is right, those other issues contain a bit more information. In short: there are slight differences, but significantly small that the embeddings are not notably affected.

As for the second possible explanation: model.encode moves the model to eval() mode automatically, so that shouldn't be the cause.

• Tom Aarsen