Average vector as reference for person identification?

[xperroni]

I'm using OSNet for people identification against a pre-recorded database. Each person in the database is represented by a set of M vectors, each computed from a picture taken at a previous time. To confirm a person's identity, the system:

1. Retrieves the set M corresponding to the claimed identity;
2. Collects a new picture of the person, then extracts a new vector p representing their appearance;
3. Computes the distance from p to every vector in M;
4. Takes the closest distance as the measurement of how similar p is to M;
5. Compares the similarity measurement to an arbitrary threshold to decide whether the person's appearance matches the pre-recorded set.

After looking at these plots, I'm wondering whether I could just compute the mean and standard deviation of M, compute the distance between p and the mean of M, then decide if p is a match by comparing the distance to some multiple of the standard deviation.

Any arguments against this updated approach?