Getting wrong results when target matches two predictions

[ShaiTAUB]

Consider this case:
pred = [[{'label': 'y', 'start': 0, 'end': 2}, {'label': 'x', 'start': 0, 'end': 2}]]
true = [[{'label': 'x', 'start': 0, 'end': 1}]]

Clearly, for tag 'x' and evaluation schema=Type, we expect to have 1 correct value, as there is a match in the label and an overlap in the span.
And indeed, when I run:

results = Evaluator(true=true, pred=pred, tags=['x']).evaluate()
results[1]['x']['ent_type']['correct']

I get a result of 1.

However, when I run:

results = Evaluator(true=true, pred=pred, tags=['x', 'y']).evaluate()
results[1]['x']['ent_type']['correct']

(asking for results for the two different tags)
I get a result of 0.

[omri374]

Swapping the order of the predictions returns the right values, hinting that this is indeed a bug

pred = [[{'label': 'x', 'start': 0, 'end': 2}, {'label': 'y', 'start': 0, 'end': 2}]]
true = [[{'label': 'x', 'start': 0, 'end': 1}]]

results = NERvaluator(true=true, pred=pred, tags=['x', 'y']).evaluate()
results[1]['x']['ent_type']['correct']

which returns 1