Issue with 4.x

[obaid]

Describe the bug

When providing a training phrase with just an "entity" the intent is not detected properly. This is working in 3.10.2 but not in 4.x.

const { NlpManager } = require('node-nlp');

manager = new NlpManager({ languages: ['en'], nlu: { useNoneFeature: false, log: false } });
manager.addNamedEntityText("FirstName", "John", ["en"], ["John","john"] );

manager.addDocument("en", "%FirstName%", "GetFirstNameIntent");

manager.train().then(() => {
    manager.process('john').then(result => console.log('result: ', result.intent));
});

To Reproduce
Run the code above and it should print:
GetFirstNameIntent

Expected behavior
We expect it to determine GetFirstNameIntent as the intent

Screenshots
N/A

Desktop (please complete the following information):

• Version 4.0.2

[obaid]

@jesus-seijas-sp any idea what could be going on here? Any help would be appreciated. :)

[jesus-seijas-sp]

Hello, the problem cames because the reduceEdges is currently less aggressive. You're replicating the same text for the entity: ['John', 'john']. It produces that the entity is identified twice, so it generates the utterance with the entity name twice replicated, that does not match. It will work correctly if you replace ['John', 'john'] with ['John'] or ['john']. As the text is normalized before entity match, it will work. Even with that, I fixed the reduce Edges in this PR: #436

[obaid]

Thanks @jesus-seijas-sp it worked when I removed "John" from the array. But then I added another possible value for that entity and it is returning the correct intent:

const { NlpManager } = require('node-nlp');

manager = new NlpManager({ languages: ['en'], nlu: { useNoneFeature: false, log: false } });
manager.addNamedEntityText("FirstName", "John", ["en"], [] );
manager.addNamedEntityText("FirstName", "Mary", ["en"], ["maryam"] );

manager.addDocument("en", "%FirstName%", "GetFirstNameIntent");

manager.train().then(() => {
    manager.process('john').then(result => console.log('result: ', result));
});

This should return GetFirstNameIntent intent back.

[jesus-seijas-sp]

Hello,
In this last example there is no reason why it should return GetFirstNameIntent, as the input is "john" but this is not defined as being an entity... the only text for an entity that you have is "maryam".

[obaid]

Maybe I am missing something but don't these two lines add John and Mary to it:

manager.addNamedEntityText("FirstName", "John", ["en"], [] );
manager.addNamedEntityText("FirstName", "Mary", ["en"], ["maryam"] );

FirstName variable can have any value between "John" and "Mary"..

[jesus-seijas-sp]

Hello,

Nope... Imagine that you have an online restaurant. You have food and beverages, and you name all your products with a code, unknown by the user. And also the names you have for the user can have sinonyms, like this:

manager.addNamedEntityText('food', 'f01', ['en'], ['pizza']);
manager.addNamedEntityText('food', 'f02', ['en'], ['pasta', 'spaghetti']);
manager.addNamedEntityText('food', 'f03', ['en'], ['burger', 'hamburger']);
manager.addNamedEntityText('beverage', 'b01', ['en'], ['coke', 'coca-cola']);
manager.addNamedEntityText('beverage', 'b02', ['en'], ['beer']);

So I have two entities, food and beverage. Food has 3 options by code, but the code is not something added into synonyms, because can be internal. Another way to understand it, if you have entities for airports, you'll want your users to tell you the name of the airport or the city, but internally you want the airport code.

[obaid]

Thanks for the clarification, appreciate the example since that helps build a better mental model. :)

[obaid]

@jesus-seijas-sp One last question, is addNamedEntityText case sensitive?

i.e.

manager.addNamedEntityText('food', 'f01', ['en'], ['pizza']);
manager.addDocument("en", "%food%", "foodIntent");

would manager.process('Pizza') yield in the correct intent (FoodIntent)?

[jesus-seijas-sp]

Hello, as there is a normalization before all, the normalization does convert to lower case... so is case unsensitive.

[obaid]

👍 Thanks