Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Device provisioning using NGSIv2 #615

Merged

Conversation

dcalvoalonso
Copy link
Contributor

Following the section Registrations of http://telefonicaid.github.io/fiware-orion/api/v2/latest/ a first implementation of Device/Entity provisioning using NGSIv2 has been implemented with this PR.
It covers:

  • Context registration
  • Entity creation
  • Removal of devices (unregister context and entity)
  • Unit tests
  • Correct linting

@dcalvoalonso
Copy link
Contributor Author

Update of context registration is still pending since although it is documented in the corresponding API documentation, it seems that it is not implemented yet in the CB.

@fgalan, could you please confirm this last point?

* For those usages not covered by the GNU Affero General Public License
* please contact with::[[email protected]]
*
* Modified by: Daniel Calvo - ATOS Research & Innovation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be Author: Daniel Calvo - ATOS Research & Innovation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is just a small adaptation of https://github.com/telefonicaid/iotagent-node-lib/blob/master/test/unit/provisioning/singleConfigurationMode-test.js for NGSIv2 features. To be honest, I am just a contributor bit not the original author.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NTC

* For those usages not covered by the GNU Affero General Public License
* please contact with::[[email protected]]
*
* Modified by: Daniel Calvo - ATOS Research & Innovation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Author: ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same reason of #615 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NTC

@@ -21,6 +21,7 @@
* please contact with::[email protected]
*
* Modified by: Federico M. Facca - Martel Innovate
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be:

Contributors: Daniel Calvo - ATOS Research & Innovation
Federico M. Facca - Martel Innovate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We agreed with @fgalan to use this approach (Modified by: xxx) and Federico has also followed the same style (please see https://github.com/telefonicaid/iotagent-node-lib/blob/master/lib/services/devices/deviceService.js in master branch). Do you think that it is more appropriate to use Contributors? In that case, we would need to change some of the headers of the master branch...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NTC (I guess)

@jmcanterafonseca
Copy link
Contributor

jmcanterafonseca commented May 29, 2018 via email

@dcalvoalonso
Copy link
Contributor Author

@jmcanterafonseca , we can open a new issue to discuss using Modified by or Contributors and do the appropriate PR. @fgalan do you agree?

@fgalan
Copy link
Member

fgalan commented May 29, 2018

@jmcanterafonseca , we can open a new issue to discuss using Modified by or Contributors and do the appropriate PR. @fgalan do you agree?

I agree. I think is a very minor issue... as minor as the diference between "Modified by" and "Contributors" is :D

@fgalan
Copy link
Member

fgalan commented May 29, 2018

With regards to:

Update of context registration is still pending since although it is documented in the corresponding API > documentation, it seems that it is not implemented yet in the CB.

@fgalan, could you please confirm this last point?

It is not implemented, as explained at https://fiware-orion.readthedocs.io/en/master/user/ngsiv2_implementation_notes/index.html#registrations:

PATCH /v2/registration/<id> is not implemented. Thus, registrations cannot be updated directly. I.e., updates must be done deleting and re-creating the registration. Please see this issue about this.

In order not to block the work in IOTA agents lib, would it be possible develop a "two step" update based on DELETE + POST as workaround (and leave a FIXME mark about removing that workaround once telefonicaid/fiware-orion#3007 gets implemented at Orion)?

@fgalan fgalan mentioned this pull request May 29, 2018
@dcalvoalonso
Copy link
Contributor Author

Final implementation of update context registration done with 5f67405. It follows a two-step approach: delete + new creation.

* @param {Object} newDevice Device object that will be stored in the database.
* @return {function} Handler to pass to the request() function.
*/
function createInitialEntityHandlerNgsi2(deviceData, newDevice, callback) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe existing createInitialEntityHandler() should be renamed to createInitialEntityHandlerNgsi1() ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with 056cba5

* @param {Object} updatedDevice Device object that will be stored in the database.
* @return {function} Handler to pass to the request() function.
*/
function updateEntityHandlerNgsi2(deviceData, updatedDevice, callback) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe existing updateEntityHandler() should be renamed to updateEntityHandlerNgsi1() ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updateEntityHandler() does not exist. As you can see in https://github.com/telefonicaid/iotagent-node-lib/blob/master/lib/services/devices/deviceService.js#L562, for ngsiv1, the createInitialEntity function is called also to perform the update. I guess that the original developers took benefit of the fact that Orion interprets APPEND as UPDATE if the entity already exists .

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If updateEntityHandler() doesn't exist (I didn't check it... I though so applying a "paraellism" to the create case) then my original comment doesn't makes sense :)

NTC

function updateEntityNgsi2(deviceData, updatedDevice, callback) {
var options = {
url: config.getConfig().contextBroker.url + '/v2/entities/' + String(deviceData.name) + '/attrs',
method: 'PUT',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PUT will replace all entiy attributes with the ones you sent from IOTA. I think this is not the desired behaviour... as the entity could be shared (i.e. has attributes out of the scope of IOTA used by other applications) that would be destroyed.

I understand that PATCH should be used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... or maybe POST (the difference is that PATCH raises an error if the attribute doesn't peviously exist, while POST creates it if it doesn't previously exist).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comment! I missed that part of the API documentation! :S

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in a34df7f

return attributeList;
}

function formatCommands(originalVector) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see several functions with the same name in different parts, in updateEntityNgsi2() and createInitialEntityNgsi2():

  • formatCommands()
  • formatAttributes()
  • jsonCat()

Are they the same? Could be refactored into a common place to avoid duplication?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with 8ce26bd

iotAgentLib.deactivate(done);
});
});

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe is a good idea to include FIXME marks in the utest cases that would need changes once telefonicaid/fiware-orion#3007 get implemented (including in the FIXME the URL of the issue)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with 37d07db

@@ -75,14 +75,13 @@ describe('Subscription tests', function() {
nock.cleanAll();
Copy link
Member

@fgalan fgalan Jun 1, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that after the changes in this PR, no usage of /v1 URLs in CB should remain. It would be a good idea to check it, e.g. recursive grep in test/unit/ngsiv2 directory.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked:

a620381@a620381-VirtualBox:~/workspace/finext/iotagent-node-lib$ grep -Rin NGSI9 test/unit/ngsiv2/
a620381@a620381-VirtualBox:~/workspace/finext/iotagent-node-lib$
a620381@a620381-VirtualBox:~/workspace/finext/iotagent-node-lib$ grep -Rin v1 test/unit/ngsiv2/
a620381@a620381-VirtualBox:~/workspace/finext/iotagent-node-lib$

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

NTC

@fgalan
Copy link
Member

fgalan commented Jun 1, 2018

Good work! :)

I have done a first round of (minor) comments.

contextBrokerMock
.matchHeader('fiware-service', 'smartGondor')
.matchHeader('fiware-servicepath', 'gardens')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

.reply(201, null, {'Location': '/v2/registrations/6319a7f5254b05844116584d'});

contextBrokerMock
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

.reply(201, null, {'Location': '/v2/registrations/6319a7f5254b05844116584d'});

contextBrokerMock
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

.reply(201, null, {'Location': '/v2/registrations/6319a7f5254b05844116584d'});

contextBrokerMock
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

.reply(201, null, {'Location': '/v2/registrations/6319a7f5254b05844116584d'});

contextBrokerMock
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock
.matchHeader('fiware-service', 'smartGondor')
.matchHeader('fiware-servicepath', '/gardens')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock
.matchHeader('fiware-service', 'smartGondor')
.matchHeader('fiware-servicepath', '/gardens')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock
.matchHeader('fiware-service', 'TestService')
.matchHeader('fiware-servicepath', '/testingPath')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock
.matchHeader('fiware-service', 'TestService')
.matchHeader('fiware-servicepath', '/testingPath')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock
.matchHeader('fiware-service', 'AlternateService')
.matchHeader('fiware-servicepath', '/testingPath')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock
.matchHeader('fiware-service', 'TestService')
.matchHeader('fiware-servicepath', '/testingPath')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock
.matchHeader('fiware-service', 'smartGondor')
.matchHeader('fiware-servicepath', '/gardens')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock
.matchHeader('fiware-service', 'smartGondor')
.matchHeader('fiware-servicepath', '/gardens')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock = nock('http://192.168.1.1:1026')
.matchHeader('fiware-service', 'smartGondor')
.matchHeader('fiware-servicepath', '/gardens')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

contextBrokerMock = nock('http://192.168.1.1:1026')
.matchHeader('fiware-service', 'smartGondor')
.matchHeader('fiware-servicepath', '/gardens')
.post('/v2/entities?options=upsert')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mock using post without payload. Comment explaining it needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed with fbc86fa

@fgalan
Copy link
Member

fgalan commented Jun 29, 2018

LGTM to 244eb8c...fbc86fa changes.

'fiware-servicepath': '/gardens'
},
json: utils.readExampleFile('./test/unit/examples/deviceProvisioningRequests/' +
'provisionDeviceActiveAtts.json')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the provisionDeviceActiveAtts.json file placed under test/unit/ngsiv2/examples?

Not very important, but up to now I understand it has been the criteria...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With 843f3ab I have added also the new test to ngsiv1 version. This was also missing there so I think it is good to check that it also works.

Therefore, we can maintain the new provisionDeviceActiveAtts.json file test/unit/examples.

Do you agree?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it makes sense... when the exac same .json is used for both v1 and v2 test variants it is ok to have it only in one place.

NTC

@fgalan
Copy link
Member

fgalan commented Jul 12, 2018

LGTM to fbc86fa...843f3ab changes.

@fgalan
Copy link
Member

fgalan commented Aug 3, 2018

LGTM 843f3ab...ad51f56

After e2e integration ok, the PR is ready to merge. Thanks @dcalvoalonso for all the effort! Good work!

@fgalan fgalan merged commit 0010aa0 into telefonicaid:master Aug 3, 2018
@dcalvoalonso
Copy link
Contributor Author

As always, thanks @fgalan for your feedback and contributions!! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants