NGSIv2 Notifications not including the Fiware-Service header for sth comet

[Siedlerchr]

I stumbled across a bug which prevents me from creating a STH comet subscription. The issuie is similar to the old NGSIv1 related issue #2584
I am trying to create a subscription for STH comet with the NGSIv2 subscription format using
NGSIv2 legacy attributes.
STH-Comet version 2.6.0 and orion 2.2.0.
I created the minimal setup without cygnus.
Both are running from docker on the same server and are accessible from the outside.

requests with ERROR in the last 60s interval
time=2019-08-22T08:16:39.107Z | lvl=WARN | corr=n/a | trans=n/a | op=OPER_STH_SERVER_LOG | from=n/a | srv=n/a | subsrv=n/a | comp=STH | msg=error=child "fiware-service" fails because [fiware-service is required]
time=2019-08-22T08:16:39.109Z | lvl=WARN | corr=2a5b5150-c4b5-11e9-be70-0242ac120105 | trans=fdef8685-5a63-4012-a88d-73bc836b2e68 | op=OPER_STH_POST | from=n/a | srv=n/a | subsrv=/ | comp=STH | msg=POST /notify, event={"request":"1566461799088:sth-comet:15:jzl0yeny:10001","timestamp":1566461799108,"tags":["validation","error","headers"],"data":{"data":null,"isBoom":true,"isServer":false,"output":{"statusCode":400,"payload":{"statusCode":400,"error":"Bad Request","message":"error=child \"fiware-service\" fails because [fiware-service is required]","validation":{"source":"headers","keys":[]}},"headers":{}}},"internal":true}

It neither works when I pass the fiware-service header and fiware-service, it doesn't matter if it's present or not.

curl -X POST \
  http://example.com:1026/v2/subscriptions/ \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: something' \
  -H 'fiware-servicepath: /' \
  -d '{
  "description": "Notify STH-Comet of all water level changes",
  "subject": {
    "entities": [
      {
        "idPattern": "WaterLevel.*"
      }
    ],
    "condition": {
      "attrs": [
      	"waterLevel"
      ]
    }
  },
  "notification": {
    "http": {
      "url": "http://example.com:8666/notify"
    },
    "attrs": [
      "waterLevel"
    ],
    "attrsFormat": "legacy"
  }
}'

[fgalan]

It would be great to know exactly which notification is Orion sending to STH. In order to do that just kill the STH process and start a listening process on the same port (e.g. nc).

[Siedlerchr]

First attempt without any addtional headers:

POST /notify HTTP/1.1
User-Agent: orion/2.2.0 libcurl/7.29.0
Host: example.com:8666
Fiware-Servicepath: /
Accept: application/json
Content-Length: 17181
Content-Type: application/json; charset=utf-8
Fiware-Correlator:  <GUID>

So the fiware-service-path is sent, but not the required fiware-service header.

When I add a 'fiware-service' header with value 'something' I don't receive any data. This is probaly because I don't have a fiware-service or service-path defined in orion.
When I supply and empty fiware-service header I have the same output as above.

[fgalan]

How Orion is running (i.e. ps ax | grep contextBroker)?

[Siedlerchr]

Inside a docker container:

/usr/bin/contextBroker -fg -multiservice -ngsiv1Autocast -dbhost mongo-db -logLevel ERROR -noCache -corsOrigin __ALL -reqMutexPolicy none

[fgalan]

Thanks for the information!

I can try to reproduce the case. You have provided me the curl you use to create the subscription and the way you run CB. Finally, could you provide me the update you are using to trigger notifications (in curl format), please?

[Siedlerchr]

Hi,
It's already the automatic initial notification after registering the subscription:

1. Register the Subscription
2. Check with list all subscriptions, see lastSucess code 400 for the sth registration.
3. Run docker-logs fiware-sth-comet on the container image and see the error about the fiware service path.

I attached the relevant parts of the docker-compose files.

services:
  mongo-db:
    image: mongo:3.6
    hostname: mongo-db
    container_name: db-mongo
#
#    ports:
#      - "27017:27017"
    networks:
      - default
    command: --bind_ip_all --smallfiles
    volumes:
      - mongo-db:/data
    restart: always

  orion:
    image: fiware/orion:2.2.0
    hostname: orion
    container_name: fiware-orion
    depends_on:
      - mongo-db
    networks:
      - default
    ports:
      - "1026:1026"
    command: -dbhost mongo-db -logLevel ERROR -noCache -corsOrigin __ALL -reqMutexPolicy none
    healthcheck:
      test: curl --fail -s http://orion:1026/version || exit 1
    restart: always

  sth-comet:
    image: fiware/sth-comet:2.6.0
    hostname: sth-comet
    container_name: fiware-sth-comet
    depends_on:
        - mongo-db
    networks:
        - default
    ports:
        - "8666:8666"
    environment:
        - STH_HOST=0.0.0.0
        - STH_PORT=8666
        - DB_PREFIX=sth_
        - DB_URI=mongo-db:27017
        - LOGOPS_LEVEL=DEBUG
    restart: always

volumes:
  mongo-db: ~

[fgalan]

It's already the automatic initial notification after registering the subscription:

Others notifications apart from initial notification also have the same problem? Or are they working fine including the fiware-service header?

[Siedlerchr]

Others notifications apart from initial notification also have the same problem? Or are they working fine including the fiware-service header?

I tested an update operation, e.g. changed the attribute of an entity with a PATH operation and I do get the same error:
No fiware-service header is sent.

time=2019-08-27T08:05:46.360Z | lvl=WARN | corr=7949b960-c8a1-11e9-9646-0242ac120105 | trans=ba58b735-992b-4949-b2e3-2ada25a7bc9c | op=OPER_STH_POST | from=n/a | srv=n/a | subsrv=/ | comp=STH | msg=POST /notify, event={"request":"1566893146357:sth-comet:15:jztjlt6k:10001","timestamp":1566893146360,"tags":["validation","error","headers"],"data":{"data":null,"isBoom":true,"isServer":false,"output":{"statusCode":400,"payload":{"statusCode":400,"error":"Bad Request","message":"error=child \"fiware-service\" fails because [fiware-service is required]","validation":{"source":"headers","keys":[]}},"headers":{}}},"internal":true}

The entity update PATCH operation:

curl -X PATCH \
  http://example.com:1026/v2/entities/urn:ngsi-ld:WaterLevel:1/attrs \
  -H 'Content-Type: application/json' \
  -d '{
    "waterLevel": {
        "type": "Integer",
        "value": 666
    }
}'

Running nc shows the following output when I execute the PATCH operation:

POST /notify HTTP/1.1
User-Agent: orion/2.2.0 libcurl/7.29.0
Host: example.com:8666
Fiware-Servicepath: /
Accept: application/json
Content-Length: 299
Content-Type: application/json; charset=utf-8
Fiware-Correlator: <GUID>

{"subscriptionId":"5d64e40a3d45bfb08943e7dd","originator":"localhost","contextResponses":[{"contextElement":{"type":"WaterLevel","isPattern":"false","id":"urn:ngsi-ld:WaterLevel:1","attributes":[{"name":"waterLevel","type":"Integer","value":666}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]}                                                                                                                                                                                                                                                                                                                                                                                                                                                          

[fgalan]

Thanks you for the feedback! I'll try to reproduce the case based in your information and report my findings here.

[fgalan]

I have done the following tests:

---

Start CB with the same parametrization (except my DB runs in localhost) and a fresh DB (i.e. no orion* databases are shown in show dbs at mongo shell at start time):

contextBroker-2.2.0 -fg -multiservice -ngsiv1Autocast -dbhost localhost -logLevel ERROR -noCache -corsOrigin __ALL -reqMutexPolicy none

Check version is as expected:

curl localhost:1026/version

{
"orion" : {
  "version" : "2.2.0",
  "uptime" : "0 d, 0 h, 0 m, 9 s",
  "git_hash" : "5a46a70de9e0b809cce1a1b7295027eea0aa757f",
  "compile_time" : "nodate",
  "compiled_by" : "fermin",
  "compiled_in" : "neodeb",
  "release_date" : "nodate",
  "doc" : "https://fiware-orion.rtfd.io/en/2.2.0/"
}
}

Create entity in service "something" and subservice "/":

curl -X POST \
  http://localhost:1026/v2/entities \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: something' \
  -H 'fiware-servicepath: /' \
  -d '{
    "id": "urn:ngsi-ld:WaterLevel:1",
	"type": "meter",
    "waterLevel": {
        "type": "Integer",
        "value": 0
    }
}'

Create subscription. Note the curl statement is same as yours, except url which point to localhost nc listening process.

curl -X POST \
  http://localhost:1026/v2/subscriptions/ \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: something' \
  -H 'fiware-servicepath: /' \
  -d '{

  "description": "Notify STH-Comet of all water level changes",
  "subject": {
    "entities": [
      {
        "idPattern": "WaterLevel.*"
      }
    ],
    "condition": {
      "attrs": [
      	"waterLevel"
      ]
    }
  },
  "notification": {
    "http": {
      "url": "http://localhost:8666/notify"
    },
    "attrs": [
      "waterLevel"
    ],
    "attrsFormat": "legacy"
  }
}'

I get at nc:

POST /notify HTTP/1.1
User-Agent: orion/2.2.0 libcurl/7.38.0
Host: localhost:8666
Fiware-Service: something
Fiware-Servicepath: /
Accept: application/json
Content-Length: 291
Content-Type: application/json; charset=utf-8
Fiware-Correlator: f42a7080-ca7b-11e9-9ef5-000c29173617

{"subscriptionId":"5d680065d6ca695cba579a01","originator":"localhost","contextResponses":[{"contextElement":{"type":"meter","isPattern":"false","id":"urn:ngsi-ld:WaterLevel:1","attributes":[{"name":"waterLevel","type":"Integer","value":0}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]}

Now updating the entity using PATCH. Note that the curl statement is same as yours except this actually included the fiware-service and fiware-service path headers (I understand their omissing is a typo in your curl):

curl -X PATCH \
  http://localhost:1026/v2/entities/urn:ngsi-ld:WaterLevel:1/attrs \
  -H 'Content-Type: application/json' \
  -H 'fiware-service: something' \
  -H 'fiware-servicepath: /' \
  -d '{
    "waterLevel": {
        "type": "Integer",
        "value": 666
    }
}'

I get at nc:

POST /notify HTTP/1.1
User-Agent: orion/2.2.0 libcurl/7.38.0
Host: localhost:8666
Fiware-Service: something
Fiware-Servicepath: /
Accept: application/json
Content-Length: 293
Content-Type: application/json; charset=utf-8
Fiware-Correlator: 2f772bd8-ca7c-11e9-b86e-000c29173617

{"subscriptionId":"5d680065d6ca695cba579a01","originator":"localhost","contextResponses":[{"contextElement":{"type":"meter","isPattern":"false","id":"urn:ngsi-ld:WaterLevel:1","attributes":[{"name":"waterLevel","type":"Integer","value":666}]},"statusCode":{"code":"200","reasonPhrase":"OK"}}]}

---

In both cases the nc output shows the expected result, i.e. with fiware-service and fiware-servicepath headers.

Thus, I'm afraid I'm not able to reproduce the problem. What I'd suggest is you to try to reproduce exactly the same "test script" (starting with a fresh DB and without any proxy or whatever extra element... I mean, just CB, mongo, nc and curl) to check if you get the same result. Maybe in that process we can have some insight of what can be going on.

[Siedlerchr]

Try creating a subscription and an entity with no fiware service header.
Then sth comet will complain about the missing header.

[fgalan]

Ok... now I think I understand what you mean from the beginning :) Let me try to explain how Orion works.

• If the original update request have fiware-service, then Orion propagates that header in associated notifications.
• If the original update request doesn't have fiware-service, then Orion doesn't include any fiware-service header in associated notifications.

This behaviour is consistent and make sense in the most of the cases (if Orion doesn't know the service, he cannot make it up). However, it happens that some of the Context Providers to which Orion sends context information may require fiware-service as mandatory information (and STH is one of those).

Thus, in this situation there are two alternatives:

1. Use entities that always belong to a service. You can use some special token for "no-service" if you need such semantics, e.g. `fiware-service: none".
2. Modify STH to support notifications without service. For instance, setting a default service to use in that case (as Perseo for instance does https://github.com/telefonicaid/perseo-fe/blob/master/config.js#L188)

The cheaper option uses to be number 1.

[Siedlerchr]

Ah okay, now I understand as well. From my point of view that sth-comet should ignore the header if it's not present or use a default one as perseo. I modified my application to now send the header, still think it's some kind of workaround. Makes it harder to quick test things in the browser

[fgalan]

Although Orion allows it, not using fiware-service header is an anti-pattern. It has drawbacks, i.e. you cannot evolve your application to use the FIWARE security framework, as this security framework needs fiware-service (along with fiware-servicepath and x-auth-token) to work.

I'd recommend you to use Postman to do quick testing. It is almost as easiest to use as a browser :) and very much powerful.

[fgalan]

Having clarified the way Orion works, I think this issue can be closed. Do you agree @Siedlerchr ?

(Maybe a new issue could be created in fiware-sth repository about the implementation of a default service and subservice, as Perseo does)

[Siedlerchr]

Thanks for the clarification again. I now use a fiware service header and it's fine. I just thought that this header is part of a specification across all services.