Receiving transfers is not enabled after depositing required amount of RDN tokens

[agatsoh]

Thanks for filing a bug report :-)

Steps to Reproduce

Checkout to the #2888
version branch
Bump versions: 1.0.0 -> 1.1.0
Node A is browser node 0x07d036DbC602F0191a4bC8e217fe48d34cD4a230
Node B is a CLI node. 0x5d3c715dfec88ee0C1a14C2B6ed3b0878612581c
1.yarn install
2. yarn workspace raiden-ts build
3. cd raiden-dapp
4. yarn build --mode production
5. npx serve -l 8080 ./dist
6. cd ../raiden-cli
7. yarn build

Starting raiden CLI command

node build/index.js --eth-rpc-endpoint http://10.104.6.8:8545 --api-address "http://localhost:5010" --keystore-path /home/devaki/.ethereum/keystore/ --address 0x5d3c715dfec88ee0C1a14C2B6ed3b0878612581c  --password-file password_file.txt --log-file node-5d3c71-mainnet-9thAug2021.log --accept-disclaimer --enable-monitoring --enableMediation

Earlier receiving was disabled to Node A because of insufficient UDC balance. After depositing to the UDC when a direct transfer was attempted from Node B to A it got a The requested target doesn't receive transfers.

Expected Result

Actual Result

$ curl -i -X POST http://localhost:5010/api/v1/payments/0x6B175474E89094C44Da98b954EedeAC495271d0F/0x07d036DbC602F0191a4bC8e217fe48d34cD4a230 -H 'Content-Type: application/json' --data-raw '{"amount": "1000000000000000000"}'
HTTP/1.1 409 Conflict
X-Powered-By: Express
Access-Control-Allow-Origin: http://localhost:*/*
Vary: Origin
Content-Type: application/json; charset=utf-8
Content-Length: 74
ETag: W/"4a-EaZME1CrQ8sYB1bUMfe8sXCKddY"
Date: Mon, 09 Aug 2021 09:59:13 GMT
Connection: keep-alive
Keep-Alive: timeout=5

{"message":"The requested target doesn't receive transfers.","details":""}

2021-08-09T09:59:13.993Z [info] => action     {
  type: 'transfer/failure',
  payload: RaidenError: The requested target doesn't receive transfers.
      at Object.assert (/home/devaki/raidenforked/light-client/raiden-ts/dist:cjs/utils/error.js:167:23)
      at validateRouteTarget (/home/devaki/raidenforked/light-client/raiden-ts/dist:cjs/services/epics/helpers.js:160:13)
      at Object.getRoute$ (/home/devaki/raidenforked/light-client/raiden-ts/dist:cjs/services/epics/helpers.js:330:5)
      at /home/devaki/raidenforked/light-client/raiden-ts/dist:cjs/services/epics/pathfinding.js:38:412
      at doInnerSub (/home/devaki/raidenforked/light-client/node_modules/rxjs/dist/cjs/internal/operators/mergeInternals.js:21:26)
      at outerNext (/home/devaki/raidenforked/light-client/node_modules/rxjs/dist/cjs/internal/operators/mergeInternals.js:16:70)
      at OperatorSubscriber._this._next (/home/devaki/raidenforked/light-client/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:28:21)
      at OperatorSubscriber.Subscriber.next (/home/devaki/raidenforked/light-client/node_modules/rxjs/dist/cjs/internal/Subscriber.js:72:18)
      at /home/devaki/raidenforked/light-client/node_modules/rxjs/dist/cjs/internal/operators/withLatestFrom.js:57:28
      at OperatorSubscriber._this._next (/home/devaki/raidenforked/light-client/node_modules/rxjs/dist/cjs/internal/operators/OperatorSubscriber.js:28:21) {
    details: { target: '0x07d036DbC602F0191a4bC8e217fe48d34cD4a230' }
  },
  meta: {
    secrethash: '0xa2b270ed59ac8c87de2dc191fff94824593fdaa122bd3b45ac6aac5eb6fdeb0e',
    direction: 'sent'
  },
  error: true
}

Logs of Browser Node A

raiden_0x07d036DbC602F0191a4bC8e217fe48d34cD4a230_2021-08-09T10_17_07.767Z.log

Logs of CLI node B

node-5d3c71-mainnet-9thAug2021.log

Additional Infos (Optional)

- Browser & version
- Browser console errors
- Device / OS

[agatsoh]

Ok direct transfer from Node B to Node A went through after quite a long time.
This brings me to the conclusion that this is a problem purely with the PFS not updating the capabilites (and not with the LC) or
the sender raiden node does not quickly update the capabilites.

$ curl -i -X POST \
> http://localhost:5010/api/v1/payments/0x6B175474E89094C44Da98b954EedeAC495271d0F/0x07d036DbC602F0191a4bC8e217fe48d34cD4a230 \
> -H 'Content-Type: application/json' \
> --data-raw '{"amount": "1000000000000000000"}'
HTTP/1.1 200 OK
X-Powered-By: Express
Access-Control-Allow-Origin: http://localhost:*/*
Vary: Origin
Content-Type: application/json; charset=utf-8
Content-Length: 482
ETag: W/"1e2-YalfQ+kKhf5oLZ2UH1QslZgj0gE"
Date: Mon, 09 Aug 2021 10:51:17 GMT
Connection: keep-alive
Keep-Alive: timeout=5

{"event":"EventPaymentSentSuccess","initiator_address":"0x5d3c715dfec88ee0C1a14C2B6ed3b0878612581c","target_address":"0x07d036DbC602F0191a4bC8e217fe48d34cD4a230","token_address":"0x6B175474E89094C44Da98b954EedeAC495271d0F","amount":"1000000000000000000","identifier":"1628506277620","secret":"0x96bf400ef222119196eb1e31bb4f28230013118548f80f8f184024659fad4687","secret_hash":"0xc5c6a726bdc1759f407a8aadeeb998f605282988d3619201f1a505e3a6731fdb","log_time":"2021-08-09T10:51:17.925Z"}

[andrevmatos]

So, from what I can see, the partner didn't see the change in target's capability (becoming able to receive transfers, Receive: 1, after UDC deposit) because they had a WebRTC channel established, and we keep the presence cache indefinitely if the RTC channel stays open. We could send the new capabilities over the RTC channel with a new message, to update partner on this, but an easier and more compatible quickfix is to disconnect the RTC channels, to trigger partners to re-fetch our capabilities from PFS.
This happen so seldom that the reconnections wouldn't have a meaningful impact on transport, and would ensure we're up to date on this, although races are inherent from distributed systems, we should do our best to be synced eventually and asap.