Lost payload in first uplink after a confirmed downlink

[cyberman54]

Testing downlink confirmation with current code (commit #6427a03) on EU868 with TTNv2 i found a strange effect. Not sure if this is a bug in LMIC, or maybe TTN or even my application.

Reproducable show case:

1. Sending 1 byte of payload as confirmed downlink from TTN console to device
2. Device receives payload correctly and sends ack
3. Ack is correctly shown in TTN console
4. The next unconfirmed uplink initiated by the device with (in my example) 17 bytes payload given to LMIC_setTxData2() results in an uplink with NO payload, that means, the payload is lost.

23081882: TXMODE, freq=867700000, **len=15**, SF=7, BW=125, CR=4/5, IH=0

I decoded the packet which is seen on the gateway in step 4.

Assuming hex-encoded packet
405E250126A00E0054D7E595

Message Type = Data
  PHYPayload = 405E250126A00E0054D7E595

( PHYPayload = MHDR[1] | MACPayload[..] | MIC[4] )
        MHDR = 40
  MACPayload = 5E250126A00E00
         MIC = 54D7E595

( MACPayload = FHDR | FPort | FRMPayload )
        FHDR = 5E250126A00E00
       FPort = 
  FRMPayload = 

      ( FHDR = DevAddr[4] | FCtrl[1] | FCnt[2] | FOpts[0..15] )
     DevAddr = 2601255E (Big Endian)
       FCtrl = A0
        FCnt = 000E (Big Endian)
       FOpts = 

Message Type = Unconfirmed Data Up
   Direction = up
        FCnt = 14
   FCtrl.ACK = true
   FCtrl.ADR = true

[terrillmoore]

@cyberman54 -- thanks for the report. I will investigate. I'm traveling today, so it may be a few days.

I have noticed that the TTN console doesn't display uplink messages consistently. You won't be able to match up to a decoder. You can see what's going on in the gateway traffic link, sort of.

The message you are showing is the zero-length (no port) ACK. The LMIC sends the ACK immediately -- it currently won't piggyback it on another message from the app. This results in exactly the message you are displaying. In EU, this will count against the 1% channel utilization constraint.

The compliance tests do exactly this.

The device uplink traffic won't show this packet. But the gateway will.

The FCntUp should be 15 for your next data.

Timing will change for this; it's possible even that this next uplink is completed with an error because "the LMIC is busy" when you submit.

The next batch of commits (on a branch) will give a better error return from LMIC_setTxData2(). But for now, check to see if the result from LMIC_setTxData() is non-zero, as that indicates an error.

[cyberman54]

My application does already checking the return value of LMIC_setTXData2(). It is zero in this case. Before calling LMIC_setTXData2() my app checks the LMIC eninge status to ensure that LMIC is not busy, otherwise my app postpones the send using a queue.

The point is: if the downlink before was unconfirmed, the uplink works as expected. But payload is missing in exactly the same scenario when the downlink was confirmed.

[cyberman54]

We can close this. You did hit the point, my app had a bug which resulted to not properly evaluate the return status of LMIC_setTXData2(). This caused the app pushing data to LMIC while LMIC is busy, instead of queuing the message. I fixed this and now it works.

Sorry to trouble you with this.

[terrillmoore]

@cyberman54 No trouble at all, I'm very grateful for your active and patient testing on this work.

[IoTopenTech]

Dear cyberman54 and Terrillmoore,
I am experiencing the same empty payload issue after downlink ACK confirmation.

Please, could you confirm that this is the intended behavior of the library (send a payload-empty frame for acknowledging the previous downlink)?
Kind regards from Madrid

[terrillmoore]

@IoTopenTech What you're seeing is an artifact of the TTN network and the TTN console. Nothing is getting lost, but the TTN console is not displaying things very clearly. The LMIC (as a design choice) does not delay uplink acknowledgements. So it sends a null uplink, with the ack bit set. Hence you get the green check mark on the ack of the confirmed downlink. (Without this, you would have had to wait until the next scheduled uplink to see the confirmation; that could be arbitrarily long. Although it's technically possible to set a timer, and do the ack after a little while, that adds test complexity and is even more dependent on the app not to go to sleep in the meanwhile.)