PoC link layer recommended changes.

[refugeesus]

Pretty straightforward here.

Single packets are obviously not reliable or descriptive of a point to point link between any number of Hotspots. These recommended changes can fix this for us. I'm fairly confident all Hotspot owners would be appreciative of higher reliability surrounding their PoC participation as well. Anyone interested give this a read, hopefully we don't have any serious gripes here. Happy to discuss implementation in greater detail and how we handle multiple simultaneous PoCs in an area if anyone needs it.

[refugeesus]

Rendered View: https://github.com/refugeesus/HIP/blob/rfk/poc-ll/0021-poc-link-layer/0021-poc-link-layer.md

[atgParasitics]

Some thoughts:

• an alternative is to just PoC / beacon more frequently, say every 20-30 blocks instead of every 120 blocks (note you mention PoC only occurring 2-3 times a day but it should be ~12 times a day (every 120 blocks).
• Wont any bit error in a droplet fail the CRC check on the concentrator causing the whole droplet to be dropped? may require removing CRC filtering.
• I'm not super confident PoC should be more reliable than "normal" data. Maybe I can just pass PoC using this method over X distance due to this added error correction but no way any LoRa device could be received by my hotspot at X distance.

The PoC link layer should attempt to provide reliable transmissions of data without requiring high error correction overhead or the need for numerous packet re-transmissions or acknowledgments.

I would say this is the goal of LoRa / LoRaWAN. This is built in and what devices expect without having to implement fountain codes. I'd argue (devil's advocate) PoC should be more device-like transmitting device representative payload sizes, datarates, and power. If we want more data to verify models or get averages from just beacon more.

Of course beaconing more means more on-chain transactions / verifications / etc. But those could also be addressed. Say maybe each challenger challenges 10 hotspots at a time and each receipt to CG includes info on each transmitter

[refugeesus]

@atgParasitics interesting comments. Seems like you know exactly what is said in internal channels!

an alternative is to just PoC / beacon more frequently, say every 20-30 blocks instead of every 120 blocks (note you mention PoC only occurring 2-3 times a day but it should be ~12 times a day (every 120 blocks).

It doesn't really matter if you have a single transmission 12 or 100 times a day spread out over the whole day (~1440 blocks). The property we are taking advantage of here is the statistical model used in a data stream.

Wont any bit error in a droplet fail the CRC check on the concentrator causing the whole droplet to be dropped? may require removing CRC filtering.

I would remove CRC filtering when this is implemented yes. It would be obsolete.

I'm not super confident PoC should be more reliable than "normal" data. Maybe I can just pass PoC using this method over X distance due to this added error correction but no way any LoRa device could be received by my hotspot at X distance.

Not sure what you're asking here. We don't use LoRaWAN in PoC at all. If you want to know if a single device sending one LoRa modulated signal could possibly reach a gateway, retrieving one droplet is sufficient. PoC needs to be able to judge coverage, including its quality. Implementing this we are able to derive these metrics with high enough resolution for the results to be meaningful as they are all related to each other. Droplets themselves resemble device like payloads with the associated data rates and transmit power.

Of course beaconing more means more on-chain transactions / verifications / etc. But those could also be addressed. Say maybe each challenger challenges 10 hotspots at a time and each receipt to CG includes info on each transmitter

This is accomplished with witness data currently, but for the same reason the in path PoC participants don't assist us in this data collection, the witnesses do not either.

[Carniverous19]

dont listen to that @atgParasitics guy. Clearly an idiot who doesn't know how to manage github accounts properly ;)

Not sure what you're asking here. We don't use LoRaWAN in PoC at all. If you want to know if a single device sending one LoRa modulated signal could possibly reach a gateway, retrieving one droplet is sufficient.

Yea maybe should have said LoRa not LoRaWAN. LoRa has error correction built in (the coding rate) so this method seems to be additional error detection / correction on top of that. So my original concern is that you could successfully receive droplets with this additional error correction that wouldnt be present for "normal" device data.
That being said, if all of this is used as metrics then maybe its ok with the understanding that only droplets received with 0 errors (after accounting for the FEC built into LoRa) would be successfully received from a device. Basically if the CRC would fail, even if disabled, then a normal LoRa device would have a corrupted packet and probably would be dropped. I guess I do see the value in the additional detail assuming the bit errors are part of evaluating the link.

Basically for a "good" signal between hotspots you would go from:

"out of 20 beacons from A, B heard 17 without error".

To:

out of the 20 droplets transmitted by A, hotspot B received 19 of them, 17 without any errors and 2 with X bits of error that would fail CRC".

Similarly for a bad / very intermittent link between hotspots would go from:

"out of 20 beacons from A, B heard 1 without error"

To:

"out of 20 droplets from A, B heard 2, 1 without error and 1 with X bits of error that would fail CRC"

Lastly you may get a lot more lora packets out of the packet forwarder if you disable CRC. In my non scientific tests on a a few in my area I do see the rate of messages go up, so these would have to be handled somehow, you probably dont want to submit payloads with bad CRC to routers / etc. device owners likely dont want to pay for corrupted data. This is minor and can be addressed elsewhere.

[refugeesus]

LoRa has error correction built in (the coding rate) so this method seems to be additional error detection / correction on top of that. So my original concern is that you could successfully receive droplets with this additional error correction that wouldnt be present for "normal" device data.

The hardware error correction encoding for all LoRa components is flawed to not working. Semtech openly admits this. The LDPC encoding will add some gain, sure, and you have to subscribe to device behavior ruling the judgement of coverage to think this was detrimental. We are using this ecc for more than just the correction here.

Basically if the CRC would fail, even if disabled, then a normal LoRa device would have a corrupted packet and probably would be dropped. I guess I do see the value in the additional detail assuming the bit errors are part of evaluating the link.

A BER evaluation is the intent.

Lastly you may get a lot more lora packets out of the packet forwarder if you disable CRC. In my non scientific tests on a a few in my area I do see the rate of messages go up, so these would have to be handled somehow, you probably dont want to submit payloads with bad CRC to routers / etc. device owners likely dont want to pay for corrupted data. This is minor and can be addressed elsewhere.

This implementation doesn't concern devices. Only PoC transactions between gateways are affected. How/Why/Who pays is also out of scope here I think.

[Carniverous19]

This implementation doesn't concern devices. Only PoC transactions between gateways are affected. How/Why/Who pays is also out of scope here I think.

Sure, agreed a bit out of scope but maybe in the (small) unintended consequences. By turning off CRC filtering it will be off for all packets received by the concentrator so you cant assume data passed to the miner passes CRC which you can today. So you may want to introduce code to check CRC for non PoC payloads before further processing / forwarding to a router, etc. or somehow handle this. What you dont want is simultaneously with "hey we introduced this great new droplet PoC feature" is having everyone on #console say "whats going on I'm getting 1-3 garbled packets for every valid one I receive, the data going through console is all garbage now"

[mfalkvidd]

As a benchmark for how much additional traffic will be forwarded, I checked the statistics for 24h on one of my gateways. This was the number of messages:

CRC Error	84919
CRC Ok	103650

So forwarding packets with failed CRC might almost double the amount of traffic from the gateway to the miner. This will affect systems where the miner is in the cloud and the gateway uses a LTE backhaul.

I am not aware of any gateways that can selectively forward error packets, so they will either forward all or none.

Adding code to the miner to drop packets with invalid CRC (to avoid getting bad packets in the console and paying for them) should be possible, but will not help the cases using LTE backhaul.

[refugeesus]

@Carniverous19 @mfalkvidd certainly things we should be noting as required peripheral changes to implementing this. Not sure where it is best to list these things other than keeping a personal note in my notebook. I'll add them to a ticket for our task management tool of choice as well.

[Carniverous19]

The other tangential concern I have both with existing PoC methods and this proposal is that you mention PoC packets arent LoRaWAN but device packets are. One thing I noticed running the middleman code is how easy it is to differentiate PoC data from device data and thats even assuming they were both LoRaWAN. With PoC payloads being different than device data its possible for middleman or other gaming software to treat them differently.

Part of me thinks it should be fairly indistinguishable so you cant give PoC data special treatment. For example, as @mfalkvidd mentions there may be cost concerns for LTE backhaul, why not just drop ALL packets that aren't PoC packets? that would mean you only use a tiny amount of LTE data and are still capable of getting full PoC rewards. Maybe this is better discussed in discord as its not specific to this proposal, just something I wasnt aware of.

[jamiew]

Reviewed, numbered and merged for discussion.

Please direct future questions, feedback or comments to the central tracking issue: #78