Beacon node rejecting attestations with UnknownHeadBlock

[mcdee]

Description

Sending attestations to the beacon node commonly results in an UnknownHeadBlock

Version

1.0.0

Present Behaviour

A submitted attestation is rejected, 50-80% of the time. A sample error is:

  {
    "code": 400,
    "message": "BAD_REQUEST: error processing attestations",
    "failures": [
      {
        "index": 0,
        "message": "Verification: UnknownHeadBlock { beacon_block_root: 0xc871ed9342683aa165845027c0d50b65153c4ce2e5691b071075dcc21b5c8bea }"
      },
      {
        "index": 1,
        "message": "Verification: UnknownHeadBlock { beacon_block_root: 0xc871ed9342683aa165845027c0d50b65153c4ce2e5691b071075dcc21b5c8bea }"
      }
    ]
  }

However this block appears to be valid, as per https://pyrmont.beaconcha.in/block/49745

Expected Behaviour

The attestation should be accepted.

Steps to resolve

Lighthouse should accept the attestation even though it is currently unaware of the head block. Given that the attestation has been signed, the validator client is not going to be producing another one for that epoch.

[paulhauner]

So this one is a bit murky in my opinion. I'll let you know my thoughts, perhaps I'm missing something.

Like all good arguments, lets start by making an uncertain assumption about an unclear specification:

The following validations MUST pass before forwarding the attestation on the subnet.
...

[IGNORE] The block being voted for (attestation.data.beacon_block_root) has been seen (via both gossip and non-gossip sources) (a client MAY queue aggregates for processing once block is retrieved).

Let's assume that "forwarding the attestation" also covers "forwarding" an attestation from the API -> Gossipsub. So, regarding that forwarding condition there are two types of clients:

1. Ones that ignore attestations for an unknown head.
2. Ones that queue attestations for an unknown head.

Lighthouse is presently (1), we'd like to be (2) but there is presently no established effort to make it happen.

So, if you're (1) client I'd say that ignoring/rejecting the attestation from the API is the correct behaviour as per the specification. If you're a (2) client then I suspect the ideal behaviour would be to accept the attestation from the API and maybe publish it later, once that head block has been seen.

I think the most important factor at play here is that when a client sends us an attestation with an unknown head block we query that client for the block (this is a method of discovering new chains). I'm not certain how Lighthouse will behave if another client fails to respond to this request, but I suspect it will down-vote them because they have either clearly defied the specification or ignored/lost our request. Perhaps @divagant-martian and/or @blacktemplar can chime in here with some clarity.

So, in summary, I think it's a little unclear how a client should handle an attestation to an unknown head. One the one hand we have the interests of the VC pushing out an attestation and then on the other hand we have the interests of the BN maintain a good score on the network (which ultimately is in the interests of the VC, too).

[AgeManning]

I think #635 is also somewhat relevant here.

And yep, if a client doesn't send us the block they referenced they get penalized.

[mcdee]

Given that #635 would address this in (in my opinion) a satisfactory way I'll close this issue in favour of that one.