Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adr: mvp light client #311

Merged
merged 30 commits into from
May 14, 2021
+299 −7
Merged

adr: mvp light client #311

Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
9b18942
First stab on mvp light client adr
liamsi May 3, 2021
8aabbfa
Add more details about IPFS & testing
liamsi May 4, 2021
283cfc8
Add thoughts about DAHeader via RPC
liamsi May 4, 2021
8ab6c09
minor cleanup
liamsi May 4, 2021
64fdfaf
more details on required changes:
liamsi May 4, 2021
c10756f
Update LightBlock comment
liamsi May 4, 2021
96f9903
properly introduce DAHeader abbreviation and add a negative plus some…
liamsi May 4, 2021
58b1506
Add Store requirements & note on skipping verification UX
liamsi May 5, 2021
7d06ae2
fix sentence
liamsi May 6, 2021
cef2776
Add Provider changes
liamsi May 6, 2021
9e8989b
Update docs/lazy-adr/adr-004-mvp-light-client.md
liamsi May 8, 2021
a062faf
Update docs/lazy-adr/adr-004-mvp-light-client.md
liamsi May 8, 2021
257dbbc
Update docs/lazy-adr/adr-004-mvp-light-client.md
liamsi May 8, 2021
1b119a7
Update docs/lazy-adr/adr-004-mvp-light-client.md
liamsi May 8, 2021
0369f0f
Minor changes (title & cmd help text) and moving IPFS section
liamsi May 8, 2021
eb64c8e
Remove LAZY prefix from all ADRs
liamsi May 8, 2021
eca6147
add missing word
liamsi May 8, 2021
64044aa
fix typo from review feedback
liamsi May 8, 2021
2dddade
Fix diff
liamsi May 8, 2021
a275f1e
fix other diff
liamsi May 8, 2021
f3006e2
link to DAHeader earlier
liamsi May 8, 2021
747c397
Add alternative for LightBlock and fix typo
liamsi May 8, 2021
17c9e5a
Changes to `Client` and usage of `ValidateAvailability`
liamsi May 8, 2021
7c91f68
cleanup some todos
liamsi May 8, 2021
48b0430
Add note about DHT Client/Server modes
liamsi May 8, 2021
9a08608
add numSamples to UX
liamsi May 9, 2021
7fb463c
Update docs/lazy-adr/adr-004-mvp-light-client.md
liamsi May 10, 2021
54b9b5d
Resolve a bunch of todos and fix sentence
liamsi May 10, 2021
630e967
Update docs/lazy-adr/adr-004-mvp-light-client.md
liamsi May 10, 2021
e5cf047
Update docs/lazy-adr/adr-004-mvp-light-client.md
liamsi May 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add Store requirements & note on skipping verification UX
@liamsi
liamsi committed May 5, 2021
commit 58b1506bfaeae1bc4887b3661564c76d63d09128
15 changes: 14 additions & 1 deletion docs/lazy-adr/adr-004-mvp-light-client.md
View file
Original file line number Diff line number Diff line change
@@ -82,6 +82,9 @@ It makes sense to make this mostly opaque to the user as everything around IPFS
This IPFS path should simply be a sub-directory inside the light client's [directory](https://github.com/lazyledger/lazyledger-core/blob/cbf1f1a4a0472373289a9834b0d33e0918237b7f/cmd/tendermint/commands/light.go#L86-L87).
We can later add the ability to let users configure the IPFS setup more granular.

**Note:** DAS should only be compatible to sequential verification.
In case a light client is parametrized to run DAS and skipping verification, the CLI should return an easy-to-understand warning or even an error explaining why this does not make sense.

### Light Client Protocol with DAS

#### Running an IPFS node
@@ -121,6 +124,17 @@ Currently, only the [`http` provider](https://github.com/tendermint/tendermint/b
Hence, as _a first implementation step_, we should augment the `Provider` and the `LightBlock` to optionally include the DAHeader (details below).
In parallel but in a separate pull request, we add a separate RPC endpoint to download the DAHeader for a certain height.

#### Store DataAvailabilityHeader

For full nodes to be able to serve the `DataAvailabilityHeader` without having to recompute it each time, it needs to be stored somewhere.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we need to store DataAvailabilityHeader not recompute it, but to actually access block's data. You can't access block's data without DAHeader, thus you can't recompute it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Things would have been so much easier if we just accessed block's data by Header.DataHash in the first place. We then would not need to store DAHeader in both Full and Light node stores, but only Header, as planned.

Copy link
Member Author

@liamsi liamsi May 10, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we need to store DataAvailabilityHeader not recompute it [...]

That is what the text in the ADR says 🤔 With the existing code we store the Block Data but not the DAHeader and hence fullnodes can recompute it on demand. That is far from optimal but it works for the mvp.

Things would have been so much easier if we just accessed block's data by Header.DataHash in the first place.

Well, we can still do that. It's a bit orthogonal to this ADR I guess?

We then would not need to store DAHeader in both Full and Light node stores, but only Header, as planned.

Wouldn't it still make sense to store the DAHeader. Especially for nodes that do not have the full block data?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, we can still do that. It's a bit orthogonal to this ADR I guess?

Yep, that just a quick look retrospectively at how things could be done better. Nothing against this PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't it still make sense to store the DAHeader. Especially for nodes that do not have the full block data?

If you have DataHash you can access full DAHeader, thus you can do anything you do with DAHeader, thus storing/treating it specifically is not needed, as it would be stored somewhere on the network.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as it would be stored somewhere on the network.

That still means nodes will store the daheader in their local data stores. I guess the question here is: should we additionally store the daheader separately or not? This rather plays into adr 001 and #182. I would argue that most nodes will need the get the DAHeader and it could be useful to provide a shortcut via storing it as one blob instead of in ipld only (even if that is redundant). But maybe that is not necessary.

We will see after we (you?) have written the adr for "datahash -> DaHeader" and after we have implemented this feature. Then we can measure latencies and decide.

While this is independent of the concrete serving mechanism, it is more so relevant for the RPC endpoint.
There is ongoing work to make the Tendermint Store only store Headers and the DataAvailabilityHeader in [#218](https://github.com/lazyledger/lazyledger-core/pull/218/) / [#182](https://github.com/lazyledger/lazyledger-core/issues/182).

At the time writing this ADR, another pull request ([#312](https://github.com/lazyledger/lazyledger-core/pull/312)) is in the works with a more isolated change that adds the `DataAvailabilityHeader` to the `BlockID`.
Hence, the DAHeader is [stored](https://github.com/lazyledger/lazyledger-core/blob/50f722a510dd2ba8e3d31931c9d83132d6318d4b/store/store.go#L355-L367) along the [`BlockMeta`](https://github.com/lazyledger/lazyledger-core/blob/50f722a510dd2ba8e3d31931c9d83132d6318d4b/types/block_meta.go#L11-L17) there.
For a first implementation, we could first build on top of #312 and adapt to the


#### DAS

The changes for DAS are very simple from a high-level perspective assuming that the light client has the ability to download the DAHeader along with the required data (signed header + validator set) of a given height:
@@ -174,7 +188,6 @@ diff --git a/types/light.go b/types/light.go
}
```

- TODO: rpc endpoint
- TODO: Provider