Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Pallets] Evaluate completeness of events #275

Closed
4 tasks done
Tracked by #282
sea212 opened this issue Aug 25, 2021 · 0 comments · Fixed by #334
Closed
4 tasks done
Tracked by #282

[Pallets] Evaluate completeness of events #275

sea212 opened this issue Aug 25, 2021 · 0 comments · Fixed by #334
Assignees
Labels
p:high High priority, prioritize the resolution of this issue t:maintenance The issue describes necessary maintenance

Comments

@sea212
Copy link
Member

sea212 commented Aug 25, 2021

To be able to effectively use Subsquid, we have to output the information that should be processed (and is not already part of the call itself) within events.
It is possible to query the state of blocks in the past by sending a RPC to an archival node. However, this is not the desired approach, since it introduces additional problems, such as timeout handling, knowledge of low-level node information outside the node (how did the storage structure look at every point in time), etc.

This must be done before the upcoming launch of the next testnet

Update meeting 1st Sep. 2021
Most of the information can be gathered via events. Whenever a new structure is stored, such as a Market or a Pool, an event is emitted that contains the id, the structure itself and the account id. All following events which introduce changes to those data structures must contain the id of the specific instance of the data structure and, if available, the account id that instructed the changes.
By using this approach, Subsquid is able to collect the complete data set whenever a new instance of a data structure is created (such as a Market) and trace all subsequent changes to it (since the changes can be joined using the corresponding ids). In addition, any additional information such as the timestamp, block number and account id, can be used to execute complex queries over many different datasets to gain additional insights.
A corner-case are the spot prices: They are calculated on demand for a pair, therefore we cannot trace every spot price through events. In this case, the SDK will use a RPC to calculate the prices. This allows us to gather data about the price history in addition to the quantity (and therefore implicitly the volume) history. This data in turn can be used in our research to evaluate our market scoring rule and in response the behavior of the users. Querying Subsquid prices in contrast to using a RPC to fetch the price history is by magnitudes faster.

TODOs

@sea212 sea212 added p:critical Critical priority, drop everything to resolve this issue t:maintenance The issue describes necessary maintenance labels Aug 25, 2021
@sea212 sea212 changed the title [Pallets] Evaluate completness of events [Pallets] Evaluate completeness of events Aug 25, 2021
@sea212 sea212 added p:high High priority, prioritize the resolution of this issue and removed p:critical Critical priority, drop everything to resolve this issue labels Aug 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p:high High priority, prioritize the resolution of this issue t:maintenance The issue describes necessary maintenance
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants