Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable ovm1 data UI #1509

Closed
wants to merge 17 commits into from
Closed

Conversation

chuxinh
Copy link
Contributor

@chuxinh chuxinh commented Sep 5, 2022

Add ovm1 to explorer UI

For Dune Engine V2
I've checked that:
General checks:

  • I tested the query on dune.com after compiling the model with dbt compile (compiled queries are written to the target directory)
  • I used "refs" to reference other models in this repo and "sources" to reference raw or decoded tables
  • if adding a new model, I added a test
  • the filename is unique and ends with .sql
  • each sql file is a select statement and has only one view, table or function defined
  • column names are lowercase_snake_cased
  • if adding a new model, I edited the dbt project YAML file with new directory path for both models and seeds (if applicable)
  • if adding a new model, I edited the alter table macro to display new database object (table or view) in UI explorer
  • if adding a new materialized table, I edited the optimize table macro

Join logic:

  • if joining to base table (i.e. ethereum transactions or traces), I looked to make it an inner join if possible

Incremental logic:

  • I used is_incremental & not is_incremental jinja block filters on both base tables and decoded tables
    • where block_time >= date_trunc("day", now() - interval '1 week')
  • if joining to base table (i.e. ethereum transactions or traces), I applied join condition where block_time >= date_trunc("day", now() - interval '1 week')
  • if joining to prices view, I applied join condition where minute >= date_trunc("day", now() - interval '1 week')

@dune-eng
Copy link

dune-eng commented Sep 5, 2022

Workflow run id 2996658084 approved.

@chuxinh
Copy link
Contributor Author

chuxinh commented Sep 6, 2022

Also would like to know if airdrop_optimism.address can also be updated? I think it's referencing the csv file name or instead i can update what the alter table marco is calling?
We'd like to have it changed to op_airdrop_1 cc: @MSilb7

@soispoke soispoke self-requested a review September 6, 2022 05:54
Copy link
Contributor

@soispoke soispoke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @chuxinh, thanks for your submission. It seems like you're are trying to expose tables that don't exist on Dune v2 ?

(if if does exist and I missed it, could you send me the links of where the table are in the spellbook github repo please ? https://github.com/duneanalytics/spellbook)

@chuxinh
Copy link
Contributor Author

chuxinh commented Sep 6, 2022

Hi @chuxinh, thanks for your submission. It seems like you're are trying to expose tables that don't exist on Dune v2 ?

(if if does exist and I missed it, could you send me the links of where the table are in the spellbook github repo please ? https://github.com/duneanalytics/spellbook)

Thanks @soispoke!
So those ovm1 data have been loaded as csv as i'm aware of and I don't have any link to them. So it's very similar to what the team has done in this airdrop address list that you did: #1420

@soispoke
Copy link
Contributor

soispoke commented Sep 6, 2022

Oh alright, let me get back to you quickly on these ones then, I think we need to find a better way to have these tables deployed/exposed somehow.

@soispoke
Copy link
Contributor

soispoke commented Sep 6, 2022

@chuxinh I think one good way to do this would be to turn these .csv files into static tables that can be seen, modified and exposed easily on spellbook.

A good example of this format is tokens here: https://github.com/duneanalytics/spellbook/blob/main/models/tokens/ethereum/tokens_ethereum_erc20.sql

What do you think ?

@chuxinh
Copy link
Contributor Author

chuxinh commented Sep 6, 2022

@chuxinh I think one good way to do this would be to turn these .csv files into static tables that can be seen, modified and exposed easily on spellbook.

A good example of this format is tokens here: https://github.com/duneanalytics/spellbook/blob/main/models/tokens/ethereum/tokens_ethereum_erc20.sql

What do you think ?

Thanks for getting back on this! Yea originally Jeff and Antonio helped on this and uploaded as CSVs.
Is it possible that we can get some help on your side to reorganise the ovm1 existing schema and those tables?

@soispoke
Copy link
Contributor

soispoke commented Sep 6, 2022

Yes ofc, what would be super useful is to transform those csv files into static SQL tables as I mentioned (in this PR for example), and then I can polish everything up, make sure schemas are right and tables are exposed properly. But since I don't have access to the original files rn, I still need you to reformat/resubmit those files as tables if that's ok?

@chuxinh
Copy link
Contributor Author

chuxinh commented Sep 6, 2022

Yes ofc, what would be super useful is to transform those csv files into static SQL tables as I mentioned (in this PR for example), and then I can polish everything up, make sure schemas are right and tables are exposed properly. But since I don't have access to the original files rn, I still need you to reformat/resubmit those files as tables if that's ok?

Thank you!!
So all the original files are resting here: https://github.com/duneanalytics/spellbook/tree/main/deprecated-dune-v1-abstractions/optimism2/ovm1
So i guess it's better to have them loaded as .sql files as the example you showed. However, i think i also remember Meghan has mentioned about the limitations on github storage (or something similar that i'm not using the right word for). Is that still a concern or I can proceed

@soispoke
Copy link
Contributor

soispoke commented Sep 6, 2022

Oh thanks for pointing out where the original files are. Ok yeah I remember something like this. Meghan is OOO but how about we try to go through the process for 1 file so you're not spending too much time on this (maybe with the biggest file ?), and if everything goes well we just do the same for all the other ones.

@chuxinh
Copy link
Contributor Author

chuxinh commented Sep 6, 2022

Right, I will leave this one open for now and create a separate one and see how it goes
Will do that my time tomorrow :D

@soispoke
Copy link
Contributor

soispoke commented Sep 6, 2022

sounds great thanks !

@soispoke soispoke mentioned this pull request Sep 6, 2022
14 tasks
@chuxinh chuxinh closed this Sep 8, 2022
@chuxinh chuxinh deleted the enable-ovm1-data-ui branch September 21, 2022 05:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants