Rust Portal-Hive to Hive Transition Tracker Issue

[KolbyML]

Back in October Felix proposed the idea of us merging portal-hive back into hive. Here is a tracker PR which will track the progress of it. Originally I made 1 big PR, but looking back on that it probably wasn't the best idea and could be seen as overwhelming to review. So I am back with a different attept in making a more gradual transition.

PR's to complete transition of the Rust Portal Hive code

• Add portal client definitions clients: add portal network client definitions #977
• Add ci for portal simulators .circleci: Add CI for rust simulators #980
• Add portal history/rpc-compat simulator simulators/portal/history/rpc-compat: add portal network history rpc-compat simulator #978
• Add portal history/portal-interop simulator simulators/portal/history/portal-interop: add portal network history portal-interop simulator #987
• Add portal history/portal-mesh simulator simulators/portal/history/portal-mesh: add portal network history portal-mesh simulator #988
• Add portal history/portal-bridge simulator simulators/portal/history/trin-bridge: add portal network history trin-bridge simulator #989
• Add portal beacon/rpc-compat simulator simulators/portal/beacon/rpc-compat: add portal network beacon rpc-compat simulator #990
• Add portal (simulator which is made during this transition period in the portal-hive repo) simulator (PR to be made)
• Add hivesim-rs hivesim-rs: add hivesim-rs a rust implementation of hivesim #1126

Then one day after the above is done and hivesim-rs is "stable" we can transition that too.

I can't wait till this happens as I want to write some EL -> portal-bridge -> portal client tests.

[fjl]

Hey @KolbyML, it's actually np to have the big PR, I'm just very slow.

[KolbyML]

With the merge of #980 it depends on turning on Use of setup workflows must be enabled in project settings (Project settings > Advanced -> Dynamic config using setup workflows) in circleci settings. That PR was based off this circleci documentation https://github.com/CircleCI-Public/api-preview-docs/blob/path-filtering/docs/path-filtering.md#setup-workflows

No worries. :D I am just excited with the smaller PR way I actually have something merged now which is so cool!

[fjl]

I have changed the setting in CircleCI

[KolbyML]

I made an error in the formatting of the continue_circleci file in #980
here is a PR to fix that #982 now CI should work again.

[KolbyML]

The Rust CI works and only runs on the filtered path very cool. I am going to open the PR's for the other simulators tomorrow. I think it is nice with this split approach because it is also a lot less work I realized on my end to maintain the PR, with any changes we make on portal-hive in the mean time during the transitionary period.

[fjl]

I have merged the simulators as-is. However, I think there are some things that could be improved:

• The simulator names are very long. It might be better to remove one directory level and call them simulators/portal/history-{rpc,interop,mesh,bridge} etc.
• I've added a 'portal' role to the clients. Don't know if hivesim-rs supports it, but hive clients have roles
  which you will get through the client-types response: https://github.com/ethereum/hive/blob/master/docs/simulators.md#getting-available-client-types
  The idea with roles is that, if you run a simulator with all available clients, it will pick the ones that it can actually use.

[KolbyML]

Here is a PR with the readme we had on portal-hive #991

I would be down to remove the portal-prefix from interop/mesh/bridge and compat suffix from rpc

I would prefer simulators/portal/history/{rpc,interop,mesh,bridge} Because as we add more networks there will be simulators which use multiple networks. E.x. simulators which test the interopibility of the beacon and history network. beacon-history/rpc or beacon-history-rpc. I like beacon-history/rpc because then as we add more simulators which could possibly use multiple networks they are contained in a folder specifying networks used, instead of a long list of misc simulaters in the explorer.

If you want (networks used)-(simulator name) I am fine with that, it might just get messy as we add simulators for testing the interopibility of multiple portal networks working together.

I've added a 'portal' role to the clients. Don't know if hivesim-rs supports it, but hive clients have roles
which you will get through the client-types response:

I will look into this and resolve any issues if they exist

[KolbyML]

Another note is trin-bridge can only be used with the portal-bridge simulator and is required for portal-bridge simulator to be ran

[fjl]

Looking at the simulators again, I also noticed the actual suite implementations are kind of small. One thing to note is that there is no one-to-one correspondence between simulators and test suites. You can run multiple suites from one simulator, so we could also just add a portal/history simulator that contains all of the history network test suites. This would also cut down the amount of Dockerfiles and Rust projects.

[KolbyML]

:D I didn't even know that was possible. I think that is a great idea! I haven't read the simulators written in go-lang as they were never on ethereum/portal-hive at the time I started to write tests for it we just had 2 different simulators rpc-compat and portal-interop. I will read one of the go simulators to get a better idea on the best practices for structuring simulators and make a PR with the change you suggested 👍

[KolbyML]

Here https://github.com/ethereum/hive/pull/994/files here is a PR with the requested layout :D