Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix miner sync and improve oracle performance #1025

Merged
merged 5 commits into from
Mar 9, 2022
Merged

Conversation

area
Copy link
Member

@area area commented Feb 18, 2022

Several improvements here:

Sync issues

We tried syncing from scratch on QA and ran in to difficulty; I wasn't confident I hadn't just done a load of messing around on QA at some point, so assumed that the sync was good.

When trying to sync up production from scratch, where I was confident that we didn't have such issues, we ran in to problems again. I've tracked them down to a negative reputation update, and the origin skill key being incorrectly worked out against the latest reputation update log, rather than the historical reputation update log that we're trying to sync against.

Leaving this as a draft for now, as I've not successfully sync'd production yet, so we might need further changes, but it's looking promising:
screenshot-2022-02-18_14-47-39

Production sync'd from scratch successfully, so will be taking this out of draft

Strictly, the call the getSkill to work out nParents doesn't need a blockTag, as nParents can never change, but just in case we do something that renders that not true in the future, I decided to add the blockNumber tag there.

This PR also incorporates improvements made to the reputation oracle so that it's more performant.

Improve sync experience

By recursively calling sync if necessary, we allow sync to complete successfully even if more cycles complete during the syncing process (which will certainly be the case while syncing from scratch).

Improve queries for last confirmed state

Keep most recently confirmed state in memory for oracle. Previously, only the next state was kept in memory, and any other reputation queries (i.e. all of them) had to hit the DB. By keeping the most recently confirmed state in memory, the majority of queries are easier to serve (as the majority of reputation queries are for reputation in the most recently confirmed state) and we hit the database much, much less.

Improve saving the current state to DB

I also changed the schema of the reputations table, adding a composite primary key across roothash/colony/skillid/user. This combination was already unique (as the schema upgrade on the existing table proved), and so arguably should have been a key already (though is not free, it comes with a significant storage overhead). By making this composite primary key, the 'insert reputation' query can now be an 'INSERT OR IGNORE' - this should be fine, as such a value should never need to be updated (even if a sync goes wrong, which will result in wrong root hashes). This means we don't have to read the database to see if we need to add the reputation or not, and can simply fire off all of the write requests and let the database deal with any unexpected duplicates. This resulted in the time for saveCurrentState on production dropping from 15 minutes to a few seconds.

Improve test coverage for client

With changes to the client and more branches, our coverage dropped below our (very arbitrary) mark, so had to bulk out the tests to compensate (without feeling bad about lowering the mark!)

@area area added the bug label Feb 18, 2022
Copy link
Member

@kronosapiens kronosapiens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😻

@area area changed the title Fix miner sync Fix miner sync and improve oracle performance Mar 3, 2022
@area area marked this pull request as ready for review March 3, 2022 11:45
@kronosapiens kronosapiens merged commit db96dfd into develop Mar 9, 2022
@kronosapiens kronosapiens deleted the fix/miner-sync branch March 9, 2022 00:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants