Etherscan metadata variations #976

kuzdogan · 2023-04-05T09:24:51Z

_{View in Huly HI-713}

experiment to create files varations before running verification searching for perfect matches

…bytecode WIP * add tryToFindOriginalMetadata to CheckedContract: tries to recontruct the original metadata by iterating on all files variations * call tryToFindOriginalMetadata after `matchWithDeployedBytecode` if match is not perfect

codecov-commenter · 2023-04-05T12:47:35Z

Codecov Report

Patch coverage: 72.72% and project coverage change: -4.24 ⚠️

Comparison is base (1ef0e73) 77.56% compared to head (ad406a8) 73.33%.

Additional details and impacted files

@@             Coverage Diff             @@
##           staging     #976      +/-   ##
===========================================
- Coverage    77.56%   73.33%   -4.24%     
===========================================
  Files           29       38       +9     
  Lines         1373     2171     +798     
  Branches       254      416     +162     
===========================================
+ Hits          1065     1592     +527     
- Misses         180      356     +176     
- Partials       128      223      +95

Flag	Coverage Δ
lib-sourcify	`66.11% <ø> (?)`
server	`77.46% <72.72%> (-0.11%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
.../server/controllers/VerificationController-util.ts	`83.82% <57.14%> (-0.96%)`	⬇️
src/server/controllers/VerificationController.ts	`75.09% <100.00%> (+0.19%)`	⬆️

... and 9 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

marcocastignoli · 2023-04-05T13:26:23Z

@kuzdogan PR is not ready but if you want to check what I did till now and give me some feedback, it's appreciated.

packages/lib-sourcify/src/lib/CheckedContract.ts

test/etherscan.js

packages/lib-sourcify/package.json

packages/lib-sourcify/src/lib/CheckedContract.ts

…hash by updating the source section of the metadata and generating the metadata CID

* I translated the ipfsHash.cpp file from the solidity repo * now that everything is synchronous I simplified the code that replace the ipfs and bzz hash in the urls array in metadata.sources * I added some comments to clearify the storebyhash grouped by variation

marcocastignoli · 2023-04-11T10:26:39Z

TODO

Add tests
handle bzz encoded metadata

marcocastignoli · 2023-04-11T11:51:39Z

Tests are failing because https://ipfs.io is down

EDIT: it's not down, but it doesn't support anymore url with double slashes

https://ipfs.io//ipfs/QmT82ab2aGqBc5owXEqmiE1NYfYEdWCKpc4V11ncummXit

instead of

https://ipfs.io/ipfs/QmT82ab2aGqBc5owXEqmiE1NYfYEdWCKpc4V11ncummXit

marcocastignoli · 2023-04-11T14:14:00Z

@kuzdogan now I think we are ready for merge, wanna review again?

packages/lib-sourcify/src/lib/CheckedContract.ts

packages/lib-sourcify/src/lib/hashFunctions/ipfsHash.ts

packages/lib-sourcify/src/lib/hashFunctions/swarmHash.ts

packages/lib-sourcify/src/lib/verification.ts

packages/lib-sourcify/test/functions.spec.ts

kuzdogan · 2023-04-12T10:32:17Z

Added some comments. Specifically how we return the match at verification.ts needs to change.

Don't we have verification test cases? Like the one that is our starting point from Etherscan?

marcocastignoli · 2023-04-12T10:46:01Z

Don't we have verification test cases? Like the one that is our starting point from Etherscan?

what do you mean?

kuzdogan · 2023-04-12T11:15:13Z

Don't we have verification test cases? Like the one that is our starting point from Etherscan?

what do you mean?

I see the function is just unit tested but the reason we develop this at the first place is to be able to verify contracts perfectly when they have common and known changes in the metadata that we can try to fix. I was referring to the contracts in this issue #936 We should test the verification on those, as well as find and add other cases that we know.

marcocastignoli · 2023-04-12T11:56:18Z

The contract in the test is actually one of those, the one from this comment: #936 (comment)

kuzdogan · 2023-04-12T12:15:46Z

The contract in the test is actually one of those, the one from this comment: #936 (comment)

Good, let's use it in the verification tests then. With that we will have tested the whole workflow and not just that part

* test in verification.spec.ts

packages/lib-sourcify/src/lib/types.ts

packages/lib-sourcify/test/verification.spec.ts

Co-authored-by: Kaan Uzdoğan <[email protected]>

packages/lib-sourcify/src/lib/verification.ts

* refactor `verifyDeployed` so that it uses the new function ´tryToFindOriginalMetadataAndMatch´

marcocastignoli added 9 commits April 3, 2023 08:46

#936 file variation while importing from etherscan

5f79dc2

experiment to create files varations before running verification searching for perfect matches

restore etherscan standard verification

c7d41e2

Merge branch 'staging' into etherscan-metadata-variations

0abb80c

#936 tests working

f8ef3fd

update package-json lib-sourcify

6ed3281

#936 fix linting issues

8f12e2a

wait 1 second between every etherscan request

737eed7

#936 update non-session verify from etherscan controler

c316ac5

#936 better function naming getMetadataFromCompiler

3ffa68c

marcocastignoli reviewed Apr 5, 2023

View reviewed changes

packages/lib-sourcify/src/lib/CheckedContract.ts Show resolved Hide resolved

marcocastignoli reviewed Apr 5, 2023

View reviewed changes

test/etherscan.js Show resolved Hide resolved

marcocastignoli marked this pull request as ready for review April 5, 2023 14:25

kuzdogan commented Apr 5, 2023

View reviewed changes

packages/lib-sourcify/package.json Outdated Show resolved Hide resolved

packages/lib-sourcify/src/lib/CheckedContract.ts Outdated Show resolved Hide resolved

packages/lib-sourcify/src/lib/CheckedContract.ts Outdated Show resolved Hide resolved

marcocastignoli added 4 commits April 6, 2023 10:04

#936 instead of recompiling each variation, recalculate the metadata …

e40e826

…hash by updating the source section of the metadata and generating the metadata CID

#936 fix error on non existance of the path in sources

8a0c863

#936 add coments to group by variations to make it more clear

39997c0

marcocastignoli added 3 commits April 11, 2023 14:27

fix default ipfs gateway

cdbd80b

#936 add test for tryToFindOriginalMetadata

acc19ac

#936 add swarmBzzr0Hash and test metadata with both ipfs and swarm

82cd7cc

Add comments

5a63a90

kuzdogan commented Apr 12, 2023

View reviewed changes

#936 refactor tryToFindOriginalMetadata adding types

47e9a0c

* test in verification.spec.ts

kuzdogan commented Apr 13, 2023

View reviewed changes

packages/lib-sourcify/src/lib/types.ts Show resolved Hide resolved

#976 add license type in MetadataSources

03a23fb

marcocastignoli mentioned this pull request Apr 13, 2023

Investigate what might be causing partial matches from single-file Etherscan contracts #936

Open

kuzdogan commented Apr 13, 2023

View reviewed changes

packages/lib-sourcify/test/verification.spec.ts Outdated Show resolved Hide resolved

#976 change test name for wrong end of line

856e29a

kuzdogan commented Apr 13, 2023

View reviewed changes

packages/lib-sourcify/test/verification.spec.ts Outdated Show resolved Hide resolved

Update packages/lib-sourcify/test/verification.spec.ts

b0523dd

Co-authored-by: Kaan Uzdoğan <[email protected]>

kuzdogan commented Apr 13, 2023

View reviewed changes

packages/lib-sourcify/src/lib/verification.ts Outdated Show resolved Hide resolved

marcocastignoli added 2 commits April 17, 2023 16:08

#976 try to find original metadata for each verification method

ad406a8

* refactor `verifyDeployed` so that it uses the new function ´tryToFindOriginalMetadataAndMatch´

Merge branch 'staging' into etherscan-metadata-variations

a1cf2fa

kuzdogan merged commit 486732c into staging Apr 20, 2023

kuzdogan deleted the etherscan-metadata-variations branch May 10, 2023 14:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Etherscan metadata variations #976

Etherscan metadata variations #976

kuzdogan commented Apr 5, 2023 •

edited

Loading

codecov-commenter commented Apr 5, 2023 •

edited

Loading

marcocastignoli commented Apr 5, 2023

marcocastignoli commented Apr 11, 2023 •

edited

Loading

marcocastignoli commented Apr 11, 2023 •

edited

Loading

marcocastignoli commented Apr 11, 2023

kuzdogan commented Apr 12, 2023 •

edited

Loading

marcocastignoli commented Apr 12, 2023 •

edited

Loading

kuzdogan commented Apr 12, 2023

marcocastignoli commented Apr 12, 2023

kuzdogan commented Apr 12, 2023

Etherscan metadata variations #976

Etherscan metadata variations #976

Conversation

kuzdogan commented Apr 5, 2023 • edited Loading

codecov-commenter commented Apr 5, 2023 • edited Loading

Codecov Report

marcocastignoli commented Apr 5, 2023

marcocastignoli commented Apr 11, 2023 • edited Loading

marcocastignoli commented Apr 11, 2023 • edited Loading

marcocastignoli commented Apr 11, 2023

kuzdogan commented Apr 12, 2023 • edited Loading

marcocastignoli commented Apr 12, 2023 • edited Loading

kuzdogan commented Apr 12, 2023

marcocastignoli commented Apr 12, 2023

kuzdogan commented Apr 12, 2023

kuzdogan commented Apr 5, 2023 •

edited

Loading

codecov-commenter commented Apr 5, 2023 •

edited

Loading

marcocastignoli commented Apr 11, 2023 •

edited

Loading

marcocastignoli commented Apr 11, 2023 •

edited

Loading

kuzdogan commented Apr 12, 2023 •

edited

Loading

marcocastignoli commented Apr 12, 2023 •

edited

Loading