Added EIP-4844 Headers for BlockHeaderOutput. Issue: 6933 #6937

ymonye · 2024-03-26T11:01:28Z

Added EIP-4844 & EIP-4788 Headers for the BlockHeaderOutput, following Go-Ethereum headers:
https://github.com/ethereum/go-ethereum/blob/738b5a586e329965539877434b695bb61015d4c7/core/types/block.go#L65

getBlock() now uses blockHeaderSchema instead of blockSchema.

Original Issue: #6933

Description

Please include a summary of the changes and be sure to follow our Contribution Guidelines.

Type of change

Bug fix (non-breaking change which fixes an issue)
[*] New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[*] I have selected the correct base branch.
[*] I have performed a self-review of my own code.
I have commented my code, particularly in hard-to-understand areas.
I have made corresponding changes to the documentation.
[*] My changes generate no new warnings.
Any dependent changes have been merged and published in downstream modules.
[*] I ran npm run lint with success and extended the tests and types if necessary.
[*] I ran npm run test:unit with success.
I ran npm run test:coverage and my test cases cover all the lines and branches of the added code.
I ran npm run build and tested dist/web3.min.js in a browser.
[*] I have tested my code on the live network.
I have checked the Deploy Preview and it looks correct.
I have updated the CHANGELOG.md file in the root folder.
I have linked Issue(s) with this PR in "Linked Issues" menu.

…es blockHeaderSchema instead of blockSchema.

luu-alex · 2024-03-26T16:21:18Z

Thanks for this, could u remove the yarn.lock and update the changelog for web3-eth and web3-types
Could u give PR edit access to contributors of this project as well, just in case i need to update this pr

luu-alex · 2024-03-26T18:53:16Z

packages/web3-eth/src/rpc_method_wrappers.ts

@@ -257,7 +258,7 @@ export async function getBlock<ReturnFormat extends DataFormat>(
 			hydrated,
 		);
 	}
-	return format(blockSchema, response as unknown as Block, returnFormat);


I believe you would want to edit blockSchema and add the new properties there, not editing blockHeaderSchema and just using blockSchema

I believe you would want to edit blockSchema and add the new properties there, not editing blockHeaderSchema and just using blockSchema

Ok, yeah I'd noticed both the withdrawals & withdrawalsRoot properties only appear under blockHeaderSchema, in the schemas.ts file. When I call getBlock() on web3.js v1.9, it outputs both of those properties. This is why I'd assumed getBlock() probably needed blockHeaderSchema defined within v4's rpc_method_wrappers.ts file instead of blockSchema, and thought it was a typo.

Should blockSchema & blockHeaderSchema have identical properties then?

Yea its a bit confusing, It seems like blockSchema needs to also have updated properties of withdrawals and withdrawalsRoot. blockHeaderSchema is only being used for subscriptions. We may need to update the blockHeaderSchema as well, if you want to check and confirm result of a subscription the result you can also edit blockheaderSchema, otherwise you can focus on just updating blockSchema

Ok, I've updated both blockSchema & blockHeaderSchema in schemas.ts, with blockHeaderSchema retaining the below author & excessDataGas properties while missing from blockSchema. Please let me know if blockSchema should also have these.

blockSchema & blockHeaderSchema also have different nested arrays for the transactions property, and were left as originally found.

author: { format: 'bytes32', }, excessDataGas: { format: 'uint', },

The BlockOutput & BlockHeaderOutput interfaces in eth_types.ts were also updated accordingly.

Under eth_types.ts, I'd noticed a number of properties are formatted as either bigint | number or Numbers, whether they are defined under BlockOutput or BlockHeaderOutput.

For example: totalDifficulty in BlockOutput:
readonly totalDifficulty?: bigint | number;

Whereas in BlockHeaderOutput, totalDifficulty is formatted as:
readonly totalDifficulty?: Numbers;

I'm unsure if it should be one or the other, but I've left as-is. Also the extra properties: author & excessDataGas were not added to the BlockOutput interface, as they are not present in blockSchema. However I did add everything else from BlockHeaderOutput interface into BlockOutput, while formatting the ones with Numbers as bigint | number. Please advise if that was the correct move.

Also, let me know if those author & excessDataGase properites should also be present across blockSchema/blockHeaderSchema & BlockOutput/BlockHeaderOutput.

yarn.lock had its changes reverted, as well as rpc_method_wrappers.ts.

Also, it appears the 'newHeads' & 'newBlockHeaders' subscriptions only return formatted data from the BlockBase interface in eth_types.ts, and not from the BlockHeaderOutput interface.

https://docs.web3js.org/api/web3-eth/class/NewHeadsSubscription

Adding a note here so I don't forget after sleep. Looks like more parameters are missing from v4.6 yet present in v1.9, outside the scope of EIP-4844 changes, specifically across other eth_* functions. I'll do a much proper rundown thru the next couple of days, likely into the weekend.

For ex: the id parameter is missing in the logs array, from eth_getTransactionReceipt. Example below from an ETH mainnet request:

web3 v4.6:
{"address":"0xb584d4be1a5470ca1a8778e9b86c81e165204599","blockHash":"0x47c58ebbaaff9a9a9ee60d74029224df7cac63b5c09972d4306e7500d8078b55","blockNumber":"19529894","data":"0x0000000000000000000000000000000000000000000000000000022ecb25c000","logIndex":"469","removed":false,"topics":["0x90890809c654f11d6e72a28fa60149770a0d11ec6c92319d6ceb2bb0a4ea1a15","0x0000000000000000000000000a25e45792fef9fb600a81f0d7ac4ab9e893fc71","0x0000000000000000000000000000000000000000000000000000000000000064"],"transactionHash":"0xe2bad713807549fe3bb0bf0a69dce0ddc634c8dbec655660ea038279ef5bdb91","transactionIndex":"170"}

web3 v1.9:
{"address":"0xb584D4bE1A5470CA1a8778E9B86c81e165204599","blockHash":"0x47c58ebbaaff9a9a9ee60d74029224df7cac63b5c09972d4306e7500d8078b55","blockNumber":19529894,"data":"0x0000000000000000000000000000000000000000000000000000022ecb25c000","id":"log_ff3f9ddd","logIndex":469,"removed":false,"topics":["0x90890809c654f11d6e72a28fa60149770a0d11ec6c92319d6ceb2bb0a4ea1a15","0x0000000000000000000000000a25e45792fef9fb600a81f0d7ac4ab9e893fc71","0x0000000000000000000000000000000000000000000000000000000000000064"],"transactionHash":"0xe2bad713807549fe3bb0bf0a69dce0ddc634c8dbec655660ea038279ef5bdb91","transactionIndex":170}

ymonye · 2024-03-27T10:44:00Z

Thanks for this, could u remove the yarn.lock and update the changelog for web3-eth and web3-types Could u give PR edit access to contributors of this project as well, just in case i need to update this pr

I think I did this, gave you access to my forked repo

jdevcs

remove changes in yarn.lock,

ymonye · 2024-03-27T19:40:18Z

remove changes in yarn.lock,

Done

codecov · 2024-03-28T10:32:06Z

Codecov Report

Merging #6937 (8fdfa11) into 4.x (c4e039a) will increase coverage by 0.00%.
Report is 2 commits behind head on 4.x.
The diff coverage is 100.00%.

Additional details and impacted files

@@           Coverage Diff           @@
##              4.x    #6937   +/-   ##
=======================================
  Coverage   91.95%   91.96%           
=======================================
  Files         215      215           
  Lines        8188     8197    +9     
  Branches     2208     2214    +6     
=======================================
+ Hits         7529     7538    +9     
  Misses        659      659

Flag	Coverage Δ
UnitTests	`91.96% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
web3	`∅ <ø> (∅)`
web3-core	`∅ <ø> (∅)`
web3-errors	`∅ <ø> (∅)`
web3-eth	`∅ <ø> (∅)`
web3-eth-abi	`∅ <ø> (∅)`
web3-eth-accounts	`∅ <ø> (∅)`
web3-eth-contract	`∅ <ø> (∅)`
web3-eth-ens	`∅ <ø> (∅)`
web3-eth-iban	`∅ <ø> (∅)`
web3-eth-personal	`∅ <ø> (∅)`
web3-net	`∅ <ø> (∅)`
web3-providers-http	`∅ <ø> (∅)`
web3-providers-ipc	`∅ <ø> (∅)`
web3-providers-ws	`∅ <ø> (∅)`
web3-rpc-methods	`∅ <ø> (∅)`
web3-utils	`∅ <ø> (∅)`
web3-validator	`∅ <ø> (∅)`

luu-alex · 2024-04-15T13:33:15Z

sorry for the long awaited response, looks good :) this will be included in the next release. Thanks for your efforts!

jdevcs · 2024-04-15T14:52:33Z

packages/web3-eth/src/schemas.ts

 		},
-		logsBloom: {
-			format: 'bytes256',
+        blobGasUsed: {


reference: ethereum/execution-apis@057892b#diff-7c184b1c6b52b3ebe701b069a613639ad30699c60b9b352be78b64ac0206f424R82

jdevcs · 2024-04-15T14:56:34Z

packages/web3-eth/src/schemas.ts

+		parentBeaconBlockRoot: {
+			format: 'bytes32',
+		},
+		parentHash: {
+			format: 'bytes32',
+		},
+		receiptsRoot: {
+			format: 'bytes32',
+		},
+		sha3Uncles: {
+			format: 'bytes32',
 		},
 		size: {
 			format: 'uint',
 		},
+		stateRoot: {
+			format: 'bytes32',
+		},
+		timestamp: {
+			format: 'uint',
+		},
+		totalDifficulty: {
+			format: 'uint',
+		},


these fields are already in current schema but this PR is only rearranging these to diff lines.

jdevcs · 2024-04-15T15:14:59Z

packages/web3-types/src/eth_types.ts

+    readonly baseFeePerGas?: Numbers;
+    readonly blobGasUsed?: Numbers;
+    readonly difficulty?: Numbers;
+    readonly excessBlobGas?: Numbers;
+    readonly extraData?: Bytes;
+    readonly gasLimit: Numbers;
+    readonly gasUsed: Numbers;
+    readonly hash?: HexString32Bytes;
+    readonly logsBloom?: Bytes;
+    readonly miner?: HexString;
+    readonly nonce?: Numbers;
+    readonly number?: Numbers;
+    readonly parentBeaconBlockRoot?: HexString32Bytes;
+    readonly parentHash?: HexString32Bytes;
+    readonly receiptsRoot?: HexString32Bytes;
+    readonly sha3Uncles: HexString32Bytes[];
+    readonly stateRoot?: HexString32Bytes;
+    readonly timestamp: Numbers;
+    readonly transactionsRoot?: HexString32Bytes;
+    readonly withdrawalsRoot?: HexString32Bytes;
+
+    // These fields are returned when the RPC client is Nethermind,
+    // but aren't available in other clients such as Geth
+    readonly author?: Address;
+    readonly totalDifficulty?: Numbers;
+    readonly size?: Numbers;
+    readonly excessDataGas?: Numbers;
+    readonly mixHash?: HexString32Bytes;
+    readonly transactions?: TransactionOutput[];
+    readonly uncles?: Uncles;
+    readonly withdrawals?: Withdrawals[];


this is also rearranging most of field except adding ( eip 4844, 4895, 4788 )s

jdevcs

lgtm, @luu-alex plus add changelog in release PR for this:
added support of eip4844, eip4895, eip4788 block fields

Added EIP-4844 Headers for BlockHeaderOutput. getBlock() now referenc…

740bd93

…es blockHeaderSchema instead of blockSchema.

ymonye requested review from avkos, jdevcs, luu-alex and Muhammad-Altabba as code owners March 26, 2024 11:01

luu-alex reviewed Mar 26, 2024

View reviewed changes

jdevcs requested changes Mar 27, 2024

View reviewed changes

Updated blockSchema & blockHeaderSchema to reflect shared properties.

804ee89

ymonye force-pushed the eip4844headers-6933 branch from 3b53788 to 804ee89 Compare March 27, 2024 19:36

Removed changes from yarn.lock.

ec6a4fc

ymonye added 3 commits March 27, 2024 19:53

Removed author & excessDataGas from the BlockOutput interface.

d0a63c3

Separated old and new properties in the BlockOutput interface.

38a5b21

Moved the Withdrawals interface above BlockOutput.

8fdfa11

avkos approved these changes Mar 28, 2024

View reviewed changes

luu-alex approved these changes Apr 15, 2024

View reviewed changes

jdevcs reviewed Apr 15, 2024

View reviewed changes

jdevcs approved these changes Apr 15, 2024

View reviewed changes

jdevcs merged commit 578ebb6 into web3:4.x Apr 15, 2024
52 of 54 checks passed

snyk-io bot mentioned this pull request May 21, 2024

[Snyk] Upgrade web3 from 4.3.0 to 4.8.0 Hawthorne001/rpgf_calculator#1

Open

chaitanyapotti mentioned this pull request May 31, 2024

[Snyk] Upgrade web3 from 4.4.0 to 4.8.0 Web3Auth/web3auth-web#1835

Closed

skydrige mentioned this pull request Jun 13, 2024

[Snyk] Upgrade web3 from 4.8.0 to 4.9.0 skydrige/CryptoExchangeGateway#6

Open

aashutoshrathi mentioned this pull request Jun 15, 2024

[Snyk] Upgrade web3 from 4.8.0 to 4.9.0 aashutoshrathi/dcart#777

Closed

Magd3 mentioned this pull request Jun 27, 2024

[Snyk] Upgrade web3 from 4.0.1 to 4.9.0 Magd3/Majd#2

Open

swirlds-automation mentioned this pull request Jun 29, 2024

[Snyk] Upgrade web3 from 4.8.0 to 4.9.0 hashgraph/hedera-json-rpc-relay#2637

Merged

This was referenced Jul 17, 2024

[Snyk] Upgrade web3 from 4.7.0 to 4.10.0 didik-snyk/js-18-a#5

Open

[Snyk] Upgrade web3 from 4.7.0 to 4.10.0 didik-snyk/js-18#8

Open

chaitanyapotti mentioned this pull request Jul 25, 2024

[Snyk] Upgrade web3 from 4.8.0 to 4.10.0 Web3Auth/web3auth-web#1880

Open

luu-alex mentioned this pull request Oct 15, 2024

Include EIP-4788 #7312

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added EIP-4844 Headers for BlockHeaderOutput. Issue: 6933 #6937

Added EIP-4844 Headers for BlockHeaderOutput. Issue: 6933 #6937

ymonye commented Mar 26, 2024 •

edited

Loading

luu-alex commented Mar 26, 2024 •

edited

Loading

luu-alex Mar 26, 2024

ymonye Mar 26, 2024 •

edited

Loading

luu-alex Mar 27, 2024

ymonye Mar 27, 2024 •

edited

Loading

ymonye Mar 27, 2024 •

edited

Loading

ymonye Mar 28, 2024

ymonye commented Mar 27, 2024

jdevcs left a comment

ymonye commented Mar 27, 2024

codecov bot commented Mar 28, 2024

luu-alex commented Apr 15, 2024

jdevcs Apr 15, 2024

jdevcs Apr 15, 2024

jdevcs Apr 15, 2024

jdevcs left a comment

Added EIP-4844 Headers for BlockHeaderOutput. Issue: 6933 #6937

Added EIP-4844 Headers for BlockHeaderOutput. Issue: 6933 #6937

Conversation

ymonye commented Mar 26, 2024 • edited Loading

Description

Type of change

Checklist:

luu-alex commented Mar 26, 2024 • edited Loading

luu-alex Mar 26, 2024

Choose a reason for hiding this comment

ymonye Mar 26, 2024 • edited Loading

Choose a reason for hiding this comment

luu-alex Mar 27, 2024

Choose a reason for hiding this comment

ymonye Mar 27, 2024 • edited Loading

Choose a reason for hiding this comment

ymonye Mar 27, 2024 • edited Loading

Choose a reason for hiding this comment

ymonye Mar 28, 2024

Choose a reason for hiding this comment

ymonye commented Mar 27, 2024

jdevcs left a comment

Choose a reason for hiding this comment

ymonye commented Mar 27, 2024

codecov bot commented Mar 28, 2024

Codecov Report

luu-alex commented Apr 15, 2024

jdevcs Apr 15, 2024

Choose a reason for hiding this comment

jdevcs Apr 15, 2024

Choose a reason for hiding this comment

jdevcs Apr 15, 2024

Choose a reason for hiding this comment

jdevcs left a comment

Choose a reason for hiding this comment

ymonye commented Mar 26, 2024 •

edited

Loading

luu-alex commented Mar 26, 2024 •

edited

Loading

ymonye Mar 26, 2024 •

edited

Loading

ymonye Mar 27, 2024 •

edited

Loading

ymonye Mar 27, 2024 •

edited

Loading