Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update #13

Open
wants to merge 63 commits into
base: mainnet
Choose a base branch
from
Open

Update #13

wants to merge 63 commits into from

Conversation

Giems
Copy link

@Giems Giems commented Sep 17, 2024

No description provided.

pei-mysten and others added 30 commits June 20, 2024 20:19
## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
…8392)

This reverts commit 1d48056.

## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
…MystenLabs#18415) (MystenLabs#18419)

## Description 

Fix a typo issue in an old execution version 

## Test plan 

Historical replays
…alidator` crate into `sui` to release v1.28 branch (MystenLabs#18436)

This PR integrates `sui-test-validator` binary into `sui start`, but
keeps the crate for now before being deprecated.
- the default behaviour of the existing `sui start` is preserved
- the default `sui-test-validator` behaviour can be achieved by passing
two flags: `sui start --force-regenesis --with-faucet`.
- adds a `sui-test-validator.sh` bash script that is executable to keep
backward compatibility with the previous binary
- keeps the `sui-test-validator` binary for now.
- changed default port for GraphQL to 9125

One difference in flag/options usage is that `sui-test-validator` used
to start `graphql` service when the `graphql-port` arg was passed. To
keep it consistent across all three services, they each have their own
flag to use to start the service with default host and port, and a port
can be provided to it as well:
- `--with-faucet` or `--with-faucet=9123`
- `--with-indexer` or `--with-indexer=9124`
- `--with-graphql` or `--with-graphql=9125`

Previously, `sui-test-validator` exposed a few flags and options (see
orig code

[here](https://github.com/MystenLabs/sui/blob/4f52e053f2869328d9fe62501853fbd93c8624d7/crates/sui-test-validator/src/main.rs#L30-L85)),
which were migrated to `sui start`. There are two new flags in `sui
start`, `--with-graphql` and `--with-faucet`, to make it easier to start
these services with default values. Due to this, the bash script checks
for multiple flags and their values and tries to ensure everything is
passed correctly.

`sui-test-validator` implemented a simple faucet wrapper of the
`sui-faucet` crate. This was moved into `faucet.rs` module in `sui`
crate.

Given that there are three extra services that can be started, plus a
configuration variable to start the network without a fullnode, here is
an explanation of what each service requires:
`graphql requires indexer` - without an indexer, the GraphQL will have
no data to serve. `--with-indexer` will be implied if `--with-graphql`
is passed. `indexer requires fullnode` - without a fullnode, the indexer
will start ingesting mainnet data with default settings. The program
throws an error here.

Existing tests.

---

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol:
- [ ] Nodes (Validators and Full nodes):
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [x] CLI: Integrated `sui-test-validator` into `sui start`.

`sui start` can now be used to also start an indexer, a GraphQL service,
or a faucet. Note that you need to build `sui` with `--features
indexer`, and have installed `libpq` to start the indexer and GraphQL
services. By default, this feature is turned off to preserve backward
compatibility of `sui start`. Alternatively, you can download the latest
release and use the `sui-pg` binary, which is built using the `indexer`
feature.
- the default behaviour of the existing `sui start` is preserved
- the default `sui-test-validator` behaviour can be achieved by passing
two flags: `sui start --force-regenesis --with-faucet`.
- adds a `sui-test-validator.sh` bash script to `sui/scripts` folder
that is executable to keep backward compatibility with the previous
binary
- [ ] Rust SDK:
## Description 

CP MystenLabs#18396 and
MystenLabs#18324

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
## Description 

CP MystenLabs#18281

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:

Co-authored-by: Mark Logan <[email protected]>
## Description 

Same as MystenLabs#18444 but against the
release branch

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
…) (MystenLabs#18449)

Create new epoch flags for state accum v2, since the old one was already
written in some places. Also fix chain-specific logic

Cherry pick of MystenLabs#18448
## Description 

Fixes a bug I introduced with MystenLabs#18204 in the bash script building
`sui-pg` binary.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
…#18476)

## Description 
Cherry pick MystenLabs#18469 into release branch v1.28

Prior to MystenLabs#18204, `sui start` accepted a network config file when using
`--network.config`, and `sui-test-validator` accepted a config directory
when using the `--config-dir`, so we need to support both. That PR
introduces a bug as in it accepts only directories.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
…ndexer to compile for Rust 1.75 (MystenLabs#18490) (MystenLabs#18496)

## Description 

These changes ensure that brew can compile our code to release `sui`
binary, and similarly, that `cargo install [..] sui --branch testnet`
does not break. We need this to go with the testnet release today.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
…ystenLabs#18712)

## Description

Always pick whole UTF8 characters.

## Test plan

Tested against a package containing a UTF8 constant.

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [x] Nodes (Validators and Full nodes): Bugfix for fetching package
information over JSONRPC from fullnodes, where a package containing UTF8
string constants could fail to disassemble.
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
…ystenLabs#18713)

## Description

Always pick whole UTF8 characters.

## Test plan

Tested against a package containing a UTF8 constant.

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol:
- [x] Nodes (Validators and Full nodes): Bugfix for fetching package
information over JSONRPC from fullnodes, where a package containing UTF8
string constants could fail to disassemble.
- [ ] Indexer:
- [ ] JSON-RPC:
- [ ] GraphQL:
- [ ] CLI:
- [ ] Rust SDK:
- [ ] REST API:
## Description 

- Cherry pick 18827 into 1.30
- Changes protocol version 53, which has already hit devnet, but not
testnet

## Test plan 

- CI

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

- Cherry pick MystenLabs#18842
- Source only change, does not affect devnet/testnet or any other build 

## Test plan 

- CI

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
NorbertBodziony and others added 25 commits August 12, 2024 12:26
## Description 

Adds recent Option changes to release branch. 

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
…itting… (MystenLabs#19135)

…_module` (MystenLabs#18740)

## Description 

Cherry pick event indexer + graphql changes.
## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:

---------

Co-authored-by: wlmyng <[email protected]>
Co-authored-by: Lu Zhang <[email protected]>
## Description 

MystenLabs#19112: this emits more metrics from rocksdb, to help with
investigations into external fullnode write stalls.

## Test plan 

CI

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:

## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
)

## Description 

enable bridge creation on mainnet by setting `bridge` is true. Before
this change, the value true for testnet and devnet, but not mainnet. So
this only applies to mainnet.


## Test plan 

it creates a new version 56 and enables bridge feature flag on mainnet.
This change creates
*
[sui_protocol_config__test__Mainnet_version_56.snap](https://github.com/MystenLabs/sui/blob/4b2e1ab739cee34e4075717b52766eb3a07946df/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_56.snap)
and
*
[sui_protocol_config__test__Testnet_version_56.snap](https://github.com/MystenLabs/sui/blob/4b2e1ab739cee34e4075717b52766eb3a07946df/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_56.snap)

and it does NOT change
*
[sui_protocol_config__test__Testnet_version_55.snap](https://github.com/MystenLabs/sui/blob/4b2e1ab739cee34e4075717b52766eb3a07946df/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_55.snap)
and

[sui_protocol_config__test__Mainnet_version_55.snap](https://github.com/MystenLabs/sui/blob/4b2e1ab739cee34e4075717b52766eb3a07946df/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_55.snap)

I did the following checks:
*
[sui_protocol_config__test__Mainnet_version_56.snap](https://github.com/MystenLabs/sui/blob/4b2e1ab739cee34e4075717b52766eb3a07946df/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_56.snap)
is different from existing
[sui_protocol_config__test__Mainnet_version_55.snap](https://github.com/MystenLabs/sui/blob/4b2e1ab739cee34e4075717b52766eb3a07946df/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Mainnet_version_55.snap)
for bridge: [diff](https://www.diffchecker.com/50VcWjj8/)
*
[sui_protocol_config__test__Testnet_version_56.snap](https://github.com/MystenLabs/sui/blob/4b2e1ab739cee34e4075717b52766eb3a07946df/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_56.snap)
and
[sui_protocol_config__test__Testnet_version_55.snap](https://github.com/MystenLabs/sui/blob/4b2e1ab739cee34e4075717b52766eb3a07946df/crates/sui-protocol-config/src/snapshots/sui_protocol_config__test__Testnet_version_55.snap)
are the same: [diff](https://www.diffchecker.com/6NbvoYmk/)

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [x] Protocol: 
- [x] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
…s#19251)

Change AuthorityAggregator to always pass through request_events flag,
even to the validators we aren't sampling for objects.

In addition, up the sample size of validators to 10 and log when we get
unlucky and reach quorum without having recieved input or output
objects.
Copy link

github-actions bot commented Sep 17, 2024

⚠️ 🦋 Changesets Warning: This PR has changes to public npm packages, but does not contain a changeset. You can create a changeset easily by running pnpm changeset in the root of the Sui repo, and following the prompts. If your change does not need a changeset (e.g. a documentation-only change), you can ignore this message. This warning will be removed when a changeset is added to this pull request.

Learn more about Changesets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.