Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add simulateTransaction endpoint #1610

Merged
merged 29 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
45749b1
initial version
tsachiherman Sep 30, 2024
23c8b1d
Merge branch 'main' into tsachi/simulate_transaction
tsachiherman Sep 30, 2024
65df47b
update
tsachiherman Sep 30, 2024
6fc64c9
linter
tsachiherman Sep 30, 2024
693b7c0
update
tsachiherman Oct 1, 2024
0da1d84
rollback unwanted changes
tsachiherman Oct 1, 2024
1a2e340
correct typo
tsachiherman Oct 1, 2024
8253c81
Merge branch 'main' into tsachi/simulate_transaction
tsachiherman Oct 1, 2024
7d26831
add encoding to state keys.
tsachiherman Oct 1, 2024
75c07e6
update
tsachiherman Oct 2, 2024
a306b46
Merge branch 'main' into tsachi/simulate_transaction
tsachiherman Oct 2, 2024
5379398
remove comments
tsachiherman Oct 2, 2024
ff86ecf
Merge branch 'tsachi/simulate_transaction' of github.com:ava-labs/hyp…
tsachiherman Oct 2, 2024
f42b284
Merge branch 'main' into tsachi/simulate_transaction
tsachiherman Oct 3, 2024
c08b8cf
add keys marshaler
tsachiherman Oct 3, 2024
793a6b0
step
tsachiherman Oct 3, 2024
7f3868e
Merge branch 'main' into tsachi/simulate_transaction
tsachiherman Oct 4, 2024
5399ff3
update recorder.
tsachiherman Oct 7, 2024
c172c59
update
tsachiherman Oct 7, 2024
89a134d
update
tsachiherman Oct 8, 2024
f5d4dd8
update
tsachiherman Oct 8, 2024
e3855fe
linter
tsachiherman Oct 8, 2024
9c74334
update per CR
tsachiherman Oct 9, 2024
267e13b
lint
tsachiherman Oct 9, 2024
b6bd4f2
Update state/recorder.go
tsachiherman Oct 9, 2024
e51d8fd
Update state/recorder.go
tsachiherman Oct 9, 2024
689d0ec
Update state/recorder.go
tsachiherman Oct 9, 2024
8f020d8
fix PR
tsachiherman Oct 9, 2024
65c9c42
Merge branch 'main' into tsachi/simulate_transaction
tsachiherman Oct 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions api/jsonrpc/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ package jsonrpc

import (
"context"
"errors"
"fmt"
"strings"
"time"
Expand Down Expand Up @@ -235,3 +236,55 @@ func Wait(ctx context.Context, interval time.Duration, check func(ctx context.Co
}
return ctx.Err()
}

type SimulatedActionResponse struct {
Output []byte
Error error
}

type SimulateTransactionResponse struct {
SimulatedActions []SimulatedActionResponse
ReadKeys []string
WriteKeys []string
AllocateKeys []string
BlockHeight uint64
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we re-use the types specified in server.go as we do for the existing server/client implementations?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


func (cli *JSONRPCClient) SimulateTransaction(ctx context.Context, txn []byte) (*SimulateTransactionResponse, error) {
args := &SimulateTransactionArgs{
Tx: txn,
}

resp := new(SimulateTransactionReply)
err := cli.requester.SendRequest(
ctx,
"simulateTransaction",
args,
resp,
)
if err != nil {
return nil, err
}
if resp.Error != "" {
return nil, fmt.Errorf("failed to execute action: %s", resp.Error)
}

// copy the response from resp into the model SimulateTransactionResponse
output := SimulateTransactionResponse{
ReadKeys: resp.ReadKeys,
WriteKeys: resp.WriteKeys,
AllocateKeys: resp.AllocateKeys,
BlockHeight: resp.BlockHeight,
}
for _, action := range resp.SimulatedActions {
entry := SimulatedActionResponse{
Output: action.Output,
}
if action.Error != nil {
entry.Error = errors.New(*action.Error)
}
output.SimulatedActions = append(output.SimulatedActions, entry)
}

return &output, nil
}
94 changes: 94 additions & 0 deletions api/jsonrpc/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import (
"github.com/ava-labs/hypersdk/codec"
"github.com/ava-labs/hypersdk/consts"
"github.com/ava-labs/hypersdk/fees"
"github.com/ava-labs/hypersdk/state"
"github.com/ava-labs/hypersdk/state/tstate"
)

Expand Down Expand Up @@ -235,3 +236,96 @@ func (j *JSONRPCServer) Execute(

return nil
}

type SimulateTransactionArgs struct {
Tx []byte `json:"tx"`
}

type SimulatedAction struct {
Output []byte `json:"output_b64"`
Error *string `json:"error"`
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we use codec.Bytes in API responses when passing []byte values to provide a reasonable JSON output?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


type SimulateTransactionReply struct {
SimulatedActions []SimulatedAction `json:"simulatedActions"`
ReadKeys []string `json:"readKeys"`
WriteKeys []string `json:"writeKeys"`
AllocateKeys []string `json:"allocKeys"`
BlockHeight uint64 `json:"blockheight"`
Error string `json:"error"`
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we reduce the state key fields here down to just state.Keys and add support for JSON marshal/unmarshal directly to this type?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to remove BlockHeight and Error (in favor of returning the error directly and pushing to the client to handle it) unless there's a clear need to include them in the response.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


type SimulationEnabledRules struct {
chain.Rules
}

func (SimulationEnabledRules) GetTransactionExecutionMode() chain.TransactionExecutionMode {
return chain.SimulatedTransactionExecution
}

func (j *JSONRPCServer) SimulateTransaction(
req *http.Request,
args *SimulateTransactionArgs,
reply *SimulateTransactionReply,
) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we start with simulating a single action and providing the additional arguments as needed (actor) ?

In our case, transactions include additional data that we don't need to require from the user. For example, with this API you can only simulate an action from a given address if you can generate a signature from that address.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

ctx, span := j.vm.Tracer().Start(req.Context(), "JSONRPCServer.SimulateTransaction")
defer span.End()

if reply == nil {
return errors.New("SimulateTransaction was called with a nil reply object")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be provided by the server, so we can remove this check and follow the style in the rest of the APIs of not checking if the reply value is nil.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


actionRegistry, authRegistry := j.vm.ActionRegistry(), j.vm.AuthRegistry()
rtx := codec.NewReader(args.Tx, consts.NetworkSizeLimit) // will likely be much smaller than this
tx, err := chain.UnmarshalTx(rtx, actionRegistry, authRegistry)
if err != nil {
return fmt.Errorf("%w: unable to unmarshal on public service", err)
}
if !rtx.Empty() {
return errors.New("tx has extra bytes")
}
actor := tx.Auth.Actor()

currentState, err := j.vm.ImmutableState(ctx)
if err != nil {
return err
}
reply.BlockHeight = j.vm.LastAcceptedBlock().Hght
recorder := state.NewRecorder(currentState)
simulationEnabledRules := SimulationEnabledRules{j.vm.Rules(tx.Base.Timestamp)}
for actionIdx, action := range tx.Actions {
actionOutput, err := action.Execute(ctx, simulationEnabledRules, recorder, tx.Base.Timestamp, actor, chain.CreateActionID(tx.ID(), uint8(actionIdx)))

var simAction SimulatedAction
if actionOutput == nil {
simAction.Output = []byte{}
} else {
simAction.Output, err = chain.MarshalTyped(actionOutput)
if err != nil {
return fmt.Errorf("failed to marshal output: %w", err)
}
}
if err != nil {
errString := err.Error()
simAction.Error = &errString
}

reply.SimulatedActions = append(reply.SimulatedActions, simAction)

if err != nil {
break
}
}
for key, perm := range recorder.GetStateKeys() {
if perm.Has(state.Read) {
reply.ReadKeys = append(reply.ReadKeys, key)
}
if perm.Has(state.Write) {
reply.WriteKeys = append(reply.WriteKeys, key)
}
if perm.Has(state.Allocate) {
reply.AllocateKeys = append(reply.AllocateKeys, key)
}
}
return nil
}
13 changes: 11 additions & 2 deletions chain/dependencies.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@ type (
ActionRegistry *codec.TypeParser[Action]
OutputRegistry *codec.TypeParser[codec.Typed]
AuthRegistry *codec.TypeParser[Auth]

TransactionExecutionMode int
)

const (
OnchainTransactionExecution TransactionExecutionMode = iota
SimulatedTransactionExecution
)

type Parser interface {
Expand Down Expand Up @@ -152,6 +159,8 @@ type Rules interface {
GetStorageValueWriteUnits() uint64 // per chunk

FetchCustom(string) (any, bool)

GetTransactionExecutionMode() TransactionExecutionMode // either OnchainTransactionExecution or SimulatedTransactionExecution
}

type MetadataManager interface {
Expand Down Expand Up @@ -229,8 +238,8 @@ type Action interface {
// Execute actually runs the [Action]. Any state changes that the [Action] performs should
// be done here.
//
// If any keys are touched during [Execute] that are not specified in [StateKeys], the transaction
// will revert and the max fee will be charged.
// If any keys are touched during [Execute] while running in [OnchainTransactionExecution] mode that
// are not specified in [StateKeys], the transaction will revert and the max fee will be charged.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm this comment was actually not true before. If a state key is read that was not specified, it's up to the action to decide whether to propagate it (usually should).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we revert this if we're able to remove OnchainTransactionExecution mode?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

//
// If [Execute] returns an error, execution will halt and any state changes will revert.
Execute(
Expand Down
13 changes: 10 additions & 3 deletions examples/vmwithcontracts/actions/call.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ import (
"github.com/ava-labs/hypersdk/chain"
"github.com/ava-labs/hypersdk/codec"
"github.com/ava-labs/hypersdk/consts"
"github.com/ava-labs/hypersdk/examples/vmwithcontracts/storage"
"github.com/ava-labs/hypersdk/state"
"github.com/ava-labs/hypersdk/x/contracts/runtime"

mconsts "github.com/ava-labs/hypersdk/examples/vmwithcontracts/consts"
mstorage "github.com/ava-labs/hypersdk/examples/vmwithcontracts/storage"
)

var _ chain.Action = (*Call)(nil)
Expand Down Expand Up @@ -62,16 +62,23 @@ func (t *Call) StateKeys(_ codec.Address) state.Keys {

func (t *Call) Execute(
ctx context.Context,
_ chain.Rules,
r chain.Rules,
mu state.Mutable,
timestamp int64,
actor codec.Address,
_ ids.ID,
) (codec.Typed, error) {
var limitStateKeys map[string]state.Permissions
if r.GetTransactionExecutionMode() == chain.OnchainTransactionExecution {
limitStateKeys = make(map[string]state.Permissions, len(t.SpecifiedStateKeys))
for _, statekey := range t.SpecifiedStateKeys {
limitStateKeys[statekey.Key] = statekey.Permission
}
}
resutBytes, err := t.r.CallContract(ctx, &runtime.CallInfo{
Contract: t.ContractAddress,
Actor: actor,
State: &storage.ContractStateManager{Mutable: mu},
State: mstorage.NewContractStateManager(mu, limitStateKeys),
FunctionName: t.Function,
Params: t.CallData,
Timestamp: uint64(timestamp),
Expand Down
22 changes: 22 additions & 0 deletions examples/vmwithcontracts/storage/programs.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ import (
"github.com/ava-labs/hypersdk/x/contracts/runtime"
)

var ErrUnpermissionedKeyAccessed = errors.New("unpermissioned key accessed")

// [accountStatePrefix] + [account]
func accountStateKey(account codec.Address) (k []byte) {
k = make([]byte, 2+codec.AddressLen)
Expand Down Expand Up @@ -60,6 +62,14 @@ var _ runtime.StateManager = (*ContractStateManager)(nil)

type ContractStateManager struct {
state.Mutable
limitStateKeys map[string]state.Permissions
}

func NewContractStateManager(mu state.Mutable, limitStateKeys map[string]state.Permissions) *ContractStateManager {
return &ContractStateManager{
Mutable: mu,
limitStateKeys: limitStateKeys,
}
}

func (p *ContractStateManager) GetBalance(ctx context.Context, address codec.Address) (uint64, error) {
Expand All @@ -81,6 +91,9 @@ func (p *ContractStateManager) GetContractState(address codec.Address) state.Mut

func (p *ContractStateManager) GetAccountContract(ctx context.Context, account codec.Address) (runtime.ContractID, error) {
key, _ := keys.Encode(AccountContractKey(account), 36)
if perm, has := p.limitStateKeys[string(key)]; has && !perm.Has(state.Read) {
return ids.Empty[:], ErrUnpermissionedKeyAccessed
}
result, err := p.GetValue(ctx, key)
if err != nil {
return ids.Empty[:], err
Expand All @@ -89,12 +102,18 @@ func (p *ContractStateManager) GetAccountContract(ctx context.Context, account c
}

func (p *ContractStateManager) GetContractBytes(ctx context.Context, contractID runtime.ContractID) ([]byte, error) {
if perm, has := p.limitStateKeys[string(contractID)]; has && !perm.Has(state.Read) {
return ids.Empty[:], ErrUnpermissionedKeyAccessed
}
return p.GetValue(ctx, contractID)
}

func (p *ContractStateManager) NewAccountWithContract(ctx context.Context, contractID runtime.ContractID, accountCreationData []byte) (codec.Address, error) {
newAddress := GetAddressForDeploy(0, accountCreationData)
key, _ := keys.Encode(AccountContractKey(newAddress), 36)
if perm, has := p.limitStateKeys[string(contractID)]; has && !perm.Has(state.Read) {
return codec.EmptyAddress, ErrUnpermissionedKeyAccessed
}
_, err := p.GetValue(ctx, key)
if err != nil && !errors.Is(err, database.ErrNotFound) {
return codec.EmptyAddress, err
Expand All @@ -107,6 +126,9 @@ func (p *ContractStateManager) NewAccountWithContract(ctx context.Context, contr

func (p *ContractStateManager) SetAccountContract(ctx context.Context, account codec.Address, contractID runtime.ContractID) error {
key, _ := keys.Encode(AccountContractKey(account), 36)
if perm, has := p.limitStateKeys[string(key)]; has && !perm.Has(state.Write) && !perm.Has(state.Allocate) {
return ErrUnpermissionedKeyAccessed
}
return p.Insert(ctx, key, contractID)
}

Expand Down
2 changes: 1 addition & 1 deletion examples/vmwithcontracts/vm/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ func (j *JSONRPCServer) simulate(ctx context.Context, t actions.Call, actor code
if err != nil {
return nil, 0, err
}
recorder := storage.NewRecorder(currentState)
recorder := state.NewRecorder(currentState)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we remove this from vmwithcontracts/ since it can use the general purpose API after this PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

startFuel := uint64(1000000000)
callInfo := &runtime.CallInfo{
Contract: t.ContractAddress,
Expand Down
4 changes: 4 additions & 0 deletions genesis/rules.go
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,10 @@ func (*Rules) FetchCustom(string) (any, bool) {
return nil, false
}

func (*Rules) GetTransactionExecutionMode() chain.TransactionExecutionMode {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason that we need to add this mode to Rules rather than just tracking what state keys were touched by executing an action?

If we just wrap the current view produced by the VM and run Execute against it, I don't think that we should need to add key tracking within vmwithcontracts at all and can remove this extra function on Rules.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

return chain.OnchainTransactionExecution
}

type ImmutableRuleFactory struct {
Rules chain.Rules
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,26 +1,24 @@
// Copyright (C) 2024, Ava Labs, Inc. All rights reserved.
// See the file LICENSE for licensing terms.

package storage
package state

import (
"context"
"errors"

"github.com/ava-labs/avalanchego/database"
"github.com/ava-labs/avalanchego/utils/set"

"github.com/ava-labs/hypersdk/state"
)

type Recorder struct {
State state.Immutable
State Immutable
changedValues map[string][]byte
ReadState set.Set[string]
WriteState set.Set[string]
}

func NewRecorder(db state.Immutable) *Recorder {
func NewRecorder(db Immutable) *Recorder {
return &Recorder{State: db, changedValues: map[string][]byte{}}
}

Expand Down Expand Up @@ -50,10 +48,10 @@ func (r *Recorder) GetValue(ctx context.Context, key []byte) (value []byte, err
return r.State.GetValue(ctx, key)
}

func (r *Recorder) GetStateKeys() state.Keys {
result := state.Keys{}
func (r *Recorder) GetStateKeys() Keys {
aaronbuchwald marked this conversation as resolved.
Show resolved Hide resolved
result := Keys{}
for key := range r.ReadState {
result.Add(key, state.Read)
result.Add(key, Read)
}
for key := range r.WriteState {
if _, err := r.State.GetValue(context.Background(), []byte(key)); err != nil && errors.Is(err, database.ErrNotFound) {
Expand All @@ -62,9 +60,9 @@ func (r *Recorder) GetStateKeys() state.Keys {
continue
}
// wasn't found so needs to be allocated
result.Add(key, state.Allocate)
result.Add(key, Allocate)
}
result.Add(key, state.Write)
result.Add(key, Write)
}
return result
}
Loading