Overview | Hive Commands | Simulators | Clients
This guide explains how to write hive simulators.
A simulator is a program written against the HTTP-based simulation API provided by hive. Simulators can be written in any programming language as long as they are packaged using docker.
Simulators live in the simulators/ directory of the hive repository. There is a
dedicated sub-directory for every simulator. When hive runs a simulation, it first builds
an image using docker build in the simulator directory, using the Dockerfile. The image
must contain all resources needed for testing.
When the simulator container entry point runs, the HIVE_SIMULATOR environment variable
is set to the URL of the API server.
The simulation API assumes a certain data model, and this model dictates how the API can be used. In order to do anything with the API, the simulator must first request the start of a test suite and remembers its ID. Test suites have a name and description assigned by the simulator. All other resources provided by the API are scoped to the test suite and are kept until the simulator ends the suite.
Next, the simulator can start test cases on the suite. Test cases are named and also have an ID assigned by the API server. Multiple test cases may be running at any time within a single suite. Note that test suites do not have an overall pass / fail status, the only way to signal failure is with a test. At least one test case must be started for a suite, otherwise no results can be reported.
Within the context of a test case, client containers may be started. Clients are associated with the test and are shut down automatically when the test that started them ends. If many tests are to be executed against a single client, it is good practice to create a dedicated 'client launch' test just for starting the client, and then signal the results of the other tests as individual test cases.
The simulator must report the results of all running test cases before ending the test suite.
To share common code between Hive simulators, the code can be defined as a Go module
anywhere in the Hive repository, and added to the go.work file for local development.
This Go workspace links the simulators
and library code for local development across different modules.
Hive also supports pulling the local modules into the Docker build of the simulator by
changing the build context of the simulator with a hive_context.txt file in the
simulator directory root. This defines a relative path to start the docker build from.
Local Go modules within the docker build can be linked by defining a go.work
specifically for the Go modules copied into the docker build.
Example:
/simulators/my-simulator/hive_context.txt:
../..
/simulators/my-simulator/hive.go.work:
go 1.18
use (
./my-shared-code
./my-simulator
)
/simulators/my-simulator/Dockerfile:
# ...
ADD ./simulators/my-simulator/hive.go.work /source/go.work
ADD ./my-shared-code /source/my-shared-code
ADD ./simulators/my-simulator /source/my-simulator
WORKDIR /source/my-simulator
RUN go build -v .
# .../go.work:
go 1.18
use (
// ... -- other go modules in hive
./simulators/my-shared-code
./simulators/my-simulator
)
This is the list of all environment variables that hive sets when launching simulators.
| Variable | Meaning | Hive Flag |
|---|---|---|
HIVE_SIMULATOR |
URL of the API server | |
HIVE_TEST_PATTERN |
Regular expression, selects suites/tests | --sim.limit |
HIVE_PARALLELISM |
Integer, sets test concurrency | --sim.parallelism |
HIVE_RANDOM_SEED |
Integer, sets simulator random seed number | --sim.randomseed |
HIVE_LOGLEVEL |
Decimal 0-5, configures simulator log levels | --sim.loglevel |
While simulators may be written in any language (they're just docker containers after all), hive provides a Go library that wraps the simulation API in a way that resembles the standard library "testing" package. Be sure to check the Go API reference of package hivesim for more information about writing simulators in Go.
Simulators are contained in the hive repository as independent Go modules. To create one,
first create a new subdirectory in ./simulators and initialize a Go module there:
mkdir ./simulators/ethereum/my-simulation
cd ./simulators/ethereum/my-simulation
go mod init github.com/ethereum/hive/simulators/ethereum/my-simulation
go get github.com/ethereum/hive/hivesim@latest
Now create the simulator program file my-simulation.go.
package main
import "github.com/ethereum/hive/hivesim"
func main() {
suite := hivesim.Suite{
Name: "my-suite",
Description: "This test suite performs some tests.",
}
// add a plain test (does not run a client)
suite.Add(hivesim.TestSpec{
Name: "the-test",
Description: "This is an example test case.",
Run: runMyTest,
})
// add a client test (starts the client)
suite.Add(hivesim.ClientTestSpec{
Name: "the-test-2",
Description: "This is an example test case.",
Files: map[string]string{"/genesis.json": "genesis.json"},
Run: runMyClientTest,
})
// Run the tests. This waits until all tests of the suite
// have executed.
hivesim.MustRunSuite(hivesim.New(), suite)
}
func runMyTest(t *hivesim.T) {
// write your test code here
}
func runMyClientTest(t *hivesim.T, c *hivesim.Client) {
// write your test code here
}
The package hivesim provides automatic test case generation that can be used to compile all the test cases that a given simulator runs into a markdown file.
The documentation can be generated by executing the simulator binary with the environment variable
HIVE_DOCS_MODE set to "true".
The document generator recurses into every test suite and test function that the simulator defines
and collects their names and descriptions, as defined in the hivesim.Suite, hivesim.TestSpec
or hivesim.ClientTestSpec objects.
If the test object contains no description, it will not be listed in the documentation generated.
A TESTS.md file that serves as an index document will be generated, inside of it a listing of
all the test suites will be included, along with the links to all test suite markdown files.
For every test suite a TESTS-<Suite Name>.md file will be generated, containing the listing of
all test cases included in the suite.
The Location field of the hivesim.Suite can be used to specify a subdirectory where the
markdown file for this given suite will be placed.
The Category field of hivesim.TestSpec or hivesim.ClientTestSpec can be used to generate
test categories in which the test cases will be grouped for readability purposes.
The following environment variables can be used to configure document generation:
HIVE_DOCS_MODE: Enable test case documentation generation (set to "true").HIVE_SIMULATOR_NAME: Name of the simulator for which the documentation is being generated. If unset, the path of the simulator executable will be used to parse the simulator's name.HIVE_DOCS_OUTPUT_DIR: Output root directory for all generated markdown files. If unset, the current working directory will be used.
The simulator needs to have a Dockerfile in order to run.
As can be seen in the client test Files: part, the simulation requires a genesis.json
file that specifies the genesis state of the client. An example of genesis.json can be
found in the simulators/devp2p/init/ directory. You can copy an existing genesis block
or create your own. Make sure to add all support files to container in the Dockerfile. The
Dockerfile might look like this:
FROM golang:1-alpine AS builder
RUN apk --no-cache add gcc musl-dev linux-headers
ADD . /source
WORKDIR /source
RUN go build -o ./sim .
# Build the runner container.
FROM alpine:latest
ADD . /
COPY --from=builder /source/sim /
ENTRYPOINT ["./sim"]
You can test this build by running docker build . in the simulator directory.
Finally, go back to the root of the repository (cd ../../..) and run the simulation.
./hive --sim my-simulation --client go-ethereum,besu
You can check the results using hiveview.
This section lists all HTTP endpoints provided by the simulation API. Almost all API
endpoints consume and respond with payloads of type application/json.
When there is an error, the response will have a non 2xx status code and a response body containing JSON like:
{"error": "error message here"}
POST /testsuite
content-type: application/json
{"name": "test-suite-name", "description": "this suite does..."}
This request signals the start of a test suite. The API responds with a test suite ID.
200 OK
content-type: application/json
1
DELETE /testsuite/{suite}
This request ends a test suite. The simulator must end all running test cases before ending the test suite.
Response:
200 OK
POST /testsuite/{suite}/test
content-type: application/json
{"name": "test case name", "description": "..."}
The API responds with a test case ID.
200 OK
content-type: application/json
2
POST /testsuite/{suite}/test/{test}
content-type: application/json
{"pass": true, "details": "this is the test output"}
This request reports the result of a test case and ends the test case. Clients launched in the context of the test case are terminated by this request.
Response:
200 OK
GET /clients
This returns a JSON array of client definitions available to the simulation run. Clients
have a name, version, and meta for metadata as defined in the client interface
documentation.
Response
200 OK
content-type: application/json
[
{
"name": "go-ethereum",
"version": "Geth/v1.10.0-unstable-8e547eec-20210224/linux-amd64/go1.16",
"meta": {
"roles": [
"eth1"
]
}
},
{
"name": "besu",
"version": "besu/v21.1.1-dev-f1c74ed2/linux-x86_64/oracle_openjdk-java-11",
"meta": {
"roles": [
"eth1"
]
}
}
]
POST /testsuite/{suite}/test/{test}/node
content-type: multipart/form-data; boundary=--boundary--
--boundary--
content-disposition: form-data; name=config
{
"client": "go-ethereum_latest",
"environment": {"HIVE_CHAIN_ID": "8"}
}
--boundary--
content-disposition: form-data; name=/genesis.json; filename=genesis.json
{
"difficulty": "0x20000",
"gasLimit": "0xFFFFFFFF",
...
}
--boundary----
This request starts a client container. Unlike with other requests, this request must be
encoded as multipart/form-data. The config form parameter contains a client launch
configuration:
{
"client": "<client type>",
"networks: ["<network>"],
"environment": {
"HIVE_xxx": "<value>",
"HIVE_yyy": "<value>"
}
}
The "client" field is mandatory and gives the client type to be started. It must match
one of the names returned by the /clients endpoint.
"networks" is optional and configures networks to which the client will be connected
before it starts to run. Network names are supplied as a comma-separated list. The client
container will not be created if any of the given networks doesn't exist.
"environment" configures environment variables to be set in the client container. All
variable names must start with prefix HIVE_. Please see the client interface
documentation for environment variables supported by Ethereum clients.
The submitted form data may also contain files. Any form parameters with a non-empty filename are copied into the client container as files. Note: the form parameter name is used as the destination file name. The 'filename' submitted in the form is ignored. This is because multipart/form-data does not support specifying directory components in 'filename'.
Response:
200 OK
content-type: application/json
{"id": "<container-id>", "ip": "172.1.2.4"}
GET /testsuite/{suite}/test/{test}/node/{container}
This request returns basic information about a running client.
Response:
200 OK
content-type: application/json
{"id":"abcdef1234","name":"go-ethereum_latest"}
POST /testsuite/{suite}/test/{test}/node/{container}/exec
content-type: application/json
{
"command": ["my-script", "arg1"]
}
This request invokes a script in the client container. The script must be present in the
client container's filesystem in the /hive-bin directory.
Response:
200 OK
content-type: application/json
{
"exitCode": 0,
"stdout": "output",
"stderr": "error output"
}
DELETE /testsuite/{suite}/test/{test}/node/{container}
This terminates the given client container immediately. Using this endpoint is usually not required because all clients associated with a test will be shut down when the test ends.
Response:
200 OK
POST /testsuite/{suite}/network/{network}
This request creates a network. Unlike with other APIs, networks do not have IDs. Instead, the network name is assigned by the simulator.
Response:
200 OK
DELETE /testsuite/{suite}/network/{network}
This request removes a network. Note: the request will fail if any containers are still connected to the network.
Response:
200 OK
POST /testsuite/{suite}/network/{network}/{container}
This request connects a client container to a network. You can use any client container ID
as the container. You can also use "simulation" as the container ID, in which case the
container running the simulator will be connected.
Response:
200 OK
DELETE /testsuite/{suite}/network/{network}/{container}
This request disconnects a container from a network. As with the connect request, use any
client container ID or "simulation" as the container value.
Response:
200 OK
GET /testsuite/{suite}/network/{network}/{container}
This returns the IP of a container on the given network.
Response:
200 OK
content-type: application/json
"172.22.0.2"