Cluster Tests
Cluster tests introduce different failures to the system and verify that safety is not breached during those experiments. The structure of the tests is inspired by Diem cluster tests. You can run the tests on both local machine and remote linux servers. Implementation can be found here.
Table of contents
Structure
Major components of tests include:
Experiment is some condition we want to test our system with such as restart some nodes.
HealthCheck is how we verify whether the system is running correctly or not. Runner verifies that cluster is healthy before and after running each experiment. Health is verified in three aspects:
- Safety: nodes must have the same state at a certain block height
- Liveness: the system must commits blocks and transactions
- Leader Rotation: the system must rotate leader
Setup Development Environment
- Install dependencies
# MacOS xcode-select --install
# Ubuntu sudo apt-get install build-essential
- Download and install
go 1.16
- Prepare the repo
git clone https://github.com/aungmawjj/juria-blockchain cd juria-blockchain go mod tidy
Local Cluster
Run tests on local cluster.
cd tests
go run .
The test script will compile juria
and set up a cluster of 4 nodes with different ports on the local machine. Runner will run Experiments and perform HealthChecks.
NOTE: Network simulation experiments are only run on the remote linux cluster.
Remote Linux Cluster
Configure main.go
to run tests on remote linux servers.
RemoteLinuxCluster = true // set true
Provide two files:
serverkey
: ssh key for remote loginhosts
: hostnames (ip addresses) with each line for one server
Run tests.
go run .