Skip to content

Commit

Permalink
Migrate message experiment (#278)
Browse files Browse the repository at this point in the history
related to #237 


Similar to the other migrated experiments I followed same approach as
described here #268


> The experiment was executed and verified via the integration test
against a self-managed cluster.
> 
> I moved the experiment into the chaos-experiments/camunda-cloud/test/
folder and migrated it, with that approach I was able to execute the
experiment with eze and running against my self-managed zell-chaos zeebe
cluster.

Log output:

```
Create ChaosToolkit instance
Open workers: [zbchaos, readExperiments].
Handle read experiments job [key: 2251799813685265]
Read experiments successful, complete job with: {"experiments":[{"contributions":{"availability":"high","reliability":"high"},"description":"This fake experiment is just to test the integration with Zeebe and zbchaos workers","method":[{"name":"Show again the version","provider":{"arguments":["version"],"path":"zbchaos","timeout":900,"type":"process"},"tolerance":0,"type":"action"}],"rollbacks":[],"steady-state-hypothesis":{"probes":[{"name":"Show version","provider":{"arguments":["version"],"path":"zbchaos","timeout":900,"type":"process"},"tolerance":0,"type":"probe"}],"title":"Zeebe is alive"},"title":"This is a fake experiment","version":"0.1.0"},{"contributions":{"availability":"high","reliability":"high"},"description":"Zeebe message correlation should work even if the leader was restarted on which the message was published.","method":[{"name":"Deploy process model with catch event","provider":{"arguments":["deploy","process","--processModelPath","bpmn/msg-catch.bpmn"],"path":"zbchaos","type":"process"},"tolerance":0,"type":"action"},{"name":"Publish message to partition one","provider":{"arguments":["publish","--partitionId","1"],"path":"zbchaos","type":"process"},"tolerance":0,"type":"action"},{"name":"Terminate leader of partition 1 non-gracefully","provider":{"arguments":["terminate","broker","--role","LEADER","--partitionId","1"],"path":"zbchaos","type":"process"},"type":"action"},{"name":"All pods should be ready","provider":{"arguments":["verify","readiness"],"path":"zbchaos","timeout":900,"type":"process"},"tolerance":0,"type":"probe"},{"name":"Should be able to create a process instance and await the message correlation","provider":{"arguments":["verify","instance-creation","--awaitResult","--bpmnProcessId","oneReceiveMsgEvent","--variables","{\"key\": \"0\"}"],"path":"zbchaos","type":"process"},"tolerance":0,"type":"probe"}],"rollbacks":[],"steady-state-hypothesis":{"probes":[{"name":"All pods should be ready","provider":{"arguments":["verify","readiness"],"path":"zbchaos","timeout":900,"type":"process"},"tolerance":0,"type":"probe"}],"title":"Zeebe is alive"},"title":"Zeebe message correlation experiment","version":"0.1.0"}]}.
Handle zbchaos job [key: 2251799813685328]
Running command with args: [version] 
zbchaos development (commit: HEAD)
Handle zbchaos job [key: 2251799813685374]
Running command with args: [version] 
zbchaos development (commit: HEAD)
Handle zbchaos job [key: 2251799813685417]
Running command with args: [version] 
zbchaos development (commit: HEAD)
Handle zbchaos job [key: 2251799813685508]
Running command with args: [verify readiness] 
Connecting to zell-chaos
Running experiment in self-managed environment.
All Zeebe nodes are running.
Handle zbchaos job [key: 2251799813685553]
Running command with args: [deploy process --processModelPath bpmn/msg-catch.bpmn] 
Connecting to zell-chaos
Running experiment in self-managed environment.
Successfully created port forwarding tunnel
Deploy file bpmn/msg-catch.bpmn (size: 2986 bytes).
Deployed process model bpmn/msg-catch.bpmn successful with key 2251799813685249.
Deployed given process model bpmn/msg-catch.bpmn, under key 2251799813685249!
Handle zbchaos job [key: 2251799813685609]
Running command with args: [publish --partitionId 1] 
Connecting to zell-chaos
Running experiment in self-managed environment.
Successfully created port forwarding tunnel
Send message 'msg', with correaltion key '0' (ASCII: 48) 
Message was sent and returned key 2251799813685251, which corresponds to partition: 1
Handle zbchaos job [key: 2251799813685652]
Running command with args: [terminate broker --role LEADER --partitionId 1] 
Connecting to zell-chaos
Running experiment in self-managed environment.
Successfully created port forwarding tunnel
Found Broker zell-chaos-zeebe-0 as LEADER for partition 1.
Terminated zell-chaos-zeebe-0
Handle zbchaos job [key: 2251799813685696]
Running command with args: [verify readiness] 
Connecting to zell-chaos
Running experiment in self-managed environment.
Pod zell-chaos-zeebe-0 is in phase Pending, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Pending, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Pending, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
Pod zell-chaos-zeebe-0 is in phase Running, but not ready. Wait for some seconds.
All Zeebe nodes are running.
Handle zbchaos job [key: 2251799813686232]
Running command with args: [verify instance-creation --awaitResult --bpmnProcessId oneReceiveMsgEvent --variables {"key": "0"}] 
Connecting to zell-chaos
Running experiment in self-managed environment.
Successfully created port forwarding tunnel
Send create process instance command, with BPMN process ID 'oneReceiveMsgEvent' and version '-1' (-1 means latest) [variables: '{"key": "0"}', awaitResult: true]
Created process instance with key 2251799813685252 on partition 1, required partition 1.
The steady-state was successfully verified!
Handle zbchaos job [key: 2251799813686305]
Running command with args: [verify readiness] 
Connecting to zell-chaos
Running experiment in self-managed environment.
All Zeebe nodes are running.
Instance 2251799813685255 [definition 2251799813685253 ] completed
--- PASS: Test_ShouldBeAbleToRunExperiments (61.86s)
PASS

Process finished with the exit code 0
```
  • Loading branch information
ChrisKujawa authored Dec 8, 2022
2 parents 5ae7df5 + 7380c39 commit 3bdb41d
Show file tree
Hide file tree
Showing 6 changed files with 56 additions and 71 deletions.
2 changes: 1 addition & 1 deletion go-chaos/cmd/deploy.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ func init() {

deployCmd.AddCommand(deployProcessModelCmd)
deployProcessModelCmd.Flags().StringVar(&processModelPath, "processModelPath", "",
"Specify the path to a BPMN process model, which should be deployed. Defaults to a benchmark process model with one task (included in zbchaos).")
"Specify the path to a BPMN process model, which should be deployed. Defaults to a benchmark process model with one task (included in zbchaos). If the path starts with 'bpmn/' zbchaos will look for a referenced model bundled within the cli, like: 'bpmn/one_task.bpmn'.")

deployCmd.AddCommand(deployMultiVersionProcessModelCmd)
deployMultiVersionProcessModelCmd.Flags().IntVar(&versionCount, "versionCount", 10,
Expand Down
1 change: 0 additions & 1 deletion go-chaos/integration/integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,6 @@ func Test_ShouldBeAbleToRunExperiments(t *testing.T) {
// given
internal.Verbosity = true
cmd.Verbose = true
cmd.JsonLogging = true
ctx := context.Background()
container := CreateEZEContainer(t, ctx)
defer container.StopLogProducer()
Expand Down
22 changes: 11 additions & 11 deletions go-chaos/internal/bpmn/msg-catch.bpmn
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<?xml version="1.0" encoding="UTF-8"?>
<bpmn:definitions xmlns:bpmn="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:dc="http://www.omg.org/spec/DD/20100524/DC" xmlns:zeebe="http://camunda.org/schema/zeebe/1.0" xmlns:di="http://www.omg.org/spec/DD/20100524/DI" xmlns:modeler="http://camunda.org/schema/modeler/1.0" id="Definitions_1toz9lw" targetNamespace="http://bpmn.io/schema/bpmn" exporter="Camunda Modeler" exporterVersion="5.2.0" modeler:executionPlatform="Camunda Cloud" modeler:executionPlatformVersion="8.0.0">
<bpmn:process id="Process_0vxxyp6" isExecutable="true">
<bpmn:definitions xmlns:bpmn="http://www.omg.org/spec/BPMN/20100524/MODEL" xmlns:bpmndi="http://www.omg.org/spec/BPMN/20100524/DI" xmlns:dc="http://www.omg.org/spec/DD/20100524/DC" xmlns:zeebe="http://camunda.org/schema/zeebe/1.0" xmlns:di="http://www.omg.org/spec/DD/20100524/DI" xmlns:modeler="http://camunda.org/schema/modeler/1.0" id="Definitions_1toz9lw" targetNamespace="http://bpmn.io/schema/bpmn" exporter="Camunda Modeler" exporterVersion="5.5.1" modeler:executionPlatform="Camunda Cloud" modeler:executionPlatformVersion="8.0.0">
<bpmn:process id="oneReceiveMsgEvent" isExecutable="true">
<bpmn:startEvent id="StartEvent_1" name="Start">
<bpmn:extensionElements />
<bpmn:outgoing>Flow_1gq79wv</bpmn:outgoing>
Expand All @@ -22,15 +22,7 @@
</bpmn:extensionElements>
</bpmn:message>
<bpmndi:BPMNDiagram id="BPMNDiagram_1">
<bpmndi:BPMNPlane id="BPMNPlane_1" bpmnElement="Process_0vxxyp6">
<bpmndi:BPMNEdge id="Flow_12i7fo9_di" bpmnElement="Flow_12i7fo9">
<di:waypoint x="308" y="97" />
<di:waypoint x="372" y="97" />
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge id="Flow_1gq79wv_di" bpmnElement="Flow_1gq79wv">
<di:waypoint x="215" y="97" />
<di:waypoint x="272" y="97" />
</bpmndi:BPMNEdge>
<bpmndi:BPMNPlane id="BPMNPlane_1" bpmnElement="oneReceiveMsgEvent">
<bpmndi:BPMNShape id="_BPMNShape_StartEvent_2" bpmnElement="StartEvent_1">
<dc:Bounds x="179" y="79" width="36" height="36" />
<bpmndi:BPMNLabel>
Expand All @@ -49,6 +41,14 @@
<dc:Bounds x="380" y="122" width="20" height="14" />
</bpmndi:BPMNLabel>
</bpmndi:BPMNShape>
<bpmndi:BPMNEdge id="Flow_1gq79wv_di" bpmnElement="Flow_1gq79wv">
<di:waypoint x="215" y="97" />
<di:waypoint x="272" y="97" />
</bpmndi:BPMNEdge>
<bpmndi:BPMNEdge id="Flow_12i7fo9_di" bpmnElement="Flow_12i7fo9">
<di:waypoint x="308" y="97" />
<di:waypoint x="372" y="97" />
</bpmndi:BPMNEdge>
</bpmndi:BPMNPlane>
</bpmndi:BPMNDiagram>
</bpmn:definitions>
44 changes: 0 additions & 44 deletions go-chaos/internal/bpmn/oneReceiveMsgEvent.bpmn

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -15,41 +15,66 @@
"tolerance": 0,
"provider": {
"type": "process",
"path": "verify-readiness.sh",
"path": "zbchaos",
"arguments": ["verify", "readiness"],
"timeout": 900
}
}
]
},
"method": [
{
"name": "Deploy process model with catch event",
"type": "action",
"tolerance": 0,
"provider": {
"type": "process",
"path": "zbchaos",
"arguments": ["deploy", "process", "--processModelPath", "bpmn/msg-catch.bpmn"]
}
},
{
"type": "action",
"name": "Publish message to partition one",
"tolerance": 0,
"provider": {
"type": "process",
"path": "publish-message.sh",
"timeout": 900
"path": "zbchaos",
"arguments": ["publish", "--partitionId", "1"]
}
},
{
"type": "action",
"name": "Terminate leader of partition 1 non-gracefully",
"provider": {
"type": "process",
"path": "terminate-partition.sh",
"arguments": [ "Leader", "1" ]
"path": "zbchaos",
"arguments": ["terminate", "broker", "--role", "LEADER", "--partitionId", "1"]
}
},
{
"name": "Should be able to create a process and await the message correlation",
"type": "probe",
"tolerance": 0,
"provider": {
"type": "process",
"path": "await-message-correlation.sh",
"timeout": 900
}
"name": "All pods should be ready",
"type": "probe",
"tolerance": 0,
"provider": {
"type": "process",
"path": "zbchaos",
"arguments": ["verify", "readiness"],
"timeout": 900
}
},
{
"name": "Should be able to create a process instance and await the message correlation",
"type": "probe",
"tolerance": 0,
"provider": {
"type": "process",
"path": "zbchaos",
"arguments": ["verify", "instance-creation",
"--awaitResult",
"--bpmnProcessId", "oneReceiveMsgEvent",
"--variables", "{\"key\": \"0\"}"]
}
}
],
"rollbacks": []
Expand Down
7 changes: 6 additions & 1 deletion go-chaos/internal/zeebe.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
"errors"
"fmt"
"os"
"strings"
"time"

"github.com/camunda/zeebe/clients/go/v8/pkg/pb"
Expand Down Expand Up @@ -260,7 +261,11 @@ func readBPMNFileOrDefault(fileName string) ([]byte, string, error) {

if len(fileName) == 0 {
fileName = "bpmn/one_task.bpmn"

bpmnBytes, err = bpmnContent.ReadFile(fileName)
if err != nil {
return nil, "", err
}
} else if strings.Contains(fileName, "bpmn/") {
bpmnBytes, err = bpmnContent.ReadFile(fileName)
if err != nil {
return nil, "", err
Expand Down

0 comments on commit 3bdb41d

Please sign in to comment.