GoSteps is a go library that helps in running functions as steps and reminds you to step out and get active (kidding!).
The idea behind gosteps is to define set of functions as chain-of-steps and execute them in a sequential fashion.
Note
go-steps v1 is a breaking change and older v0 models won't work with the new version. To continue to use the v0, you can either use the v0 sub-package of the library or the v0.3.0 tag. For usage documentation of v0, refer to the v0 README.
Functions that needs execution are defined as a Step, and each step is a part of a collection of multiple Steps called Branch, that runs sequentially. Each step can have multiple Branches as possible next steps, and the branch to be executed is determined based on a ResolverFuntion of that step.
go get github.com/TanmoySG/go-stepsDifferent types defined in GoSteps are required to create a step chain.
The Step type contains the requirments to execute a step function and is a part of a Branch.
type Step struct {
Name StepName `json:"name"`
Function StepFn `json:"-"`
StepOpts StepOpts `json:"stepConfig"`
Branches *Branches `json:"branches"`
StepArgs map[string]interface{} `json:"stepArgs"`
}
// example step
step := gosteps.Step{
Name: "add",
Function: Add,
StepArgs: map[string]interface{}{"n1": 5},
StepOpts: gosteps.StepOpts{},
Branches: &gosteps.Branches{},
}| Field | Description |
|---|---|
| Name | Name of step |
| Function | The function to execute |
| StepOpts | Options/Configurations of the step |
| Branches | Branches are a sequentially executable collection of steps. |
| StepArgs | Any additional arguments/variables needed to be passed to the step for execution. |
StepOpts
The StepOpts type contains the configurations/options for the step execution.
type StepOpts struct {
ErrorsToRetry []error `json:"errorsToRetry"`
ErrorPatternsToRetry []regexp.Regexp `json:"errorPatternsToRetry"`
RetryAllErrors bool `json:"retryAllErrors"`
MaxRunAttempts int `json:"maxAttempts"`
RetrySleep time.Duration `json:"retrySleep"`
}| Field | Description |
|---|---|
| ErrorsToRetry | a set of errors for which a step should be retried. |
| ErrorPatternsToRetry | a set of StepErrorPatterns for which a step should be retried, if error matches pattern |
| RetryAllErrors | A boolean type flag which specifies if a step needs to retry for any error, irrespective of those in ErrorsToRetry |
| MaxRunAttempts | Max attempts are the number of times the step is ran/executed (first run + retries). If not set, it'll run once. |
| RetrySleep | Sleep duration (type time.Duration) between each re-attempts |
Function
Defines a step function of type StepFn.
// StepFn Type
type StepFn func(ctx GoStepsCtx) StepResult
// example function
func Add(ctx GoStepsCtx) StepResult {
// do something
return gosteps.MarkStateComplete()
}The Branch type is a collection of steps that are to be executed sequentially.
// Steps type - array of Step
type Steps []Step
type Branch struct {
BranchName BranchName `json:"branchName"`
Steps Steps `json:"steps"`
}
// example branch
branch := gosteps.Branch{
BranchName: "divide",
Steps: gosteps.Steps{
{
Name: "step.divide",
Function: func(c gosteps.GoStepsCtx) gosteps.StepResult {
// do something
return gosteps.MarkStateComplete()
},
},
{ /* other steps*/ },
},
}| Field | Description |
|---|---|
| BranchName | Name of the branch |
| Steps | Steps are a collection of steps that are to be executed sequentially. It is a array of Step ([]Step) |
Branches
The Branches type contains an array of branch that are exececuted conditionally, and resolver function that determines the branch to be executed.
type Branches struct {
Branches []Branch `json:"branches"`
Resolver ResolverFn `json:"-"`
}
// example branches
branches := &gosteps.Branches{
Resolver: ResolverFn,
Branches: []gosteps.Branch{
{
BranchName: "branch1",
Steps: gosteps.Steps{
{
Name: "branch1.step1",
Function: StepFn1,
},
},
},
{
BranchName: "branch2",
Steps: gosteps.Steps{
{
Name: "branch2.step1",
Function: StepFn2,
},
},
},
},
}| Field | Description |
|---|---|
| Branches | Branches are a collection of branches that are to be executed conditionally. It is an array of Branch |
| Resolver | Resolver function to determine the branch to be executed. |
ResolverFn
The ResolverFn type is a function that determines the branch to be executed.
// ResolverFn Type
type ResolverFn func(ctx GoStepsCtx) BranchName
// example resolver function
func nextStepResolver(ctx GoStepsCtx) BranchName {
// do something
return "branchName"
}To see all the types available in go-steps see types.go
GoStepsCtx is a custom context type, instance of which is passed between steps to store and share data, progress, state, etc between steps and branches.
type GoStepsCtx struct {
Data GoStepsCtxData `json:"data"`
StepsProgress map[StepName]StepProgress `json:"stepsProgress"`
}
// creating a new context
ctx := gosteps.NewGoStepsContext()| Field | Description |
|---|---|
| Data | Data is a map of key-value pairs that can be used to store and share data between steps and branches |
| StepsProgress | StepsProgress is a map of StepName and StepProgress that stores the progress of each step |
To set data in the context, use the SetData method.
ctx.SetData("key", "value")To get data from the context, use the GetData method.
value := ctx.GetData("key")To set multiple key-value pair in the context, use the WithData method that expects K-V data to be of type map[string]interface{}.
ctx.WithData(map[string]interface{}{
"key1": "value1",
"key2": "value2",
})These functions can be used within Step Functions and Resolver Functions to store and retrieve data and use them in the execution.
To define a step chain, create a Branch with the list of Steps to be run sequentially. Use the gosteps.Steps type or []gosteps.Step type to define the steps and pass it to the NewGoStepsRunner method, which returns the executable step chain instance.
root := gosteps.NewGoStepsRunner(steps)To execute the step chain use the Execute method, passing the context.
ctx := gosteps.NewGoStepsContext()
root.Execute(ctx)You can initialize GoStepsCtx with data using the WithData method, too.
ctx := gosteps.NewGoStepsContext().WithData(map[string]interface{}{
"key1": "value1",
"key2": "value2",
})The step function must return a StepResult type, which contains the status of the step, message returned by the step function, and errors.
type StepResult struct {
StepData GoStepsCtxData `json:"stepData"`
StepState StepState `json:"stepState"`
StepMessage *string `json:"stepMessage"`
StepError error `json:"stepError,omitempty"`
}GoSteps require a step function to return one of the following states.
StepStateComplete StepState = "StepStateComplete" // step completed successfully
StepStateFailed StepState = "StepStateFailed" // step failed to complete, without error
StepStateSkipped StepState = "StepStateSkipped" // step was skipped
StepStatePending StepState = "StepStatePending" // step is pending, should be retried
StepStateError StepState = "StepStateError" // step failed to complete, with errorFunctions are defined to return the state of the step, and the message and error are optional. If the step is failed, the error should be returned.
gosteps.MarkStateComplete()
gosteps.MarkStateFailed()
gosteps.MarkStateSkipped()
gosteps.MarkStatePending()
gosteps.MarkStateError()In addition to the state, the step function can also return a message and error.
// add data to the step result
gosteps.MarkStateComplete().WithData(map[string]interface{}{"key": value})
// add message to the step result
gosteps.MarkStateComplete().WithMessage("message")
// add error to the step result of type StepError
gosteps.MarkStateError().WithError(errors.New("error message"))Some steps-chains might need to branch out conditionally, i.e run a different sets of steps based some condition. To achieve this, define the possible next Branches in the Branches field of the Step of type gosteps.Branches and pass the resolver function to the Resolver field.
stepWithBranches := gosteps.Step{
Name: "stepWithBranches",
Function: Function,
Branches: &gosteps.Branches{
Resolver: func(ctx gosteps.GoStepsCtx) gosteps.BranchName {
// do something
return gosteps.BranchName("branch1")
},
Branches: []gosteps.Branch{
{
BranchName: "branch1",
Steps: gosteps.Steps{ /* steps */ },
},
{
BranchName: "branch2",
Steps: gosteps.Steps{ /* steps */ },
},
},
},
}If a step function is defined for the step, the resolver runs after the function is executed. If no step function is defined the resolver function runs and returns the name of the branch to be executed next.
If the resolver function returns a branch name that is not defined in the branches, the execution will move on with execution of the main branch instead of any conditional branches.
Steps are retired if the StepState is not StepStateComplete or StepStateSkipped.
Steps are retried based for errors that are defined in StepOpts.ErrorsToRetry field. While steps are retried only for the stated errors, they can also be retried for any errors, irrespective of those mentioned in ErrorsToRetry, by setting StepOpts.RetryAllErrors as true.
The retry mechanism runs for StepOpts.MaxRunAttempts, that included the initial run. If a sleep duration is passed in the StepOpts.RetrySleep, then the step runner waits for that duration before executing the next retry of that step.
gosteps.StepOpts{
ErrorsToRetry: []error{errors.New("error")},
ErrorPatternsToRetry: []regexp.Regexp{*regexp.MustCompile("err*")},
RetryAllErrors: false,
MaxRunAttempts: 5,
RetrySleep: 5 * time.Second,
}GoSteps uses the [zerolog](https://github.com/rs/zerolog) package to enable logging within GoSteps. Initialize the logger using the gosteps.NewGoStepsLogger method, passing the output type and options.
// output : of type io.Writer, example os.Stdout, for more options refer to zerolog documentation: https://github.com/rs/zerolog?tab=readme-ov-file#multiple-log-output
output := zerolog.MultiLevelWriter(os.Stdout, runLogFile)
// opts : of type *LoggerOpts, if nil, default options are used to enable step level logging, set StepLoggingEnabled to true
opts := LoggerOpts{
StepLoggingEnabled : true,
}
logger := gosteps.NewGoStepsLogger(output, opts)The logger can be passed to the GoStepsCtx.Use() method to enable logging.
ctx := gosteps.NewGoStepsContext()
ctx.Use(logger)If StepLoggingEnabled is set to true then the step run is also logged with the step name, state, message, and error. In cases where logs are required within the step function, the logger can be accessed using the ctx.Log() method.
func(c gosteps.GoStepsCtx) gosteps.StepResult {
c.Log("this is a message", gosteps.InfoLevel)
return gosteps.MarkStateComplete()
}Sample code can be found in the example directory.