Step Logging and changes to StepError type to errors (#18)

This pull request includes significant updates to the `go-steps` library, focusing on logging enhancements, error handling improvements, and updates to the example usage. The most important changes include the introduction of a logging mechanism using `zerolog`, modifications to error handling structures, and updates to the example code to demonstrate the new features. ### Logging Enhancements: * Introduced a new logging mechanism using the `zerolog` package. The logger can be initialized with `gosteps.NewGoStepsLogger` and used within the context (`go_step_logger.go`). * Added the ability to log messages within step functions using the `ctx.Log()` method (`go_step_context.go`). [[1]](diffhunk://#diff-3bafa3cd28d98fc07e391f9aba1af23ba6420002f0356e2fe1358644db157a4dR15-R43) [[2]](diffhunk://#diff-3bafa3cd28d98fc07e391f9aba1af23ba6420002f0356e2fe1358644db157a4dR52-R63) ### Error Handling Improvements: * Replaced the `StepError` type with the standard `error` type for better compatibility and simplicity (`README.md`, `go_step_result.go`, `go_step_types.go`). [[1]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L267-R267) [[2]](diffhunk://#diff-d76f5092acbb9c53d8513a04815710651343423c14b2b953d107df6e0fe6d942L20-R20) * Updated the `StepResult` struct and related methods to handle standard errors (`go_step_result.go`). ### Example Code Updates: * Enhanced the example in `example/multistep-example/main.go` to demonstrate the new logging and error handling features, including retry logic and logging within steps. [[1]](diffhunk://#diff-b68ca80123c7291a266892f8c0fa6b75d36c4697a12426d696930f66ee96b015R5-R29) [[2]](diffhunk://#diff-b68ca80123c7291a266892f8c0fa6b75d36c4697a12426d696930f66ee96b015L24) [[3]](diffhunk://#diff-b68ca80123c7291a266892f8c0fa6b75d36c4697a12426d696930f66ee96b015R54) [[4]](diffhunk://#diff-b68ca80123c7291a266892f8c0fa6b75d36c4697a12426d696930f66ee96b015R84) [[5]](diffhunk://#diff-b68ca80123c7291a266892f8c0fa6b75d36c4697a12426d696930f66ee96b015R99-R109) [[6]](diffhunk://#diff-b68ca80123c7291a266892f8c0fa6b75d36c4697a12426d696930f66ee96b015R118-R126) [[7]](diffhunk://#diff-b68ca80123c7291a266892f8c0fa6b75d36c4697a12426d696930f66ee96b015L104-R149) ### Documentation Updates: * Updated the `README.md` to reflect the changes in error handling and to include examples of the new logging functionality. [[1]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L41-R78) [[2]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L301-R301) [[3]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L358-R389) ### Miscellaneous: * Updated the `go.mod` file to include the `zerolog` package and other dependencies. ### Issue - Closes #19 - Closes #3
TanmoySG · Nov 17, 2024 · aaa4e62 · aaa4e62
2 parents ecf3bef + 4b7d81f
commit aaa4e62
Show file tree

Hide file tree

Showing 12 changed files with 367 additions and 104 deletions.
diff --git a/.gitignore b/.gitignore
@@ -21,4 +21,6 @@
 go.work
 
 # macOS junk
-.DS_Store
+.DS_Store
+
+*.log
diff --git a/README.md b/README.md
@@ -17,7 +17,7 @@ The idea behind `gosteps` is to define set of functions as chain-of-steps and ex
 
 ### Installation
 
-```bash 
+```bash
 go get github.com/TanmoySG/go-steps
 ```
 
@@ -38,47 +38,47 @@ type Step struct {
   StepOpts        StepOpts               `json:"stepConfig"`
   Branches        *Branches              `json:"branches"`
   StepArgs        map[string]interface{} `json:"stepArgs"`
-  StepResult      *StepResult            `json:"stepResult"`
+}
+
+// example step
+step := gosteps.Step{
+  Name: "add",
+  Function: Add,
+  StepArgs: map[string]interface{}{"n1": 5},
+  StepOpts: gosteps.StepOpts{},
+  Branches: &gosteps.Branches{},
 }
 ```
 
-| Field      | Description                                                                                                |
-|------------|------------------------------------------------------------------------------------------------------------|
-| Name       | Name of step                                                                                               |
-| Function   | The function to execute                                                                                    |
-| StepOpts   | Options/Configurations of the step                                                                         |
-| Branches   | Branches are a sequentially executable collection of  steps.                                               |
-| StepArgs   | Any additional arguments need to pass to  the step.                                                        |
-| StepResult | The results - statues of step, message returned by the step function and, errors are defined in StepResult |
+| Field    | Description                                                                       |
+|----------|-----------------------------------------------------------------------------------|
+| Name     | Name of step                                                                      |
+| Function | The function to execute                                                           |
+| StepOpts | Options/Configurations of the step                                                |
+| Branches | Branches are a sequentially executable collection of  steps.                      |
+| StepArgs | Any additional arguments/variables needed to be passed to the step for execution. |
 
 **StepOpts**
 
 The `StepOpts` type contains the configurations/options for the step execution.
 
 ```go
 type StepOpts struct {
-  ErrorsToRetry  []StepError   `json:"errorsToRetry"`
-  RetryAllErrors bool          `json:"retryAllErrors"`
-  MaxRunAttempts int           `json:"maxAttempts"`
-  RetrySleep     time.Duration `json:"retrySleep"`
-}
-
-// example step
-step := gosteps.Step{
-  Name: "add",
-  Function: Add,
-  StepArgs: map[string]interface{}{"n1": 5},
-  StepOpts: gosteps.StepOpts{},
-  Branches: &gosteps.Branches{},
+  ErrorsToRetry        []error         `json:"errorsToRetry"`
+  ErrorPatternsToRetry []regexp.Regexp `json:"errorPatternsToRetry"`
+  RetryAllErrors       bool            `json:"retryAllErrors"`
+  MaxRunAttempts       int             `json:"maxAttempts"`
+  RetrySleep           time.Duration   `json:"retrySleep"`
 }
 ```
 
-| Field          | Description                                                                                                          |
-|----------------|----------------------------------------------------------------------------------------------------------------------|
-| ErrorsToRetry  | a set of `StepError`s for which a step should be retried.                                                            |
-| RetryAllErrors | A boolean type flag which specifies if a step needs to retry for any error, irrespective of those in `ErrorsToRetry` |
-| MaxRunAttempts | Max attempts are the number of times the step is ran/executed (first run + retries). If not set, it'll run once.     |
-| RetrySleep     | Sleep duration (type time.Duration) between each re-attempts                                                         |
+| Field                | Description                                                                                                          |
+|----------------------|----------------------------------------------------------------------------------------------------------------------|
+| ErrorsToRetry        | a set of errors for which a step should be retried.                                                                  |
+| ErrorPatternsToRetry | a set of `StepErrorPattern`s for which a step should be retried, if error matches pattern                            |
+| RetryAllErrors       | A boolean type flag which specifies if a step needs to retry for any error, irrespective of those in `ErrorsToRetry` |
+| MaxRunAttempts       | Max attempts are the number of times the step is ran/executed (first run + retries). If not set, it'll run once.     |
+| RetrySleep           | Sleep duration (type time.Duration) between each re-attempts                                                         |
 
 **Function**
 
@@ -264,7 +264,7 @@ type StepResult struct {
   StepData    GoStepsCtxData `json:"stepData"`
   StepState   StepState      `json:"stepState"`
   StepMessage *string        `json:"stepMessage"`
-  StepError   *StepError     `json:"stepError,omitempty"`
+  StepError   error     `json:"stepError,omitempty"`
 }
 ```
 
@@ -298,20 +298,7 @@ gosteps.MarkStateComplete().WithData(map[string]interface{}{"key": value})
 gosteps.MarkStateComplete().WithMessage("message")
 
 // add error to the step result of type StepError
-gosteps.MarkStateError().WithError(stepError1)
-
-// wrap any non-GoStep StepErrors and add to the step result
-// it wraps any error passed to it as StepError
-gosteps.MarkStateError().WithWrappedError(fmt.Errorf("error"))
-```
-
-To define a GoStep error, use the `StepError` type.
-
-```go
-stepError1 := gosteps.StepError{
-  StepErrorNameOrId: "error1",
-  StepErrorMessage:  "error message",
-}
+gosteps.MarkStateError().WithError(errors.New("error message"))
 ```
 
 ### Conditional Branching
@@ -355,19 +342,50 @@ The retry mechanism runs for `StepOpts.MaxRunAttempts`, that included the initia
 
 ```go
 gosteps.StepOpts{
-  ErrorsToRetry: []gosteps.StepError{
-    stepError1,
-  },
+  ErrorsToRetry: []error{errors.New("error")},
+  ErrorPatternsToRetry: []regexp.Regexp{*regexp.MustCompile("err*")},
   RetryAllErrors: false,
   MaxRunAttempts: 5,
   RetrySleep: 5 * time.Second,
 }
 ```
 
+### Logging
+
+GoSteps uses the `[zerolog`](<https://github.com/rs/zerolog>) package to enable logging within GoSteps. Initialize the logger using the `gosteps.NewGoStepsLogger` method, passing the output type and options.
+
+```go
+// output : of type io.Writer, example os.Stdout, for more options refer to zerolog documentation: https://github.com/rs/zerolog?tab=readme-ov-file#multiple-log-output
+output := zerolog.MultiLevelWriter(os.Stdout, runLogFile)
+
+// opts   : of type *LoggerOpts, if nil, default options are used to enable step level logging, set StepLoggingEnabled to true
+opts := LoggerOpts{ 
+    StepLoggingEnabled : true,
+}
+
+logger := gosteps.NewGoStepsLogger(output, opts)
+```
+
+The logger can be passed to the `GoStepsCtx.Use()` method to enable logging.
+
+```go
+ctx := gosteps.NewGoStepsContext()
+ctx.Use(logger)
+```
+
+If `StepLoggingEnabled` is set to `true` then the step run is also logged with the step name, state, message, and error. In cases where logs are required within the step function, the logger can be accessed using the `ctx.Log()` method.
+
+```go
+func(c gosteps.GoStepsCtx) gosteps.StepResult {
+    c.Log("this is a message", gosteps.InfoLevel)
+    return gosteps.MarkStateComplete()
+}
+```
+
 ### Example
 
 Sample code can be found in the [example](./example/) directory.
 
-### Help!
+### Help
 
 If you want to help fix the above constraint or other bugs/issues, feel free to raise an Issue or Pull Request with the changes. It'd be an immense help!
diff --git a/example/multistep-example/main.go b/example/multistep-example/main.go
@@ -2,14 +2,31 @@ package main
 
 import (
 	"fmt"
+	"os"
+	"regexp"
+	"time"
 
 	gosteps "github.com/TanmoySG/go-steps"
+	"github.com/rs/zerolog"
 )
 
 func main() {
 
+	count := 0
+
+	runLogFile, _ := os.OpenFile(
+		"myapp.log",
+		os.O_APPEND|os.O_CREATE|os.O_WRONLY,
+		0664,
+	)
+	output := zerolog.MultiLevelWriter(os.Stdout, runLogFile)
+
+	logger := gosteps.NewGoStepsLogger(output, &gosteps.LoggerOpts{StepLoggingEnabled: true})
+
 	ctx := gosteps.NewGoStepsContext()
 
+	ctx.Use(logger)
+
 	multipleDivide := gosteps.Step{
 		Name: "multipleDivide",
 		Function: func(c gosteps.GoStepsCtx) gosteps.StepResult {
@@ -21,7 +38,6 @@ func main() {
 		Branches: &gosteps.Branches{
 			Resolver: func(ctx gosteps.GoStepsCtx) gosteps.BranchName {
 				nx := ctx.GetData("result").(int)
-
 				if nx%2 == 0 {
 					return gosteps.BranchName("divide")
 				}
@@ -35,6 +51,7 @@ func main() {
 							Name: "step3.divide",
 							Function: func(c gosteps.GoStepsCtx) gosteps.StepResult {
 								res := c.GetData("result").(int) / 2
+								c.Log("this is a message", gosteps.LogLevel(zerolog.ErrorLevel))
 								return gosteps.MarkStateComplete().WithData(map[string]interface{}{
 									"result": res,
 								})
@@ -64,6 +81,7 @@ func main() {
 		{
 			Name: "add",
 			Function: func(c gosteps.GoStepsCtx) gosteps.StepResult {
+				c.Log("this is a message")
 
 				res := c.GetData("n1").(int) + c.GetData("n2").(int)
 				return gosteps.MarkStateComplete().WithData(map[string]interface{}{
@@ -78,6 +96,17 @@ func main() {
 		{
 			Name: "subtract",
 			Function: func(c gosteps.GoStepsCtx) gosteps.StepResult {
+				if count < 2 {
+					count++
+					time.Sleep(2 * time.Second)
+					return gosteps.MarkStatePending()
+				}
+
+				if count < 4 {
+					count++
+					return gosteps.MarkStateError().WithError(fmt.Errorf("errpr"))
+				}
+
 				res := c.GetData("n1").(int) - c.GetData("result").(int)
 				return gosteps.MarkStateComplete().WithData(map[string]interface{}{
 					"result": res,
@@ -86,6 +115,15 @@ func main() {
 			StepArgs: map[string]interface{}{
 				"n1": 5,
 			},
+			StepOpts: gosteps.StepOpts{
+				MaxRunAttempts: 5,
+				ErrorPatternsToRetry: []regexp.Regexp{
+					*regexp.MustCompile("err*"),
+				},
+				ErrorsToRetry: []error{
+					fmt.Errorf("errpr"),
+				},
+			},
 		},
 		multipleDivide,
 		{
@@ -101,12 +139,12 @@ func main() {
 		{
 			Name: "print",
 			Function: func(c gosteps.GoStepsCtx) gosteps.StepResult {
-				fmt.Println("result", c.GetData("result"))
+				c.Log(fmt.Sprintf("result %v", c.GetData("result")))
 				return gosteps.MarkStateComplete()
 			},
 		},
 	}
 
-	stepChain := gosteps.NewStepChain(steps)
-	stepChain.Execute(ctx)
+	stepsProcessor := gosteps.NewStepsProcessor(steps)
+	stepsProcessor.Execute(ctx)
 }
diff --git a/go.mod b/go.mod
@@ -2,10 +2,16 @@ module github.com/TanmoySG/go-steps
 
 go 1.18
 
-require github.com/stretchr/testify v1.9.0
+require (
+	github.com/rs/zerolog v1.33.0
+	github.com/stretchr/testify v1.9.0
+)
 
 require (
 	github.com/davecgh/go-spew v1.1.1 // indirect
+	github.com/mattn/go-colorable v0.1.13 // indirect
+	github.com/mattn/go-isatty v0.0.20 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
+	golang.org/x/sys v0.27.0 // indirect
 	gopkg.in/yaml.v3 v3.0.1 // indirect
 )
diff --git a/go.sum b/go.sum
@@ -1,9 +1,27 @@
+github.com/coreos/go-systemd/v22 v22.5.0/go.mod h1:Y58oyj3AT4RCenI/lSvhwexgC+NSVTIJ3seZv2GcEnc=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/godbus/dbus/v5 v5.0.4/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
+github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
+github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
+github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
+github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
+github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
+github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
+github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/rs/xid v1.5.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
+github.com/rs/zerolog v1.33.0 h1:1cU2KZkvPxNyfgEmhHAz/1A9Bz+llsdYzklWFzgp0r8=
+github.com/rs/zerolog v1.33.0/go.mod h1:/7mN4D5sKwJLZQ2b/znpjC3/GQWY/xaDXUM0kKWRHss=
 github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
 github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
+golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.27.0 h1:wBqf8DvsY9Y/2P8gAfPDEYNuS30J4lPHJxXSb/nJZ+s=
+golang.org/x/sys v0.27.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=